May 26 15:17:59.900606 ip-10-0-139-198 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory May 26 15:17:59.900619 ip-10-0-139-198 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory May 26 15:17:59.900630 ip-10-0-139-198 systemd[1]: kubelet.service: Failed with result 'resources'. May 26 15:17:59.900953 ip-10-0-139-198 systemd[1]: Failed to start Kubernetes Kubelet. May 26 15:18:10.070130 ip-10-0-139-198 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. May 26 15:18:10.070147 ip-10-0-139-198 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 9f98d71c887e4ea6b32f6b442aacc71c -- May 26 15:20:51.132439 ip-10-0-139-198 systemd[1]: Starting Kubernetes Kubelet... May 26 15:20:51.661898 ip-10-0-139-198 kubenswrapper[2547]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 26 15:20:51.661898 ip-10-0-139-198 kubenswrapper[2547]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. May 26 15:20:51.661898 ip-10-0-139-198 kubenswrapper[2547]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 26 15:20:51.661898 ip-10-0-139-198 kubenswrapper[2547]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. May 26 15:20:51.661898 ip-10-0-139-198 kubenswrapper[2547]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 26 15:20:51.663943 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.663851 2547 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" May 26 15:20:51.666983 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.666967 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 26 15:20:51.666983 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.666983 2547 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 26 15:20:51.667071 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.666987 2547 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 26 15:20:51.667071 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.666990 2547 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 26 15:20:51.667071 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.666994 2547 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 26 15:20:51.667071 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.666998 2547 feature_gate.go:328] unrecognized feature gate: OVNObservability May 26 15:20:51.667071 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667002 2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 26 15:20:51.667071 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667005 2547 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 26 15:20:51.667071 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667008 2547 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 26 15:20:51.667071 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667011 2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 26 15:20:51.667071 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667013 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 26 15:20:51.667071 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667016 2547 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 26 15:20:51.667071 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667019 2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 26 15:20:51.667071 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667026 2547 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 26 15:20:51.667071 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667029 2547 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 26 15:20:51.667071 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667032 2547 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 26 15:20:51.667071 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667035 2547 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 26 15:20:51.667071 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667037 2547 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 26 15:20:51.667071 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667040 2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 26 15:20:51.667071 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667042 2547 feature_gate.go:328] unrecognized feature gate: SignatureStores May 26 15:20:51.667071 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667045 2547 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 26 15:20:51.667539 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667048 2547 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 26 15:20:51.667539 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667051 2547 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 26 15:20:51.667539 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667066 2547 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 26 15:20:51.667539 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667069 2547 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 26 15:20:51.667539 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667071 2547 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 26 15:20:51.667539 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667074 2547 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 26 15:20:51.667539 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667077 2547 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 26 15:20:51.667539 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667080 2547 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 26 15:20:51.667539 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667083 2547 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 26 15:20:51.667539 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667086 2547 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 26 15:20:51.667539 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667089 2547 feature_gate.go:328] unrecognized feature gate: NewOLM May 26 15:20:51.667539 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667092 2547 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 26 15:20:51.667539 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667094 2547 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 26 15:20:51.667539 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667097 2547 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 26 15:20:51.667539 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667099 2547 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 26 15:20:51.667539 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667102 2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 26 15:20:51.667539 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667104 2547 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 26 15:20:51.667539 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667107 2547 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 26 15:20:51.667539 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667109 2547 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 26 15:20:51.667539 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667112 2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 26 15:20:51.668023 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667115 2547 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 26 15:20:51.668023 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667117 2547 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 26 15:20:51.668023 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667120 2547 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 26 15:20:51.668023 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667122 2547 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 26 15:20:51.668023 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667125 2547 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 26 15:20:51.668023 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667127 2547 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 26 15:20:51.668023 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667130 2547 feature_gate.go:328] unrecognized feature gate: DualReplica May 26 15:20:51.668023 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667132 2547 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 26 15:20:51.668023 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667135 2547 feature_gate.go:328] unrecognized feature gate: Example2 May 26 15:20:51.668023 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667137 2547 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 26 15:20:51.668023 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667140 2547 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 26 15:20:51.668023 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667142 2547 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 26 15:20:51.668023 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667145 2547 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 26 15:20:51.668023 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667148 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 26 15:20:51.668023 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667153 2547 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 26 15:20:51.668023 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667157 2547 feature_gate.go:328] unrecognized feature gate: Example May 26 15:20:51.668023 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667159 2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 26 15:20:51.668023 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667162 2547 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 26 15:20:51.668023 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667165 2547 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 26 15:20:51.668023 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667168 2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 26 15:20:51.668535 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667171 2547 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 26 15:20:51.668535 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667173 2547 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 26 15:20:51.668535 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667176 2547 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 26 15:20:51.668535 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667179 2547 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 26 15:20:51.668535 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667181 2547 feature_gate.go:328] unrecognized feature gate: PinnedImages May 26 15:20:51.668535 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667187 2547 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 26 15:20:51.668535 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667190 2547 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 26 15:20:51.668535 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667192 2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 26 15:20:51.668535 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667195 2547 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 26 15:20:51.668535 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667197 2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 26 15:20:51.668535 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667201 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 26 15:20:51.668535 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667204 2547 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 26 15:20:51.668535 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667206 2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 26 15:20:51.668535 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667209 2547 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 26 15:20:51.668535 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667211 2547 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 26 15:20:51.668535 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667214 2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 26 15:20:51.668535 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667216 2547 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 26 15:20:51.668535 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667219 2547 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 26 15:20:51.668535 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667221 2547 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 26 15:20:51.668535 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667224 2547 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 26 15:20:51.669015 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667226 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 26 15:20:51.669015 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667229 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 26 15:20:51.669015 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667231 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 26 15:20:51.669015 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667233 2547 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 26 15:20:51.669015 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.667236 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 26 15:20:51.669015 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668430 2547 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 26 15:20:51.669015 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668436 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 26 15:20:51.669015 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668440 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 26 15:20:51.669015 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668443 2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 26 15:20:51.669015 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668447 2547 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 26 15:20:51.669015 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668450 2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 26 15:20:51.669015 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668453 2547 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 26 15:20:51.669015 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668456 2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 26 15:20:51.669015 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668459 2547 feature_gate.go:328] unrecognized feature gate: Example2 May 26 15:20:51.669015 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668461 2547 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 26 15:20:51.669015 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668465 2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 26 15:20:51.669015 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668468 2547 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 26 15:20:51.669015 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668472 2547 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 26 15:20:51.669015 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668475 2547 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 26 15:20:51.669015 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668478 2547 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 26 15:20:51.669513 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668480 2547 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 26 15:20:51.669513 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668483 2547 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 26 15:20:51.669513 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668486 2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 26 15:20:51.669513 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668488 2547 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 26 15:20:51.669513 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668491 2547 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 26 15:20:51.669513 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668493 2547 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 26 15:20:51.669513 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668497 2547 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 26 15:20:51.669513 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668499 2547 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 26 15:20:51.669513 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668502 2547 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 26 15:20:51.669513 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668504 2547 feature_gate.go:328] unrecognized feature gate: SignatureStores May 26 15:20:51.669513 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668507 2547 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 26 15:20:51.669513 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668509 2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 26 15:20:51.669513 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668512 2547 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 26 15:20:51.669513 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668514 2547 feature_gate.go:328] unrecognized feature gate: NewOLM May 26 15:20:51.669513 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668518 2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 26 15:20:51.669513 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668521 2547 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 26 15:20:51.669513 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668524 2547 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 26 15:20:51.669513 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668527 2547 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 26 15:20:51.669513 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668530 2547 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 26 15:20:51.669513 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668532 2547 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 26 15:20:51.670047 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668535 2547 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 26 15:20:51.670047 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668538 2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 26 15:20:51.670047 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668540 2547 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 26 15:20:51.670047 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668543 2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 26 15:20:51.670047 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668545 2547 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 26 15:20:51.670047 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668548 2547 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 26 15:20:51.670047 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668551 2547 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 26 15:20:51.670047 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668553 2547 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 26 15:20:51.670047 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668556 2547 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 26 15:20:51.670047 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668558 2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 26 15:20:51.670047 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668561 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 26 15:20:51.670047 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668564 2547 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 26 15:20:51.670047 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668566 2547 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 26 15:20:51.670047 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668568 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 26 15:20:51.670047 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668571 2547 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 26 15:20:51.670047 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668573 2547 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 26 15:20:51.670047 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668576 2547 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 26 15:20:51.670047 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668578 2547 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 26 15:20:51.670047 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668581 2547 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 26 15:20:51.670047 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668583 2547 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 26 15:20:51.670569 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668586 2547 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 26 15:20:51.670569 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668588 2547 feature_gate.go:328] unrecognized feature gate: PinnedImages May 26 15:20:51.670569 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668591 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 26 15:20:51.670569 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668593 2547 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 26 15:20:51.670569 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668595 2547 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 26 15:20:51.670569 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668598 2547 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 26 15:20:51.670569 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668601 2547 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 26 15:20:51.670569 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668606 2547 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 26 15:20:51.670569 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668609 2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 26 15:20:51.670569 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668612 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 26 15:20:51.670569 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668614 2547 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 26 15:20:51.670569 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668617 2547 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 26 15:20:51.670569 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668619 2547 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 26 15:20:51.670569 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668623 2547 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 26 15:20:51.670569 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668626 2547 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 26 15:20:51.670569 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668629 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 26 15:20:51.670569 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668632 2547 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 26 15:20:51.670569 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668635 2547 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 26 15:20:51.670569 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668637 2547 feature_gate.go:328] unrecognized feature gate: Example May 26 15:20:51.671038 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668639 2547 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 26 15:20:51.671038 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668642 2547 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 26 15:20:51.671038 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668644 2547 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 26 15:20:51.671038 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668647 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 26 15:20:51.671038 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668650 2547 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 26 15:20:51.671038 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668652 2547 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 26 15:20:51.671038 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668655 2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 26 15:20:51.671038 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668657 2547 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 26 15:20:51.671038 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668660 2547 feature_gate.go:328] unrecognized feature gate: DualReplica May 26 15:20:51.671038 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668662 2547 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 26 15:20:51.671038 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668665 2547 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 26 15:20:51.671038 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.668667 2547 feature_gate.go:328] unrecognized feature gate: OVNObservability May 26 15:20:51.671038 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668749 2547 flags.go:64] FLAG: --address="0.0.0.0" May 26 15:20:51.671038 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668764 2547 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" May 26 15:20:51.671038 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668770 2547 flags.go:64] FLAG: --anonymous-auth="true" May 26 15:20:51.671038 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668774 2547 flags.go:64] FLAG: --application-metrics-count-limit="100" May 26 15:20:51.671038 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668778 2547 flags.go:64] FLAG: --authentication-token-webhook="false" May 26 15:20:51.671038 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668781 2547 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" May 26 15:20:51.671038 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668785 2547 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" May 26 15:20:51.671038 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668790 2547 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" May 26 15:20:51.671038 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668794 2547 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" May 26 15:20:51.671582 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668798 2547 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" May 26 15:20:51.671582 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668802 2547 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" May 26 15:20:51.671582 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668805 2547 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" May 26 15:20:51.671582 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668808 2547 flags.go:64] FLAG: --cgroup-driver="cgroupfs" May 26 15:20:51.671582 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668811 2547 flags.go:64] FLAG: --cgroup-root="" May 26 15:20:51.671582 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668814 2547 flags.go:64] FLAG: --cgroups-per-qos="true" May 26 15:20:51.671582 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668817 2547 flags.go:64] FLAG: --client-ca-file="" May 26 15:20:51.671582 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668820 2547 flags.go:64] FLAG: --cloud-config="" May 26 15:20:51.671582 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668823 2547 flags.go:64] FLAG: --cloud-provider="external" May 26 15:20:51.671582 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668826 2547 flags.go:64] FLAG: --cluster-dns="[]" May 26 15:20:51.671582 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668830 2547 flags.go:64] FLAG: --cluster-domain="" May 26 15:20:51.671582 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668833 2547 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" May 26 15:20:51.671582 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668836 2547 flags.go:64] FLAG: --config-dir="" May 26 15:20:51.671582 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668839 2547 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" May 26 15:20:51.671582 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668842 2547 flags.go:64] FLAG: --container-log-max-files="5" May 26 15:20:51.671582 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668848 2547 flags.go:64] FLAG: --container-log-max-size="10Mi" May 26 15:20:51.671582 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668852 2547 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" May 26 15:20:51.671582 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668855 2547 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" May 26 15:20:51.671582 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668858 2547 flags.go:64] FLAG: --containerd-namespace="k8s.io" May 26 15:20:51.671582 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668861 2547 flags.go:64] FLAG: --contention-profiling="false" May 26 15:20:51.671582 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668864 2547 flags.go:64] FLAG: --cpu-cfs-quota="true" May 26 15:20:51.671582 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668867 2547 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" May 26 15:20:51.671582 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668870 2547 flags.go:64] FLAG: --cpu-manager-policy="none" May 26 15:20:51.671582 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668873 2547 flags.go:64] FLAG: --cpu-manager-policy-options="" May 26 15:20:51.671582 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668877 2547 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" May 26 15:20:51.672224 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668880 2547 flags.go:64] FLAG: --enable-controller-attach-detach="true" May 26 15:20:51.672224 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668883 2547 flags.go:64] FLAG: --enable-debugging-handlers="true" May 26 15:20:51.672224 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668886 2547 flags.go:64] FLAG: --enable-load-reader="false" May 26 15:20:51.672224 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668889 2547 flags.go:64] FLAG: --enable-server="true" May 26 15:20:51.672224 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668892 2547 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" May 26 15:20:51.672224 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668897 2547 flags.go:64] FLAG: --event-burst="100" May 26 15:20:51.672224 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668900 2547 flags.go:64] FLAG: --event-qps="50" May 26 15:20:51.672224 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668903 2547 flags.go:64] FLAG: --event-storage-age-limit="default=0" May 26 15:20:51.672224 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668906 2547 flags.go:64] FLAG: --event-storage-event-limit="default=0" May 26 15:20:51.672224 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668909 2547 flags.go:64] FLAG: --eviction-hard="" May 26 15:20:51.672224 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668913 2547 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" May 26 15:20:51.672224 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668916 2547 flags.go:64] FLAG: --eviction-minimum-reclaim="" May 26 15:20:51.672224 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668919 2547 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" May 26 15:20:51.672224 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668921 2547 flags.go:64] FLAG: --eviction-soft="" May 26 15:20:51.672224 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668924 2547 flags.go:64] FLAG: --eviction-soft-grace-period="" May 26 15:20:51.672224 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668927 2547 flags.go:64] FLAG: --exit-on-lock-contention="false" May 26 15:20:51.672224 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668930 2547 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" May 26 15:20:51.672224 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668933 2547 flags.go:64] FLAG: --experimental-mounter-path="" May 26 15:20:51.672224 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668936 2547 flags.go:64] FLAG: --fail-cgroupv1="false" May 26 15:20:51.672224 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668939 2547 flags.go:64] FLAG: --fail-swap-on="true" May 26 15:20:51.672224 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668942 2547 flags.go:64] FLAG: --feature-gates="" May 26 15:20:51.672224 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668945 2547 flags.go:64] FLAG: --file-check-frequency="20s" May 26 15:20:51.672224 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668948 2547 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" May 26 15:20:51.672224 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668951 2547 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" May 26 15:20:51.672224 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668956 2547 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" May 26 15:20:51.672839 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668959 2547 flags.go:64] FLAG: --healthz-port="10248" May 26 15:20:51.672839 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668962 2547 flags.go:64] FLAG: --help="false" May 26 15:20:51.672839 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668964 2547 flags.go:64] FLAG: --hostname-override="ip-10-0-139-198.ec2.internal" May 26 15:20:51.672839 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668980 2547 flags.go:64] FLAG: --housekeeping-interval="10s" May 26 15:20:51.672839 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668984 2547 flags.go:64] FLAG: --http-check-frequency="20s" May 26 15:20:51.672839 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668988 2547 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" May 26 15:20:51.672839 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668992 2547 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" May 26 15:20:51.672839 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668995 2547 flags.go:64] FLAG: --image-gc-high-threshold="85" May 26 15:20:51.672839 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.668998 2547 flags.go:64] FLAG: --image-gc-low-threshold="80" May 26 15:20:51.672839 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669001 2547 flags.go:64] FLAG: --image-service-endpoint="" May 26 15:20:51.672839 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669004 2547 flags.go:64] FLAG: --kernel-memcg-notification="false" May 26 15:20:51.672839 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669007 2547 flags.go:64] FLAG: --kube-api-burst="100" May 26 15:20:51.672839 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669011 2547 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" May 26 15:20:51.672839 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669014 2547 flags.go:64] FLAG: --kube-api-qps="50" May 26 15:20:51.672839 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669017 2547 flags.go:64] FLAG: --kube-reserved="" May 26 15:20:51.672839 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669020 2547 flags.go:64] FLAG: --kube-reserved-cgroup="" May 26 15:20:51.672839 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669023 2547 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" May 26 15:20:51.672839 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669026 2547 flags.go:64] FLAG: --kubelet-cgroups="" May 26 15:20:51.672839 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669029 2547 flags.go:64] FLAG: --local-storage-capacity-isolation="true" May 26 15:20:51.672839 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669032 2547 flags.go:64] FLAG: --lock-file="" May 26 15:20:51.672839 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669035 2547 flags.go:64] FLAG: --log-cadvisor-usage="false" May 26 15:20:51.672839 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669038 2547 flags.go:64] FLAG: --log-flush-frequency="5s" May 26 15:20:51.672839 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669041 2547 flags.go:64] FLAG: --log-json-info-buffer-size="0" May 26 15:20:51.672839 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669046 2547 flags.go:64] FLAG: --log-json-split-stream="false" May 26 15:20:51.673445 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669049 2547 flags.go:64] FLAG: --log-text-info-buffer-size="0" May 26 15:20:51.673445 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669064 2547 flags.go:64] FLAG: --log-text-split-stream="false" May 26 15:20:51.673445 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669067 2547 flags.go:64] FLAG: --logging-format="text" May 26 15:20:51.673445 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669070 2547 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" May 26 15:20:51.673445 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669074 2547 flags.go:64] FLAG: --make-iptables-util-chains="true" May 26 15:20:51.673445 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669077 2547 flags.go:64] FLAG: --manifest-url="" May 26 15:20:51.673445 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669079 2547 flags.go:64] FLAG: --manifest-url-header="" May 26 15:20:51.673445 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669087 2547 flags.go:64] FLAG: --max-housekeeping-interval="15s" May 26 15:20:51.673445 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669097 2547 flags.go:64] FLAG: --max-open-files="1000000" May 26 15:20:51.673445 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669105 2547 flags.go:64] FLAG: --max-pods="110" May 26 15:20:51.673445 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669108 2547 flags.go:64] FLAG: --maximum-dead-containers="-1" May 26 15:20:51.673445 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669112 2547 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" May 26 15:20:51.673445 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669115 2547 flags.go:64] FLAG: --memory-manager-policy="None" May 26 15:20:51.673445 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669118 2547 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" May 26 15:20:51.673445 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669121 2547 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" May 26 15:20:51.673445 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669124 2547 flags.go:64] FLAG: --node-ip="0.0.0.0" May 26 15:20:51.673445 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669126 2547 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" May 26 15:20:51.673445 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669133 2547 flags.go:64] FLAG: --node-status-max-images="50" May 26 15:20:51.673445 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669137 2547 flags.go:64] FLAG: --node-status-update-frequency="10s" May 26 15:20:51.673445 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669140 2547 flags.go:64] FLAG: --oom-score-adj="-999" May 26 15:20:51.673445 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669143 2547 flags.go:64] FLAG: --pod-cidr="" May 26 15:20:51.673445 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669146 2547 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afe88671bf163c42b467d159dc6f0b784dbe28b1ed44e171edc9c06fd99cc6fc" May 26 15:20:51.673445 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669152 2547 flags.go:64] FLAG: --pod-manifest-path="" May 26 15:20:51.674013 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669155 2547 flags.go:64] FLAG: --pod-max-pids="-1" May 26 15:20:51.674013 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669158 2547 flags.go:64] FLAG: --pods-per-core="0" May 26 15:20:51.674013 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669161 2547 flags.go:64] FLAG: --port="10250" May 26 15:20:51.674013 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669164 2547 flags.go:64] FLAG: --protect-kernel-defaults="false" May 26 15:20:51.674013 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669167 2547 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-07d3c6d0a7a790937" May 26 15:20:51.674013 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669170 2547 flags.go:64] FLAG: --qos-reserved="" May 26 15:20:51.674013 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669173 2547 flags.go:64] FLAG: --read-only-port="10255" May 26 15:20:51.674013 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669176 2547 flags.go:64] FLAG: --register-node="true" May 26 15:20:51.674013 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669179 2547 flags.go:64] FLAG: --register-schedulable="true" May 26 15:20:51.674013 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669182 2547 flags.go:64] FLAG: --register-with-taints="" May 26 15:20:51.674013 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669185 2547 flags.go:64] FLAG: --registry-burst="10" May 26 15:20:51.674013 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669188 2547 flags.go:64] FLAG: --registry-qps="5" May 26 15:20:51.674013 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669191 2547 flags.go:64] FLAG: --reserved-cpus="" May 26 15:20:51.674013 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669194 2547 flags.go:64] FLAG: --reserved-memory="" May 26 15:20:51.674013 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669198 2547 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" May 26 15:20:51.674013 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669201 2547 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" May 26 15:20:51.674013 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669204 2547 flags.go:64] FLAG: --rotate-certificates="false" May 26 15:20:51.674013 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669208 2547 flags.go:64] FLAG: --rotate-server-certificates="false" May 26 15:20:51.674013 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669211 2547 flags.go:64] FLAG: --runonce="false" May 26 15:20:51.674013 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669214 2547 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" May 26 15:20:51.674013 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669217 2547 flags.go:64] FLAG: --runtime-request-timeout="2m0s" May 26 15:20:51.674013 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669220 2547 flags.go:64] FLAG: --seccomp-default="false" May 26 15:20:51.674013 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669223 2547 flags.go:64] FLAG: --serialize-image-pulls="true" May 26 15:20:51.674013 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669226 2547 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" May 26 15:20:51.674013 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669229 2547 flags.go:64] FLAG: --storage-driver-db="cadvisor" May 26 15:20:51.674013 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669232 2547 flags.go:64] FLAG: --storage-driver-host="localhost:8086" May 26 15:20:51.674688 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669235 2547 flags.go:64] FLAG: --storage-driver-password="root" May 26 15:20:51.674688 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669238 2547 flags.go:64] FLAG: --storage-driver-secure="false" May 26 15:20:51.674688 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669241 2547 flags.go:64] FLAG: --storage-driver-table="stats" May 26 15:20:51.674688 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669244 2547 flags.go:64] FLAG: --storage-driver-user="root" May 26 15:20:51.674688 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669246 2547 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" May 26 15:20:51.674688 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669250 2547 flags.go:64] FLAG: --sync-frequency="1m0s" May 26 15:20:51.674688 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669253 2547 flags.go:64] FLAG: --system-cgroups="" May 26 15:20:51.674688 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669256 2547 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" May 26 15:20:51.674688 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669261 2547 flags.go:64] FLAG: --system-reserved-cgroup="" May 26 15:20:51.674688 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669263 2547 flags.go:64] FLAG: --tls-cert-file="" May 26 15:20:51.674688 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669266 2547 flags.go:64] FLAG: --tls-cipher-suites="[]" May 26 15:20:51.674688 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669271 2547 flags.go:64] FLAG: --tls-min-version="" May 26 15:20:51.674688 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669273 2547 flags.go:64] FLAG: --tls-private-key-file="" May 26 15:20:51.674688 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669276 2547 flags.go:64] FLAG: --topology-manager-policy="none" May 26 15:20:51.674688 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669279 2547 flags.go:64] FLAG: --topology-manager-policy-options="" May 26 15:20:51.674688 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669282 2547 flags.go:64] FLAG: --topology-manager-scope="container" May 26 15:20:51.674688 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669285 2547 flags.go:64] FLAG: --v="2" May 26 15:20:51.674688 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669289 2547 flags.go:64] FLAG: --version="false" May 26 15:20:51.674688 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669293 2547 flags.go:64] FLAG: --vmodule="" May 26 15:20:51.674688 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669297 2547 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" May 26 15:20:51.674688 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.669300 2547 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" May 26 15:20:51.674688 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669407 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 26 15:20:51.674688 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669411 2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 26 15:20:51.674688 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669416 2547 feature_gate.go:328] unrecognized feature gate: PinnedImages May 26 15:20:51.675304 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669419 2547 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 26 15:20:51.675304 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669425 2547 feature_gate.go:328] unrecognized feature gate: OVNObservability May 26 15:20:51.675304 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669428 2547 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 26 15:20:51.675304 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669430 2547 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 26 15:20:51.675304 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669433 2547 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 26 15:20:51.675304 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669435 2547 feature_gate.go:328] unrecognized feature gate: SignatureStores May 26 15:20:51.675304 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669438 2547 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 26 15:20:51.675304 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669440 2547 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 26 15:20:51.675304 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669443 2547 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 26 15:20:51.675304 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669446 2547 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 26 15:20:51.675304 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669449 2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 26 15:20:51.675304 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669451 2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 26 15:20:51.675304 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669454 2547 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 26 15:20:51.675304 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669457 2547 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 26 15:20:51.675304 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669461 2547 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 26 15:20:51.675304 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669464 2547 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 26 15:20:51.675304 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669467 2547 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 26 15:20:51.675304 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669470 2547 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 26 15:20:51.675304 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669472 2547 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 26 15:20:51.675815 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669475 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 26 15:20:51.675815 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669477 2547 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 26 15:20:51.675815 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669480 2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 26 15:20:51.675815 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669482 2547 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 26 15:20:51.675815 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669485 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 26 15:20:51.675815 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669487 2547 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 26 15:20:51.675815 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669490 2547 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 26 15:20:51.675815 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669492 2547 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 26 15:20:51.675815 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669495 2547 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 26 15:20:51.675815 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669497 2547 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 26 15:20:51.675815 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669500 2547 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 26 15:20:51.675815 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669504 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 26 15:20:51.675815 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669508 2547 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 26 15:20:51.675815 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669511 2547 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 26 15:20:51.675815 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669513 2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 26 15:20:51.675815 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669523 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 26 15:20:51.675815 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669526 2547 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 26 15:20:51.675815 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669529 2547 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 26 15:20:51.675815 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669531 2547 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 26 15:20:51.675815 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669534 2547 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 26 15:20:51.676407 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669537 2547 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 26 15:20:51.676407 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669539 2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 26 15:20:51.676407 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669542 2547 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 26 15:20:51.676407 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669544 2547 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 26 15:20:51.676407 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669547 2547 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 26 15:20:51.676407 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669550 2547 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 26 15:20:51.676407 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669552 2547 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 26 15:20:51.676407 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669555 2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 26 15:20:51.676407 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669557 2547 feature_gate.go:328] unrecognized feature gate: NewOLM May 26 15:20:51.676407 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669560 2547 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 26 15:20:51.676407 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669563 2547 feature_gate.go:328] unrecognized feature gate: Example2 May 26 15:20:51.676407 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669565 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 26 15:20:51.676407 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669568 2547 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 26 15:20:51.676407 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669570 2547 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 26 15:20:51.676407 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669573 2547 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 26 15:20:51.676407 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669575 2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 26 15:20:51.676407 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669578 2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 26 15:20:51.676407 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669580 2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 26 15:20:51.676407 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669583 2547 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 26 15:20:51.676407 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669587 2547 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 26 15:20:51.676962 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669590 2547 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 26 15:20:51.676962 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669593 2547 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 26 15:20:51.676962 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669596 2547 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 26 15:20:51.676962 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669600 2547 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 26 15:20:51.676962 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669603 2547 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 26 15:20:51.676962 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669606 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 26 15:20:51.676962 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669609 2547 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 26 15:20:51.676962 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669611 2547 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 26 15:20:51.676962 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669619 2547 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 26 15:20:51.676962 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669622 2547 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 26 15:20:51.676962 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669624 2547 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 26 15:20:51.676962 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669627 2547 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 26 15:20:51.676962 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669630 2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 26 15:20:51.676962 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669632 2547 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 26 15:20:51.676962 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669634 2547 feature_gate.go:328] unrecognized feature gate: Example May 26 15:20:51.676962 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669637 2547 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 26 15:20:51.676962 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669639 2547 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 26 15:20:51.676962 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669642 2547 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 26 15:20:51.676962 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669644 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 26 15:20:51.676962 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669647 2547 feature_gate.go:328] unrecognized feature gate: DualReplica May 26 15:20:51.677544 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669649 2547 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 26 15:20:51.677544 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669652 2547 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 26 15:20:51.677544 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669654 2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 26 15:20:51.677544 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.669656 2547 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 26 15:20:51.677544 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.670513 2547 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 26 15:20:51.679555 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.679534 2547 server.go:530] "Kubelet version" kubeletVersion="v1.33.11" May 26 15:20:51.679599 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.679556 2547 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" May 26 15:20:51.679636 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679611 2547 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 26 15:20:51.679636 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679616 2547 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 26 15:20:51.679636 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679620 2547 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 26 15:20:51.679636 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679623 2547 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 26 15:20:51.679636 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679626 2547 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 26 15:20:51.679636 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679629 2547 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 26 15:20:51.679636 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679633 2547 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 26 15:20:51.679636 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679636 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 26 15:20:51.679636 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679639 2547 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 26 15:20:51.679636 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679641 2547 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 26 15:20:51.679931 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679645 2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 26 15:20:51.679931 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679647 2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 26 15:20:51.679931 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679650 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 26 15:20:51.679931 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679652 2547 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 26 15:20:51.679931 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679655 2547 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 26 15:20:51.679931 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679658 2547 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 26 15:20:51.679931 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679661 2547 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 26 15:20:51.679931 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679663 2547 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 26 15:20:51.679931 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679666 2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 26 15:20:51.679931 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679668 2547 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 26 15:20:51.679931 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679671 2547 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 26 15:20:51.679931 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679674 2547 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 26 15:20:51.679931 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679676 2547 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 26 15:20:51.679931 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679679 2547 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 26 15:20:51.679931 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679682 2547 feature_gate.go:328] unrecognized feature gate: Example2 May 26 15:20:51.679931 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679684 2547 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 26 15:20:51.679931 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679687 2547 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 26 15:20:51.679931 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679689 2547 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 26 15:20:51.679931 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679692 2547 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 26 15:20:51.679931 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679694 2547 feature_gate.go:328] unrecognized feature gate: DualReplica May 26 15:20:51.680577 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679697 2547 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 26 15:20:51.680577 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679706 2547 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 26 15:20:51.680577 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679708 2547 feature_gate.go:328] unrecognized feature gate: NewOLM May 26 15:20:51.680577 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679711 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 26 15:20:51.680577 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679715 2547 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 26 15:20:51.680577 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679719 2547 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 26 15:20:51.680577 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679722 2547 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 26 15:20:51.680577 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679725 2547 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 26 15:20:51.680577 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679727 2547 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 26 15:20:51.680577 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679730 2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 26 15:20:51.680577 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679732 2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 26 15:20:51.680577 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679735 2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 26 15:20:51.680577 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679738 2547 feature_gate.go:328] unrecognized feature gate: Example May 26 15:20:51.680577 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679741 2547 feature_gate.go:328] unrecognized feature gate: OVNObservability May 26 15:20:51.680577 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679744 2547 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 26 15:20:51.680577 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679747 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 26 15:20:51.680577 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679749 2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 26 15:20:51.680577 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679752 2547 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 26 15:20:51.680577 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679754 2547 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 26 15:20:51.681048 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679757 2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 26 15:20:51.681048 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679759 2547 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 26 15:20:51.681048 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679762 2547 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 26 15:20:51.681048 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679765 2547 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 26 15:20:51.681048 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679767 2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 26 15:20:51.681048 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679770 2547 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 26 15:20:51.681048 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679772 2547 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 26 15:20:51.681048 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679775 2547 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 26 15:20:51.681048 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679778 2547 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 26 15:20:51.681048 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679780 2547 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 26 15:20:51.681048 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679783 2547 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 26 15:20:51.681048 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679785 2547 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 26 15:20:51.681048 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679788 2547 feature_gate.go:328] unrecognized feature gate: SignatureStores May 26 15:20:51.681048 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679790 2547 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 26 15:20:51.681048 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679793 2547 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 26 15:20:51.681048 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679803 2547 feature_gate.go:328] unrecognized feature gate: PinnedImages May 26 15:20:51.681048 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679805 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 26 15:20:51.681048 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679808 2547 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 26 15:20:51.681048 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679810 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 26 15:20:51.681048 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679813 2547 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 26 15:20:51.681554 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679816 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 26 15:20:51.681554 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679818 2547 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 26 15:20:51.681554 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679821 2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 26 15:20:51.681554 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679823 2547 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 26 15:20:51.681554 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679827 2547 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 26 15:20:51.681554 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679831 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 26 15:20:51.681554 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679834 2547 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 26 15:20:51.681554 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679837 2547 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 26 15:20:51.681554 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679840 2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 26 15:20:51.681554 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679842 2547 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 26 15:20:51.681554 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679845 2547 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 26 15:20:51.681554 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679848 2547 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 26 15:20:51.681554 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679850 2547 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 26 15:20:51.681554 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679853 2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 26 15:20:51.681554 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679855 2547 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 26 15:20:51.681554 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679858 2547 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 26 15:20:51.681554 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679861 2547 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 26 15:20:51.681967 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.679866 2547 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 26 15:20:51.681967 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.679996 2547 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 26 15:20:51.681967 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680002 2547 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 26 15:20:51.681967 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680005 2547 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 26 15:20:51.681967 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680008 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 26 15:20:51.681967 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680011 2547 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 26 15:20:51.681967 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680014 2547 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 26 15:20:51.681967 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680017 2547 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 26 15:20:51.681967 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680020 2547 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 26 15:20:51.681967 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680022 2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 26 15:20:51.681967 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680025 2547 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 26 15:20:51.681967 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680033 2547 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 26 15:20:51.681967 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680036 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 26 15:20:51.681967 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680038 2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 26 15:20:51.681967 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680041 2547 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 26 15:20:51.681967 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680043 2547 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 26 15:20:51.682389 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680046 2547 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 26 15:20:51.682389 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680048 2547 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 26 15:20:51.682389 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680051 2547 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 26 15:20:51.682389 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680066 2547 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 26 15:20:51.682389 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680069 2547 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 26 15:20:51.682389 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680072 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 26 15:20:51.682389 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680075 2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 26 15:20:51.682389 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680078 2547 feature_gate.go:328] unrecognized feature gate: NewOLM May 26 15:20:51.682389 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680081 2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 26 15:20:51.682389 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680083 2547 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 26 15:20:51.682389 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680086 2547 feature_gate.go:328] unrecognized feature gate: Example2 May 26 15:20:51.682389 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680088 2547 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 26 15:20:51.682389 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680091 2547 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 26 15:20:51.682389 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680093 2547 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 26 15:20:51.682389 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680096 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 26 15:20:51.682389 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680098 2547 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 26 15:20:51.682389 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680101 2547 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 26 15:20:51.682389 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680104 2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 26 15:20:51.682389 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680106 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 26 15:20:51.682389 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680109 2547 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 26 15:20:51.682889 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680111 2547 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 26 15:20:51.682889 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680114 2547 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 26 15:20:51.682889 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680116 2547 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 26 15:20:51.682889 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680119 2547 feature_gate.go:328] unrecognized feature gate: DualReplica May 26 15:20:51.682889 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680121 2547 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 26 15:20:51.682889 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680124 2547 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 26 15:20:51.682889 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680126 2547 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 26 15:20:51.682889 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680129 2547 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 26 15:20:51.682889 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680137 2547 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 26 15:20:51.682889 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680140 2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 26 15:20:51.682889 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680143 2547 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 26 15:20:51.682889 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680145 2547 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 26 15:20:51.682889 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680148 2547 feature_gate.go:328] unrecognized feature gate: OVNObservability May 26 15:20:51.682889 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680151 2547 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 26 15:20:51.682889 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680153 2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 26 15:20:51.682889 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680155 2547 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 26 15:20:51.682889 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680158 2547 feature_gate.go:328] unrecognized feature gate: SignatureStores May 26 15:20:51.682889 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680161 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 26 15:20:51.682889 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680164 2547 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 26 15:20:51.682889 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680167 2547 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 26 15:20:51.683387 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680169 2547 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 26 15:20:51.683387 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680172 2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 26 15:20:51.683387 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680174 2547 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 26 15:20:51.683387 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680177 2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 26 15:20:51.683387 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680179 2547 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 26 15:20:51.683387 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680182 2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 26 15:20:51.683387 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680185 2547 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 26 15:20:51.683387 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680187 2547 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 26 15:20:51.683387 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680190 2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 26 15:20:51.683387 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680192 2547 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 26 15:20:51.683387 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680195 2547 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 26 15:20:51.683387 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680197 2547 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 26 15:20:51.683387 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680199 2547 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 26 15:20:51.683387 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680202 2547 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 26 15:20:51.683387 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680212 2547 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 26 15:20:51.683387 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680215 2547 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 26 15:20:51.683387 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680218 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 26 15:20:51.683387 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680220 2547 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 26 15:20:51.683387 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680223 2547 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 26 15:20:51.683387 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680226 2547 feature_gate.go:328] unrecognized feature gate: Example May 26 15:20:51.683906 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680228 2547 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 26 15:20:51.683906 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680236 2547 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 26 15:20:51.683906 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680238 2547 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 26 15:20:51.683906 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680241 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 26 15:20:51.683906 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680244 2547 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 26 15:20:51.683906 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680247 2547 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 26 15:20:51.683906 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680251 2547 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 26 15:20:51.683906 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680254 2547 feature_gate.go:328] unrecognized feature gate: PinnedImages May 26 15:20:51.683906 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680256 2547 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 26 15:20:51.683906 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680261 2547 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 26 15:20:51.683906 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:51.680264 2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 26 15:20:51.683906 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.680269 2547 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 26 15:20:51.683906 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.680393 2547 server.go:962] "Client rotation is on, will bootstrap in background" May 26 15:20:51.683906 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.682914 2547 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" May 26 15:20:51.684276 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.684032 2547 server.go:1019] "Starting client certificate rotation" May 26 15:20:51.684276 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.684196 2547 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" May 26 15:20:51.684276 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.684231 2547 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" May 26 15:20:51.711871 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.711850 2547 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" May 26 15:20:51.715151 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.715130 2547 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" May 26 15:20:51.737397 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.737375 2547 log.go:25] "Validated CRI v1 runtime API" May 26 15:20:51.743445 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.743427 2547 log.go:25] "Validated CRI v1 image API" May 26 15:20:51.743703 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.743684 2547 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" May 26 15:20:51.744964 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.744942 2547 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" May 26 15:20:51.749422 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.749402 2547 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9898dee5-d78b-4c98-8e3c-be9202282387:/dev/nvme0n1p4 ed85f936-64f7-4f64-a703-f410957222a6:/dev/nvme0n1p3] May 26 15:20:51.749481 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.749421 2547 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] May 26 15:20:51.754074 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.753957 2547 manager.go:217] Machine: {Timestamp:2026-05-26 15:20:51.75346237 +0000 UTC m=+0.480137283 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097664 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2702da699d6f2e993c9b00def823ee SystemUUID:ec2702da-699d-6f2e-993c-9b00def823ee BootID:9f98d71c-887e-4ea6-b32f-6b442aacc71c Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c1:c6:06:cf:f1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c1:c6:06:cf:f1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:56:11:d4:1a:38:46 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} May 26 15:20:51.755651 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.755639 2547 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. May 26 15:20:51.755738 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.755725 2547 manager.go:233] Version: {KernelVersion:5.14.0-570.114.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260512-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} May 26 15:20:51.757906 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.757886 2547 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] May 26 15:20:51.758079 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.757908 2547 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-198.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} May 26 15:20:51.758129 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.758089 2547 topology_manager.go:138] "Creating topology manager with none policy" May 26 15:20:51.758129 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.758098 2547 container_manager_linux.go:306] "Creating device plugin manager" May 26 15:20:51.758129 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.758106 2547 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" May 26 15:20:51.758989 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.758977 2547 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" May 26 15:20:51.761576 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.761565 2547 state_mem.go:36] "Initialized new in-memory state store" May 26 15:20:51.761696 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.761687 2547 server.go:1267] "Using root directory" path="/var/lib/kubelet" May 26 15:20:51.764383 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.764374 2547 kubelet.go:491] "Attempting to sync node with API server" May 26 15:20:51.764415 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.764387 2547 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" May 26 15:20:51.764415 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.764400 2547 file.go:69] "Watching path" path="/etc/kubernetes/manifests" May 26 15:20:51.764415 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.764409 2547 kubelet.go:397] "Adding apiserver pod source" May 26 15:20:51.764492 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.764417 2547 apiserver.go:42] "Waiting for node sync before watching apiserver pods" May 26 15:20:51.765021 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.764996 2547 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5skr2" May 26 15:20:51.765706 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.765695 2547 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" May 26 15:20:51.765738 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.765714 2547 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" May 26 15:20:51.769488 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.769468 2547 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.11-2.rhaos4.20.gitb2a8320.el9" apiVersion="v1" May 26 15:20:51.772111 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.772094 2547 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" May 26 15:20:51.772996 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.772984 2547 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" May 26 15:20:51.773039 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.773001 2547 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" May 26 15:20:51.773039 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.773007 2547 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" May 26 15:20:51.773039 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.773013 2547 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" May 26 15:20:51.773039 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.773021 2547 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" May 26 15:20:51.773039 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.773026 2547 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" May 26 15:20:51.773039 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.773032 2547 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" May 26 15:20:51.773039 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.773037 2547 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" May 26 15:20:51.773260 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.773044 2547 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" May 26 15:20:51.773260 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.773050 2547 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" May 26 15:20:51.773260 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.773079 2547 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" May 26 15:20:51.773551 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.773542 2547 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" May 26 15:20:51.774658 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.774643 2547 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" May 26 15:20:51.774658 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.774656 2547 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" May 26 15:20:51.775145 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.775131 2547 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5skr2" May 26 15:20:51.776339 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:51.776317 2547 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" May 26 15:20:51.776447 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:51.776424 2547 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-198.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" May 26 15:20:51.777929 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.777916 2547 watchdog_linux.go:99] "Systemd watchdog is not enabled" May 26 15:20:51.778007 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.777950 2547 server.go:1295] "Started kubelet" May 26 15:20:51.778045 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.778013 2547 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 May 26 15:20:51.778119 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.778026 2547 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 May 26 15:20:51.778168 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.778141 2547 server_v1.go:47] "podresources" method="list" useActivePods=true May 26 15:20:51.778797 ip-10-0-139-198 systemd[1]: Started Kubernetes Kubelet. May 26 15:20:51.778921 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.778816 2547 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-198.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope May 26 15:20:51.780289 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.780273 2547 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" May 26 15:20:51.780354 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.780296 2547 server.go:317] "Adding debug handlers to kubelet server" May 26 15:20:51.787321 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.787302 2547 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" May 26 15:20:51.787616 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:51.787598 2547 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" May 26 15:20:51.787970 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.787957 2547 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" May 26 15:20:51.788593 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.788451 2547 desired_state_of_world_populator.go:150] "Desired state populator starts to run" May 26 15:20:51.788875 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.788645 2547 volume_manager.go:295] "The desired_state_of_world populator starts" May 26 15:20:51.788875 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.788657 2547 volume_manager.go:297] "Starting Kubelet Volume Manager" May 26 15:20:51.788875 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.788740 2547 reconstruct.go:97] "Volume reconstruction finished" May 26 15:20:51.788875 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.788751 2547 reconciler.go:26] "Reconciler: start to sync state" May 26 15:20:51.789175 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:51.789147 2547 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-198.ec2.internal\" not found" May 26 15:20:51.789421 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.789407 2547 factory.go:55] Registering systemd factory May 26 15:20:51.789483 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.789426 2547 factory.go:223] Registration of the systemd container factory successfully May 26 15:20:51.789652 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.789634 2547 factory.go:153] Registering CRI-O factory May 26 15:20:51.789652 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.789650 2547 factory.go:223] Registration of the crio container factory successfully May 26 15:20:51.789788 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.789694 2547 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory May 26 15:20:51.789788 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.789709 2547 factory.go:103] Registering Raw factory May 26 15:20:51.789788 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.789719 2547 manager.go:1196] Started watching for new ooms in manager May 26 15:20:51.790274 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.790256 2547 manager.go:319] Starting recovery of all containers May 26 15:20:51.791434 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.791413 2547 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" May 26 15:20:51.801318 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:51.801172 2547 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-139-198.ec2.internal\" not found" node="ip-10-0-139-198.ec2.internal" May 26 15:20:51.802260 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.802244 2547 manager.go:324] Recovery completed May 26 15:20:51.806492 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.806479 2547 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 26 15:20:51.808188 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.808170 2547 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-198.ec2.internal" event="NodeHasSufficientMemory" May 26 15:20:51.808270 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.808207 2547 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-198.ec2.internal" event="NodeHasNoDiskPressure" May 26 15:20:51.808270 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.808221 2547 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-198.ec2.internal" event="NodeHasSufficientPID" May 26 15:20:51.808757 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.808743 2547 cpu_manager.go:222] "Starting CPU manager" policy="none" May 26 15:20:51.808757 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.808754 2547 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" May 26 15:20:51.808840 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.808771 2547 state_mem.go:36] "Initialized new in-memory state store" May 26 15:20:51.812635 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.812622 2547 policy_none.go:49] "None policy: Start" May 26 15:20:51.812674 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.812639 2547 memory_manager.go:186] "Starting memorymanager" policy="None" May 26 15:20:51.812674 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.812650 2547 state_mem.go:35] "Initializing new in-memory state store" May 26 15:20:51.862418 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.851581 2547 manager.go:341] "Starting Device Plugin manager" May 26 15:20:51.862418 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:51.851617 2547 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" May 26 15:20:51.862418 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.851632 2547 server.go:85] "Starting device plugin registration server" May 26 15:20:51.862418 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.851900 2547 eviction_manager.go:189] "Eviction manager: starting control loop" May 26 15:20:51.862418 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.851913 2547 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" May 26 15:20:51.862418 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.852079 2547 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" May 26 15:20:51.862418 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.852163 2547 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" May 26 15:20:51.862418 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.852173 2547 plugin_manager.go:118] "Starting Kubelet Plugin Manager" May 26 15:20:51.862418 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:51.852668 2547 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" May 26 15:20:51.862418 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:51.852718 2547 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-198.ec2.internal\" not found" May 26 15:20:51.907533 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.907495 2547 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" May 26 15:20:51.908695 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.908679 2547 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" May 26 15:20:51.908759 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.908712 2547 status_manager.go:230] "Starting to sync pod status with apiserver" May 26 15:20:51.908799 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.908758 2547 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." May 26 15:20:51.908799 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.908766 2547 kubelet.go:2451] "Starting kubelet main sync loop" May 26 15:20:51.908885 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:51.908813 2547 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" May 26 15:20:51.911847 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.911829 2547 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" May 26 15:20:51.952359 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.952316 2547 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 26 15:20:51.953256 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.953240 2547 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-198.ec2.internal" event="NodeHasSufficientMemory" May 26 15:20:51.953352 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.953269 2547 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-198.ec2.internal" event="NodeHasNoDiskPressure" May 26 15:20:51.953352 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.953281 2547 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-198.ec2.internal" event="NodeHasSufficientPID" May 26 15:20:51.953352 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.953313 2547 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-198.ec2.internal" May 26 15:20:51.965115 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:51.965097 2547 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-198.ec2.internal" May 26 15:20:51.965183 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:51.965123 2547 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-198.ec2.internal\": node \"ip-10-0-139-198.ec2.internal\" not found" May 26 15:20:51.981597 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:51.981578 2547 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-198.ec2.internal\" not found" May 26 15:20:52.009410 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.009391 2547 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-198.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-198.ec2.internal"] May 26 15:20:52.009472 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.009462 2547 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 26 15:20:52.010305 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.010284 2547 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-198.ec2.internal" event="NodeHasSufficientMemory" May 26 15:20:52.010371 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.010317 2547 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-198.ec2.internal" event="NodeHasNoDiskPressure" May 26 15:20:52.010371 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.010328 2547 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-198.ec2.internal" event="NodeHasSufficientPID" May 26 15:20:52.011616 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.011604 2547 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 26 15:20:52.011754 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.011742 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-198.ec2.internal" May 26 15:20:52.011801 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.011772 2547 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 26 15:20:52.012314 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.012298 2547 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-198.ec2.internal" event="NodeHasSufficientMemory" May 26 15:20:52.012364 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.012328 2547 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-198.ec2.internal" event="NodeHasNoDiskPressure" May 26 15:20:52.012364 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.012340 2547 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-198.ec2.internal" event="NodeHasSufficientPID" May 26 15:20:52.012431 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.012301 2547 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-198.ec2.internal" event="NodeHasSufficientMemory" May 26 15:20:52.012431 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.012398 2547 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-198.ec2.internal" event="NodeHasNoDiskPressure" May 26 15:20:52.012431 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.012408 2547 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-198.ec2.internal" event="NodeHasSufficientPID" May 26 15:20:52.013559 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.013546 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-198.ec2.internal" May 26 15:20:52.013615 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.013568 2547 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 26 15:20:52.014165 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.014145 2547 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-198.ec2.internal" event="NodeHasSufficientMemory" May 26 15:20:52.014238 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.014174 2547 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-198.ec2.internal" event="NodeHasNoDiskPressure" May 26 15:20:52.014238 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.014185 2547 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-198.ec2.internal" event="NodeHasSufficientPID" May 26 15:20:52.044602 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:52.044578 2547 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-198.ec2.internal\" not found" node="ip-10-0-139-198.ec2.internal" May 26 15:20:52.048802 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:52.048786 2547 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-198.ec2.internal\" not found" node="ip-10-0-139-198.ec2.internal" May 26 15:20:52.082387 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:52.082369 2547 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-198.ec2.internal\" not found" May 26 15:20:52.091220 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.091205 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0640ae7f05c27f5c076d0840b04ed3a3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-198.ec2.internal\" (UID: \"0640ae7f05c27f5c076d0840b04ed3a3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-198.ec2.internal" May 26 15:20:52.091276 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.091228 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/10db4583199c612d03dabeed2e12e039-config\") pod \"kube-apiserver-proxy-ip-10-0-139-198.ec2.internal\" (UID: \"10db4583199c612d03dabeed2e12e039\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-198.ec2.internal" May 26 15:20:52.091276 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.091245 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0640ae7f05c27f5c076d0840b04ed3a3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-198.ec2.internal\" (UID: \"0640ae7f05c27f5c076d0840b04ed3a3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-198.ec2.internal" May 26 15:20:52.183334 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:52.183312 2547 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-198.ec2.internal\" not found" May 26 15:20:52.191737 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.191717 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0640ae7f05c27f5c076d0840b04ed3a3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-198.ec2.internal\" (UID: \"0640ae7f05c27f5c076d0840b04ed3a3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-198.ec2.internal" May 26 15:20:52.191785 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.191760 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0640ae7f05c27f5c076d0840b04ed3a3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-198.ec2.internal\" (UID: \"0640ae7f05c27f5c076d0840b04ed3a3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-198.ec2.internal" May 26 15:20:52.191819 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.191789 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/10db4583199c612d03dabeed2e12e039-config\") pod \"kube-apiserver-proxy-ip-10-0-139-198.ec2.internal\" (UID: \"10db4583199c612d03dabeed2e12e039\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-198.ec2.internal" May 26 15:20:52.191819 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.191726 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0640ae7f05c27f5c076d0840b04ed3a3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-198.ec2.internal\" (UID: \"0640ae7f05c27f5c076d0840b04ed3a3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-198.ec2.internal" May 26 15:20:52.191877 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.191818 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/10db4583199c612d03dabeed2e12e039-config\") pod \"kube-apiserver-proxy-ip-10-0-139-198.ec2.internal\" (UID: \"10db4583199c612d03dabeed2e12e039\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-198.ec2.internal" May 26 15:20:52.191877 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.191837 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0640ae7f05c27f5c076d0840b04ed3a3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-198.ec2.internal\" (UID: \"0640ae7f05c27f5c076d0840b04ed3a3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-198.ec2.internal" May 26 15:20:52.284135 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:52.284077 2547 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-198.ec2.internal\" not found" May 26 15:20:52.347506 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.347486 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-198.ec2.internal" May 26 15:20:52.352153 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.352131 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-198.ec2.internal" May 26 15:20:52.384942 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:52.384918 2547 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-198.ec2.internal\" not found" May 26 15:20:52.485428 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:52.485399 2547 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-198.ec2.internal\" not found" May 26 15:20:52.585913 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:52.585846 2547 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-198.ec2.internal\" not found" May 26 15:20:52.683302 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.683258 2547 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" May 26 15:20:52.683934 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.683466 2547 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" May 26 15:20:52.683934 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.683484 2547 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" May 26 15:20:52.686414 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:52.686395 2547 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-198.ec2.internal\" not found" May 26 15:20:52.756699 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.756677 2547 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" May 26 15:20:52.765335 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.765317 2547 apiserver.go:52] "Watching apiserver" May 26 15:20:52.773817 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.773796 2547 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" May 26 15:20:52.774119 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.774099 2547 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-knssg","kube-system/konnectivity-agent-qcrp6","openshift-cluster-node-tuning-operator/tuned-spgwf","openshift-dns/node-resolver-ddnhg","openshift-image-registry/node-ca-2r56q","openshift-multus/network-metrics-daemon-5xh82","openshift-network-diagnostics/network-check-target-kkq9m","openshift-network-operator/iptables-alerter-6txjd","openshift-ovn-kubernetes/ovnkube-node-vfdvw","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrrv8","openshift-multus/multus-66nvf"] May 26 15:20:52.777088 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.777043 2547 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-25 15:15:51 +0000 UTC" deadline="2027-11-16 06:16:40.705498811 +0000 UTC" May 26 15:20:52.777088 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.777087 2547 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12926h55m47.928414956s" May 26 15:20:52.780088 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.780071 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-knssg" May 26 15:20:52.780270 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.780253 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qcrp6" May 26 15:20:52.782004 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.781988 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" May 26 15:20:52.782265 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.782250 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" May 26 15:20:52.782332 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.782256 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" May 26 15:20:52.782948 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.782902 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" May 26 15:20:52.783166 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.783142 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-gksf6\"" May 26 15:20:52.783260 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.783169 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-c624h\"" May 26 15:20:52.783260 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.783215 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" May 26 15:20:52.783684 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.783540 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" May 26 15:20:52.783862 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.783840 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" May 26 15:20:52.784026 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.784005 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2r56q" May 26 15:20:52.786012 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.785982 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" May 26 15:20:52.786120 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.786099 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" May 26 15:20:52.786214 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.786198 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rjffv\"" May 26 15:20:52.786268 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.786238 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" May 26 15:20:52.787393 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.787376 2547 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" May 26 15:20:52.788396 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.788382 2547 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-198.ec2.internal" May 26 15:20:52.788542 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.788522 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xh82" May 26 15:20:52.788639 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:52.788613 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xh82" podUID="fac6c416-4fef-4b1b-846f-ea63a14c9efe" May 26 15:20:52.790677 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.790658 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.790760 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.790745 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ddnhg" May 26 15:20:52.792526 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.792505 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" May 26 15:20:52.792639 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.792505 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" May 26 15:20:52.792639 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.792579 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-mlg7m\"" May 26 15:20:52.792739 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.792643 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" May 26 15:20:52.792739 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.792668 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wdfdj\"" May 26 15:20:52.792902 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.792888 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" May 26 15:20:52.792980 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.792965 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkq9m" May 26 15:20:52.793032 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:52.793015 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkq9m" podUID="033a1d5a-e393-4e8c-aa09-3786637f5a7e" May 26 15:20:52.795020 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.795002 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jndc7\" (UniqueName: \"kubernetes.io/projected/72e9400c-a618-45b9-8b31-3fa6b713a9a2-kube-api-access-jndc7\") pod \"tuned-spgwf\" (UID: \"72e9400c-a618-45b9-8b31-3fa6b713a9a2\") " pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.795128 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.795025 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a1723cfd-d190-4d26-9964-3c378ba0cd3b-hosts-file\") pod \"node-resolver-ddnhg\" (UID: \"a1723cfd-d190-4d26-9964-3c378ba0cd3b\") " pod="openshift-dns/node-resolver-ddnhg" May 26 15:20:52.795128 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.795040 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5mhj\" (UniqueName: \"kubernetes.io/projected/c4d73d69-aaad-48c5-bfbc-75ac14352253-kube-api-access-z5mhj\") pod \"multus-additional-cni-plugins-knssg\" (UID: \"c4d73d69-aaad-48c5-bfbc-75ac14352253\") " pod="openshift-multus/multus-additional-cni-plugins-knssg" May 26 15:20:52.795128 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.795078 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/eda06c33-9985-46af-ac9d-d006488f30ca-konnectivity-ca\") pod \"konnectivity-agent-qcrp6\" (UID: \"eda06c33-9985-46af-ac9d-d006488f30ca\") " pod="kube-system/konnectivity-agent-qcrp6" May 26 15:20:52.795286 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.795139 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/72e9400c-a618-45b9-8b31-3fa6b713a9a2-etc-sysctl-conf\") pod \"tuned-spgwf\" (UID: \"72e9400c-a618-45b9-8b31-3fa6b713a9a2\") " pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.795286 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.795170 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4f6155d9-d73c-483b-ade7-8eecd0e99883-host\") pod \"node-ca-2r56q\" (UID: \"4f6155d9-d73c-483b-ade7-8eecd0e99883\") " pod="openshift-image-registry/node-ca-2r56q" May 26 15:20:52.795286 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.795195 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vq2r\" (UniqueName: \"kubernetes.io/projected/fac6c416-4fef-4b1b-846f-ea63a14c9efe-kube-api-access-7vq2r\") pod \"network-metrics-daemon-5xh82\" (UID: \"fac6c416-4fef-4b1b-846f-ea63a14c9efe\") " pod="openshift-multus/network-metrics-daemon-5xh82" May 26 15:20:52.795286 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.795232 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/72e9400c-a618-45b9-8b31-3fa6b713a9a2-etc-systemd\") pod \"tuned-spgwf\" (UID: \"72e9400c-a618-45b9-8b31-3fa6b713a9a2\") " pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.795286 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.795254 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/72e9400c-a618-45b9-8b31-3fa6b713a9a2-lib-modules\") pod \"tuned-spgwf\" (UID: \"72e9400c-a618-45b9-8b31-3fa6b713a9a2\") " pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.795286 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.795277 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4f6155d9-d73c-483b-ade7-8eecd0e99883-serviceca\") pod \"node-ca-2r56q\" (UID: \"4f6155d9-d73c-483b-ade7-8eecd0e99883\") " pod="openshift-image-registry/node-ca-2r56q" May 26 15:20:52.795557 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.795300 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k255d\" (UniqueName: \"kubernetes.io/projected/4f6155d9-d73c-483b-ade7-8eecd0e99883-kube-api-access-k255d\") pod \"node-ca-2r56q\" (UID: \"4f6155d9-d73c-483b-ade7-8eecd0e99883\") " pod="openshift-image-registry/node-ca-2r56q" May 26 15:20:52.795557 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.795336 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rpdx\" (UniqueName: \"kubernetes.io/projected/a1723cfd-d190-4d26-9964-3c378ba0cd3b-kube-api-access-9rpdx\") pod \"node-resolver-ddnhg\" (UID: \"a1723cfd-d190-4d26-9964-3c378ba0cd3b\") " pod="openshift-dns/node-resolver-ddnhg" May 26 15:20:52.795557 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.795361 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c4d73d69-aaad-48c5-bfbc-75ac14352253-system-cni-dir\") pod \"multus-additional-cni-plugins-knssg\" (UID: \"c4d73d69-aaad-48c5-bfbc-75ac14352253\") " pod="openshift-multus/multus-additional-cni-plugins-knssg" May 26 15:20:52.795557 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.795375 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c4d73d69-aaad-48c5-bfbc-75ac14352253-tuning-conf-dir\") pod \"multus-additional-cni-plugins-knssg\" (UID: \"c4d73d69-aaad-48c5-bfbc-75ac14352253\") " pod="openshift-multus/multus-additional-cni-plugins-knssg" May 26 15:20:52.795557 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.795412 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72e9400c-a618-45b9-8b31-3fa6b713a9a2-etc-kubernetes\") pod \"tuned-spgwf\" (UID: \"72e9400c-a618-45b9-8b31-3fa6b713a9a2\") " pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.795557 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.795435 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/72e9400c-a618-45b9-8b31-3fa6b713a9a2-etc-tuned\") pod \"tuned-spgwf\" (UID: \"72e9400c-a618-45b9-8b31-3fa6b713a9a2\") " pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.795557 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.795457 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/72e9400c-a618-45b9-8b31-3fa6b713a9a2-tmp\") pod \"tuned-spgwf\" (UID: \"72e9400c-a618-45b9-8b31-3fa6b713a9a2\") " pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.795557 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.795496 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/72e9400c-a618-45b9-8b31-3fa6b713a9a2-run\") pod \"tuned-spgwf\" (UID: \"72e9400c-a618-45b9-8b31-3fa6b713a9a2\") " pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.795557 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.795523 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/72e9400c-a618-45b9-8b31-3fa6b713a9a2-sys\") pod \"tuned-spgwf\" (UID: \"72e9400c-a618-45b9-8b31-3fa6b713a9a2\") " pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.795557 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.795525 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6txjd" May 26 15:20:52.795557 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.795547 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/eda06c33-9985-46af-ac9d-d006488f30ca-agent-certs\") pod \"konnectivity-agent-qcrp6\" (UID: \"eda06c33-9985-46af-ac9d-d006488f30ca\") " pod="kube-system/konnectivity-agent-qcrp6" May 26 15:20:52.796041 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.795575 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c4d73d69-aaad-48c5-bfbc-75ac14352253-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-knssg\" (UID: \"c4d73d69-aaad-48c5-bfbc-75ac14352253\") " pod="openshift-multus/multus-additional-cni-plugins-knssg" May 26 15:20:52.796041 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.795604 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c4d73d69-aaad-48c5-bfbc-75ac14352253-cni-binary-copy\") pod \"multus-additional-cni-plugins-knssg\" (UID: \"c4d73d69-aaad-48c5-bfbc-75ac14352253\") " pod="openshift-multus/multus-additional-cni-plugins-knssg" May 26 15:20:52.796041 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.795642 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c4d73d69-aaad-48c5-bfbc-75ac14352253-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-knssg\" (UID: \"c4d73d69-aaad-48c5-bfbc-75ac14352253\") " pod="openshift-multus/multus-additional-cni-plugins-knssg" May 26 15:20:52.796041 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.795690 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c4d73d69-aaad-48c5-bfbc-75ac14352253-os-release\") pod \"multus-additional-cni-plugins-knssg\" (UID: \"c4d73d69-aaad-48c5-bfbc-75ac14352253\") " pod="openshift-multus/multus-additional-cni-plugins-knssg" May 26 15:20:52.796041 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.795718 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/72e9400c-a618-45b9-8b31-3fa6b713a9a2-etc-modprobe-d\") pod \"tuned-spgwf\" (UID: \"72e9400c-a618-45b9-8b31-3fa6b713a9a2\") " pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.796041 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.795751 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/72e9400c-a618-45b9-8b31-3fa6b713a9a2-etc-sysconfig\") pod \"tuned-spgwf\" (UID: \"72e9400c-a618-45b9-8b31-3fa6b713a9a2\") " pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.796041 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.795773 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/72e9400c-a618-45b9-8b31-3fa6b713a9a2-etc-sysctl-d\") pod \"tuned-spgwf\" (UID: \"72e9400c-a618-45b9-8b31-3fa6b713a9a2\") " pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.796041 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.795794 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/72e9400c-a618-45b9-8b31-3fa6b713a9a2-var-lib-kubelet\") pod \"tuned-spgwf\" (UID: \"72e9400c-a618-45b9-8b31-3fa6b713a9a2\") " pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.796041 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.795817 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/72e9400c-a618-45b9-8b31-3fa6b713a9a2-host\") pod \"tuned-spgwf\" (UID: \"72e9400c-a618-45b9-8b31-3fa6b713a9a2\") " pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.796041 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.795872 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1723cfd-d190-4d26-9964-3c378ba0cd3b-tmp-dir\") pod \"node-resolver-ddnhg\" (UID: \"a1723cfd-d190-4d26-9964-3c378ba0cd3b\") " pod="openshift-dns/node-resolver-ddnhg" May 26 15:20:52.796041 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.795899 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c4d73d69-aaad-48c5-bfbc-75ac14352253-cnibin\") pod \"multus-additional-cni-plugins-knssg\" (UID: \"c4d73d69-aaad-48c5-bfbc-75ac14352253\") " pod="openshift-multus/multus-additional-cni-plugins-knssg" May 26 15:20:52.796041 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.795934 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fac6c416-4fef-4b1b-846f-ea63a14c9efe-metrics-certs\") pod \"network-metrics-daemon-5xh82\" (UID: \"fac6c416-4fef-4b1b-846f-ea63a14c9efe\") " pod="openshift-multus/network-metrics-daemon-5xh82" May 26 15:20:52.797539 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.797519 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-z82q5\"" May 26 15:20:52.797633 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.797519 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" May 26 15:20:52.797839 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.797801 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" May 26 15:20:52.797839 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.797808 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" May 26 15:20:52.798072 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.798040 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:52.798553 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.798517 2547 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" May 26 15:20:52.798651 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.798581 2547 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-198.ec2.internal" May 26 15:20:52.799877 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.799818 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" May 26 15:20:52.800120 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.800077 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" May 26 15:20:52.800253 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.800237 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" May 26 15:20:52.800319 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.800273 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" May 26 15:20:52.800319 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.800291 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-zkdbs\"" May 26 15:20:52.800422 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.800331 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" May 26 15:20:52.800422 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.800219 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" May 26 15:20:52.800550 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.800480 2547 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" May 26 15:20:52.802249 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.802232 2547 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-139-198.ec2.internal"] May 26 15:20:52.802322 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.802307 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrrv8" May 26 15:20:52.802402 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.802309 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-66nvf" May 26 15:20:52.804154 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.804136 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" May 26 15:20:52.804350 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.804337 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" May 26 15:20:52.804591 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.804577 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-fxxlr\"" May 26 15:20:52.804897 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.804882 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" May 26 15:20:52.804973 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.804896 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" May 26 15:20:52.804973 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.804932 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-5lnzm\"" May 26 15:20:52.808513 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.808438 2547 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" May 26 15:20:52.808578 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.808541 2547 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-198.ec2.internal"] May 26 15:20:52.823102 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.823086 2547 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qdqzx" May 26 15:20:52.836069 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.836008 2547 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qdqzx" May 26 15:20:52.889911 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.889889 2547 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" May 26 15:20:52.896308 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.896282 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-host-run-netns\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:52.896391 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.896324 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2275m\" (UniqueName: \"kubernetes.io/projected/f5579674-2eaf-44d9-b407-585c871463c7-kube-api-access-2275m\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:52.896391 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.896360 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c4d73d69-aaad-48c5-bfbc-75ac14352253-cni-binary-copy\") pod \"multus-additional-cni-plugins-knssg\" (UID: \"c4d73d69-aaad-48c5-bfbc-75ac14352253\") " pod="openshift-multus/multus-additional-cni-plugins-knssg" May 26 15:20:52.896473 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.896422 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-host-cni-bin\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:52.896473 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.896450 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:52.896543 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.896484 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f5579674-2eaf-44d9-b407-585c871463c7-os-release\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:52.896543 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.896516 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f5579674-2eaf-44d9-b407-585c871463c7-multus-socket-dir-parent\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:52.896621 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.896541 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-run-openvswitch\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:52.896621 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.896574 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c4d73d69-aaad-48c5-bfbc-75ac14352253-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-knssg\" (UID: \"c4d73d69-aaad-48c5-bfbc-75ac14352253\") " pod="openshift-multus/multus-additional-cni-plugins-knssg" May 26 15:20:52.896621 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.896601 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/72e9400c-a618-45b9-8b31-3fa6b713a9a2-etc-modprobe-d\") pod \"tuned-spgwf\" (UID: \"72e9400c-a618-45b9-8b31-3fa6b713a9a2\") " pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.896742 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.896629 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/72e9400c-a618-45b9-8b31-3fa6b713a9a2-etc-sysctl-d\") pod \"tuned-spgwf\" (UID: \"72e9400c-a618-45b9-8b31-3fa6b713a9a2\") " pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.896742 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.896657 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/91d86e11-9ca2-4467-8929-484f81370ffe-sys-fs\") pod \"aws-ebs-csi-driver-node-mrrv8\" (UID: \"91d86e11-9ca2-4467-8929-484f81370ffe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrrv8" May 26 15:20:52.896742 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.896706 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f5579674-2eaf-44d9-b407-585c871463c7-host-run-netns\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:52.896742 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.896733 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f5579674-2eaf-44d9-b407-585c871463c7-multus-conf-dir\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:52.897095 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.896754 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/72e9400c-a618-45b9-8b31-3fa6b713a9a2-etc-modprobe-d\") pod \"tuned-spgwf\" (UID: \"72e9400c-a618-45b9-8b31-3fa6b713a9a2\") " pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.897095 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.896760 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a402afb6-d0bc-4139-8153-4e1463fb49cc-iptables-alerter-script\") pod \"iptables-alerter-6txjd\" (UID: \"a402afb6-d0bc-4139-8153-4e1463fb49cc\") " pod="openshift-network-operator/iptables-alerter-6txjd" May 26 15:20:52.897095 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.896793 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/72e9400c-a618-45b9-8b31-3fa6b713a9a2-etc-sysctl-d\") pod \"tuned-spgwf\" (UID: \"72e9400c-a618-45b9-8b31-3fa6b713a9a2\") " pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.897095 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.896806 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1723cfd-d190-4d26-9964-3c378ba0cd3b-tmp-dir\") pod \"node-resolver-ddnhg\" (UID: \"a1723cfd-d190-4d26-9964-3c378ba0cd3b\") " pod="openshift-dns/node-resolver-ddnhg" May 26 15:20:52.897095 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.896849 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c4d73d69-aaad-48c5-bfbc-75ac14352253-cni-binary-copy\") pod \"multus-additional-cni-plugins-knssg\" (UID: \"c4d73d69-aaad-48c5-bfbc-75ac14352253\") " pod="openshift-multus/multus-additional-cni-plugins-knssg" May 26 15:20:52.897095 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.896858 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c4d73d69-aaad-48c5-bfbc-75ac14352253-cnibin\") pod \"multus-additional-cni-plugins-knssg\" (UID: \"c4d73d69-aaad-48c5-bfbc-75ac14352253\") " pod="openshift-multus/multus-additional-cni-plugins-knssg" May 26 15:20:52.897095 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.896889 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jndc7\" (UniqueName: \"kubernetes.io/projected/72e9400c-a618-45b9-8b31-3fa6b713a9a2-kube-api-access-jndc7\") pod \"tuned-spgwf\" (UID: \"72e9400c-a618-45b9-8b31-3fa6b713a9a2\") " pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.897095 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.896915 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a1723cfd-d190-4d26-9964-3c378ba0cd3b-hosts-file\") pod \"node-resolver-ddnhg\" (UID: \"a1723cfd-d190-4d26-9964-3c378ba0cd3b\") " pod="openshift-dns/node-resolver-ddnhg" May 26 15:20:52.897095 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.896923 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c4d73d69-aaad-48c5-bfbc-75ac14352253-cnibin\") pod \"multus-additional-cni-plugins-knssg\" (UID: \"c4d73d69-aaad-48c5-bfbc-75ac14352253\") " pod="openshift-multus/multus-additional-cni-plugins-knssg" May 26 15:20:52.897095 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.896947 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/91d86e11-9ca2-4467-8929-484f81370ffe-registration-dir\") pod \"aws-ebs-csi-driver-node-mrrv8\" (UID: \"91d86e11-9ca2-4467-8929-484f81370ffe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrrv8" May 26 15:20:52.897095 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.896975 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/91d86e11-9ca2-4467-8929-484f81370ffe-device-dir\") pod \"aws-ebs-csi-driver-node-mrrv8\" (UID: \"91d86e11-9ca2-4467-8929-484f81370ffe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrrv8" May 26 15:20:52.897095 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.896990 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a1723cfd-d190-4d26-9964-3c378ba0cd3b-hosts-file\") pod \"node-resolver-ddnhg\" (UID: \"a1723cfd-d190-4d26-9964-3c378ba0cd3b\") " pod="openshift-dns/node-resolver-ddnhg" May 26 15:20:52.897095 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.896999 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f5579674-2eaf-44d9-b407-585c871463c7-host-run-k8s-cni-cncf-io\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:52.897095 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.897014 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c4d73d69-aaad-48c5-bfbc-75ac14352253-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-knssg\" (UID: \"c4d73d69-aaad-48c5-bfbc-75ac14352253\") " pod="openshift-multus/multus-additional-cni-plugins-knssg" May 26 15:20:52.897095 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.897026 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a402afb6-d0bc-4139-8153-4e1463fb49cc-host-slash\") pod \"iptables-alerter-6txjd\" (UID: \"a402afb6-d0bc-4139-8153-4e1463fb49cc\") " pod="openshift-network-operator/iptables-alerter-6txjd" May 26 15:20:52.897095 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.897049 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-etc-openvswitch\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:52.897095 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.897089 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/beeec92c-3a10-4520-a63d-984e02c0bbcf-ovn-node-metrics-cert\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:52.897814 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.897097 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1723cfd-d190-4d26-9964-3c378ba0cd3b-tmp-dir\") pod \"node-resolver-ddnhg\" (UID: \"a1723cfd-d190-4d26-9964-3c378ba0cd3b\") " pod="openshift-dns/node-resolver-ddnhg" May 26 15:20:52.897814 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.897114 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/beeec92c-3a10-4520-a63d-984e02c0bbcf-ovnkube-script-lib\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:52.897814 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.897145 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/91d86e11-9ca2-4467-8929-484f81370ffe-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mrrv8\" (UID: \"91d86e11-9ca2-4467-8929-484f81370ffe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrrv8" May 26 15:20:52.897814 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.897182 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9rpdx\" (UniqueName: \"kubernetes.io/projected/a1723cfd-d190-4d26-9964-3c378ba0cd3b-kube-api-access-9rpdx\") pod \"node-resolver-ddnhg\" (UID: \"a1723cfd-d190-4d26-9964-3c378ba0cd3b\") " pod="openshift-dns/node-resolver-ddnhg" May 26 15:20:52.897814 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.897210 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72e9400c-a618-45b9-8b31-3fa6b713a9a2-etc-kubernetes\") pod \"tuned-spgwf\" (UID: \"72e9400c-a618-45b9-8b31-3fa6b713a9a2\") " pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.897814 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.897235 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/72e9400c-a618-45b9-8b31-3fa6b713a9a2-tmp\") pod \"tuned-spgwf\" (UID: \"72e9400c-a618-45b9-8b31-3fa6b713a9a2\") " pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.897814 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.897262 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-systemd-units\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:52.897814 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.897288 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/beeec92c-3a10-4520-a63d-984e02c0bbcf-ovnkube-config\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:52.897814 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.897303 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72e9400c-a618-45b9-8b31-3fa6b713a9a2-etc-kubernetes\") pod \"tuned-spgwf\" (UID: \"72e9400c-a618-45b9-8b31-3fa6b713a9a2\") " pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.897814 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.897313 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f5579674-2eaf-44d9-b407-585c871463c7-multus-cni-dir\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:52.897814 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.897352 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-node-log\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:52.897814 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.897396 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/91d86e11-9ca2-4467-8929-484f81370ffe-etc-selinux\") pod \"aws-ebs-csi-driver-node-mrrv8\" (UID: \"91d86e11-9ca2-4467-8929-484f81370ffe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrrv8" May 26 15:20:52.897814 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.897436 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zqc8\" (UniqueName: \"kubernetes.io/projected/91d86e11-9ca2-4467-8929-484f81370ffe-kube-api-access-6zqc8\") pod \"aws-ebs-csi-driver-node-mrrv8\" (UID: \"91d86e11-9ca2-4467-8929-484f81370ffe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrrv8" May 26 15:20:52.897814 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.897466 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c4d73d69-aaad-48c5-bfbc-75ac14352253-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-knssg\" (UID: \"c4d73d69-aaad-48c5-bfbc-75ac14352253\") " pod="openshift-multus/multus-additional-cni-plugins-knssg" May 26 15:20:52.897814 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.897487 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-host-kubelet\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:52.897814 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.897537 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-host-slash\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:52.897814 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.897579 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/91d86e11-9ca2-4467-8929-484f81370ffe-socket-dir\") pod \"aws-ebs-csi-driver-node-mrrv8\" (UID: \"91d86e11-9ca2-4467-8929-484f81370ffe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrrv8" May 26 15:20:52.898571 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.897617 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f5579674-2eaf-44d9-b407-585c871463c7-cnibin\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:52.898571 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.897660 2547 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" May 26 15:20:52.898571 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.897649 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f5579674-2eaf-44d9-b407-585c871463c7-host-run-multus-certs\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:52.898571 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.897746 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lmmd\" (UniqueName: \"kubernetes.io/projected/033a1d5a-e393-4e8c-aa09-3786637f5a7e-kube-api-access-4lmmd\") pod \"network-check-target-kkq9m\" (UID: \"033a1d5a-e393-4e8c-aa09-3786637f5a7e\") " pod="openshift-network-diagnostics/network-check-target-kkq9m" May 26 15:20:52.898571 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.897776 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/beeec92c-3a10-4520-a63d-984e02c0bbcf-env-overrides\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:52.898571 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.897825 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c4d73d69-aaad-48c5-bfbc-75ac14352253-os-release\") pod \"multus-additional-cni-plugins-knssg\" (UID: \"c4d73d69-aaad-48c5-bfbc-75ac14352253\") " pod="openshift-multus/multus-additional-cni-plugins-knssg" May 26 15:20:52.898571 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.897855 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/72e9400c-a618-45b9-8b31-3fa6b713a9a2-etc-sysconfig\") pod \"tuned-spgwf\" (UID: \"72e9400c-a618-45b9-8b31-3fa6b713a9a2\") " pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.898571 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.897895 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/72e9400c-a618-45b9-8b31-3fa6b713a9a2-var-lib-kubelet\") pod \"tuned-spgwf\" (UID: \"72e9400c-a618-45b9-8b31-3fa6b713a9a2\") " pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.898571 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.897903 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c4d73d69-aaad-48c5-bfbc-75ac14352253-os-release\") pod \"multus-additional-cni-plugins-knssg\" (UID: \"c4d73d69-aaad-48c5-bfbc-75ac14352253\") " pod="openshift-multus/multus-additional-cni-plugins-knssg" May 26 15:20:52.898571 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.897927 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c4d73d69-aaad-48c5-bfbc-75ac14352253-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-knssg\" (UID: \"c4d73d69-aaad-48c5-bfbc-75ac14352253\") " pod="openshift-multus/multus-additional-cni-plugins-knssg" May 26 15:20:52.898571 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.897944 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/72e9400c-a618-45b9-8b31-3fa6b713a9a2-etc-sysconfig\") pod \"tuned-spgwf\" (UID: \"72e9400c-a618-45b9-8b31-3fa6b713a9a2\") " pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.898571 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.897918 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/72e9400c-a618-45b9-8b31-3fa6b713a9a2-host\") pod \"tuned-spgwf\" (UID: \"72e9400c-a618-45b9-8b31-3fa6b713a9a2\") " pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.898571 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.897981 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/72e9400c-a618-45b9-8b31-3fa6b713a9a2-var-lib-kubelet\") pod \"tuned-spgwf\" (UID: \"72e9400c-a618-45b9-8b31-3fa6b713a9a2\") " pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.898571 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.897965 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/72e9400c-a618-45b9-8b31-3fa6b713a9a2-host\") pod \"tuned-spgwf\" (UID: \"72e9400c-a618-45b9-8b31-3fa6b713a9a2\") " pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.898571 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.897996 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-var-lib-openvswitch\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:52.898571 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.898023 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nsn5\" (UniqueName: \"kubernetes.io/projected/beeec92c-3a10-4520-a63d-984e02c0bbcf-kube-api-access-5nsn5\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:52.898571 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.898047 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f5579674-2eaf-44d9-b407-585c871463c7-host-var-lib-cni-multus\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:52.899359 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.898119 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-log-socket\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:52.899359 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.898163 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fac6c416-4fef-4b1b-846f-ea63a14c9efe-metrics-certs\") pod \"network-metrics-daemon-5xh82\" (UID: \"fac6c416-4fef-4b1b-846f-ea63a14c9efe\") " pod="openshift-multus/network-metrics-daemon-5xh82" May 26 15:20:52.899359 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.898189 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z5mhj\" (UniqueName: \"kubernetes.io/projected/c4d73d69-aaad-48c5-bfbc-75ac14352253-kube-api-access-z5mhj\") pod \"multus-additional-cni-plugins-knssg\" (UID: \"c4d73d69-aaad-48c5-bfbc-75ac14352253\") " pod="openshift-multus/multus-additional-cni-plugins-knssg" May 26 15:20:52.899359 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.898222 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/eda06c33-9985-46af-ac9d-d006488f30ca-konnectivity-ca\") pod \"konnectivity-agent-qcrp6\" (UID: \"eda06c33-9985-46af-ac9d-d006488f30ca\") " pod="kube-system/konnectivity-agent-qcrp6" May 26 15:20:52.899359 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.898257 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f5579674-2eaf-44d9-b407-585c871463c7-host-var-lib-kubelet\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:52.899359 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.898283 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/72e9400c-a618-45b9-8b31-3fa6b713a9a2-etc-sysctl-conf\") pod \"tuned-spgwf\" (UID: \"72e9400c-a618-45b9-8b31-3fa6b713a9a2\") " pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.899359 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:52.898289 2547 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 26 15:20:52.899359 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.898322 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4f6155d9-d73c-483b-ade7-8eecd0e99883-host\") pod \"node-ca-2r56q\" (UID: \"4f6155d9-d73c-483b-ade7-8eecd0e99883\") " pod="openshift-image-registry/node-ca-2r56q" May 26 15:20:52.899359 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:52.898348 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fac6c416-4fef-4b1b-846f-ea63a14c9efe-metrics-certs podName:fac6c416-4fef-4b1b-846f-ea63a14c9efe nodeName:}" failed. No retries permitted until 2026-05-26 15:20:53.398333545 +0000 UTC m=+2.125008474 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fac6c416-4fef-4b1b-846f-ea63a14c9efe-metrics-certs") pod "network-metrics-daemon-5xh82" (UID: "fac6c416-4fef-4b1b-846f-ea63a14c9efe") : object "openshift-multus"/"metrics-daemon-secret" not registered May 26 15:20:52.899359 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.898364 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f5579674-2eaf-44d9-b407-585c871463c7-host-var-lib-cni-bin\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:52.899359 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.898367 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4f6155d9-d73c-483b-ade7-8eecd0e99883-host\") pod \"node-ca-2r56q\" (UID: \"4f6155d9-d73c-483b-ade7-8eecd0e99883\") " pod="openshift-image-registry/node-ca-2r56q" May 26 15:20:52.899359 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.898381 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f5579674-2eaf-44d9-b407-585c871463c7-hostroot\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:52.899359 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.898480 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/72e9400c-a618-45b9-8b31-3fa6b713a9a2-etc-sysctl-conf\") pod \"tuned-spgwf\" (UID: \"72e9400c-a618-45b9-8b31-3fa6b713a9a2\") " pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.899359 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.898513 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f5579674-2eaf-44d9-b407-585c871463c7-multus-daemon-config\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:52.899359 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.898542 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7vq2r\" (UniqueName: \"kubernetes.io/projected/fac6c416-4fef-4b1b-846f-ea63a14c9efe-kube-api-access-7vq2r\") pod \"network-metrics-daemon-5xh82\" (UID: \"fac6c416-4fef-4b1b-846f-ea63a14c9efe\") " pod="openshift-multus/network-metrics-daemon-5xh82" May 26 15:20:52.899359 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.898568 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/72e9400c-a618-45b9-8b31-3fa6b713a9a2-etc-systemd\") pod \"tuned-spgwf\" (UID: \"72e9400c-a618-45b9-8b31-3fa6b713a9a2\") " pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.899359 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.898592 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/72e9400c-a618-45b9-8b31-3fa6b713a9a2-lib-modules\") pod \"tuned-spgwf\" (UID: \"72e9400c-a618-45b9-8b31-3fa6b713a9a2\") " pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.899890 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.898616 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4f6155d9-d73c-483b-ade7-8eecd0e99883-serviceca\") pod \"node-ca-2r56q\" (UID: \"4f6155d9-d73c-483b-ade7-8eecd0e99883\") " pod="openshift-image-registry/node-ca-2r56q" May 26 15:20:52.899890 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.898640 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k255d\" (UniqueName: \"kubernetes.io/projected/4f6155d9-d73c-483b-ade7-8eecd0e99883-kube-api-access-k255d\") pod \"node-ca-2r56q\" (UID: \"4f6155d9-d73c-483b-ade7-8eecd0e99883\") " pod="openshift-image-registry/node-ca-2r56q" May 26 15:20:52.899890 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.898668 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-run-systemd\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:52.899890 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.898693 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-host-run-ovn-kubernetes\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:52.899890 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.898702 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/eda06c33-9985-46af-ac9d-d006488f30ca-konnectivity-ca\") pod \"konnectivity-agent-qcrp6\" (UID: \"eda06c33-9985-46af-ac9d-d006488f30ca\") " pod="kube-system/konnectivity-agent-qcrp6" May 26 15:20:52.899890 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.898709 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/72e9400c-a618-45b9-8b31-3fa6b713a9a2-lib-modules\") pod \"tuned-spgwf\" (UID: \"72e9400c-a618-45b9-8b31-3fa6b713a9a2\") " pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.899890 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.898717 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f5579674-2eaf-44d9-b407-585c871463c7-cni-binary-copy\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:52.899890 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.898742 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c4d73d69-aaad-48c5-bfbc-75ac14352253-system-cni-dir\") pod \"multus-additional-cni-plugins-knssg\" (UID: \"c4d73d69-aaad-48c5-bfbc-75ac14352253\") " pod="openshift-multus/multus-additional-cni-plugins-knssg" May 26 15:20:52.899890 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.898748 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/72e9400c-a618-45b9-8b31-3fa6b713a9a2-etc-systemd\") pod \"tuned-spgwf\" (UID: \"72e9400c-a618-45b9-8b31-3fa6b713a9a2\") " pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.899890 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.898769 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c4d73d69-aaad-48c5-bfbc-75ac14352253-tuning-conf-dir\") pod \"multus-additional-cni-plugins-knssg\" (UID: \"c4d73d69-aaad-48c5-bfbc-75ac14352253\") " pod="openshift-multus/multus-additional-cni-plugins-knssg" May 26 15:20:52.899890 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.898786 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c4d73d69-aaad-48c5-bfbc-75ac14352253-system-cni-dir\") pod \"multus-additional-cni-plugins-knssg\" (UID: \"c4d73d69-aaad-48c5-bfbc-75ac14352253\") " pod="openshift-multus/multus-additional-cni-plugins-knssg" May 26 15:20:52.899890 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.898793 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/72e9400c-a618-45b9-8b31-3fa6b713a9a2-etc-tuned\") pod \"tuned-spgwf\" (UID: \"72e9400c-a618-45b9-8b31-3fa6b713a9a2\") " pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.899890 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.898897 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c4d73d69-aaad-48c5-bfbc-75ac14352253-tuning-conf-dir\") pod \"multus-additional-cni-plugins-knssg\" (UID: \"c4d73d69-aaad-48c5-bfbc-75ac14352253\") " pod="openshift-multus/multus-additional-cni-plugins-knssg" May 26 15:20:52.899890 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.898933 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dnzq\" (UniqueName: \"kubernetes.io/projected/a402afb6-d0bc-4139-8153-4e1463fb49cc-kube-api-access-2dnzq\") pod \"iptables-alerter-6txjd\" (UID: \"a402afb6-d0bc-4139-8153-4e1463fb49cc\") " pod="openshift-network-operator/iptables-alerter-6txjd" May 26 15:20:52.899890 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.898961 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-run-ovn\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:52.899890 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.898999 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-host-cni-netd\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:52.899890 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.899026 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/72e9400c-a618-45b9-8b31-3fa6b713a9a2-run\") pod \"tuned-spgwf\" (UID: \"72e9400c-a618-45b9-8b31-3fa6b713a9a2\") " pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.900427 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.899077 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/72e9400c-a618-45b9-8b31-3fa6b713a9a2-sys\") pod \"tuned-spgwf\" (UID: \"72e9400c-a618-45b9-8b31-3fa6b713a9a2\") " pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.900427 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.899101 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/eda06c33-9985-46af-ac9d-d006488f30ca-agent-certs\") pod \"konnectivity-agent-qcrp6\" (UID: \"eda06c33-9985-46af-ac9d-d006488f30ca\") " pod="kube-system/konnectivity-agent-qcrp6" May 26 15:20:52.900427 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.899126 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f5579674-2eaf-44d9-b407-585c871463c7-system-cni-dir\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:52.900427 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.899140 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/72e9400c-a618-45b9-8b31-3fa6b713a9a2-run\") pod \"tuned-spgwf\" (UID: \"72e9400c-a618-45b9-8b31-3fa6b713a9a2\") " pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.900427 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.899153 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f5579674-2eaf-44d9-b407-585c871463c7-etc-kubernetes\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:52.900427 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.899175 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/72e9400c-a618-45b9-8b31-3fa6b713a9a2-sys\") pod \"tuned-spgwf\" (UID: \"72e9400c-a618-45b9-8b31-3fa6b713a9a2\") " pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.900427 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.899196 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4f6155d9-d73c-483b-ade7-8eecd0e99883-serviceca\") pod \"node-ca-2r56q\" (UID: \"4f6155d9-d73c-483b-ade7-8eecd0e99883\") " pod="openshift-image-registry/node-ca-2r56q" May 26 15:20:52.900745 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.900724 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/72e9400c-a618-45b9-8b31-3fa6b713a9a2-tmp\") pod \"tuned-spgwf\" (UID: \"72e9400c-a618-45b9-8b31-3fa6b713a9a2\") " pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.900798 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.900745 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/72e9400c-a618-45b9-8b31-3fa6b713a9a2-etc-tuned\") pod \"tuned-spgwf\" (UID: \"72e9400c-a618-45b9-8b31-3fa6b713a9a2\") " pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.901342 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.901325 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/eda06c33-9985-46af-ac9d-d006488f30ca-agent-certs\") pod \"konnectivity-agent-qcrp6\" (UID: \"eda06c33-9985-46af-ac9d-d006488f30ca\") " pod="kube-system/konnectivity-agent-qcrp6" May 26 15:20:52.914135 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.914105 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rpdx\" (UniqueName: \"kubernetes.io/projected/a1723cfd-d190-4d26-9964-3c378ba0cd3b-kube-api-access-9rpdx\") pod \"node-resolver-ddnhg\" (UID: \"a1723cfd-d190-4d26-9964-3c378ba0cd3b\") " pod="openshift-dns/node-resolver-ddnhg" May 26 15:20:52.914848 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.914819 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jndc7\" (UniqueName: \"kubernetes.io/projected/72e9400c-a618-45b9-8b31-3fa6b713a9a2-kube-api-access-jndc7\") pod \"tuned-spgwf\" (UID: \"72e9400c-a618-45b9-8b31-3fa6b713a9a2\") " pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:52.922701 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.922307 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5mhj\" (UniqueName: \"kubernetes.io/projected/c4d73d69-aaad-48c5-bfbc-75ac14352253-kube-api-access-z5mhj\") pod \"multus-additional-cni-plugins-knssg\" (UID: \"c4d73d69-aaad-48c5-bfbc-75ac14352253\") " pod="openshift-multus/multus-additional-cni-plugins-knssg" May 26 15:20:52.922701 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.922586 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k255d\" (UniqueName: \"kubernetes.io/projected/4f6155d9-d73c-483b-ade7-8eecd0e99883-kube-api-access-k255d\") pod \"node-ca-2r56q\" (UID: \"4f6155d9-d73c-483b-ade7-8eecd0e99883\") " pod="openshift-image-registry/node-ca-2r56q" May 26 15:20:52.923392 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.923371 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vq2r\" (UniqueName: \"kubernetes.io/projected/fac6c416-4fef-4b1b-846f-ea63a14c9efe-kube-api-access-7vq2r\") pod \"network-metrics-daemon-5xh82\" (UID: \"fac6c416-4fef-4b1b-846f-ea63a14c9efe\") " pod="openshift-multus/network-metrics-daemon-5xh82" May 26 15:20:52.938627 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:52.938601 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0640ae7f05c27f5c076d0840b04ed3a3.slice/crio-dcb9ab2475276cc8d2b192c75795c8321dcc9369f565ec9235e08df3129cf270 WatchSource:0}: Error finding container dcb9ab2475276cc8d2b192c75795c8321dcc9369f565ec9235e08df3129cf270: Status 404 returned error can't find the container with id dcb9ab2475276cc8d2b192c75795c8321dcc9369f565ec9235e08df3129cf270 May 26 15:20:52.938852 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:52.938834 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10db4583199c612d03dabeed2e12e039.slice/crio-9e927a797d8bb01df2be3b0ffb9d4593c8ff0252692d8632337b05c3e72e6b00 WatchSource:0}: Error finding container 9e927a797d8bb01df2be3b0ffb9d4593c8ff0252692d8632337b05c3e72e6b00: Status 404 returned error can't find the container with id 9e927a797d8bb01df2be3b0ffb9d4593c8ff0252692d8632337b05c3e72e6b00 May 26 15:20:52.941628 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:52.941605 2547 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 26 15:20:53.000298 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.000276 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-host-run-netns\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:53.000418 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.000303 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2275m\" (UniqueName: \"kubernetes.io/projected/f5579674-2eaf-44d9-b407-585c871463c7-kube-api-access-2275m\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:53.000418 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.000322 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-host-cni-bin\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:53.000418 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.000341 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:53.000418 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.000374 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f5579674-2eaf-44d9-b407-585c871463c7-os-release\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:53.000418 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.000397 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f5579674-2eaf-44d9-b407-585c871463c7-multus-socket-dir-parent\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:53.000667 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.000432 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-run-openvswitch\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:53.000667 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.000412 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:53.000667 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.000461 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/91d86e11-9ca2-4467-8929-484f81370ffe-sys-fs\") pod \"aws-ebs-csi-driver-node-mrrv8\" (UID: \"91d86e11-9ca2-4467-8929-484f81370ffe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrrv8" May 26 15:20:53.000667 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.000471 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f5579674-2eaf-44d9-b407-585c871463c7-os-release\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:53.000667 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.000486 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f5579674-2eaf-44d9-b407-585c871463c7-host-run-netns\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:53.000667 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.000508 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-run-openvswitch\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:53.000667 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.000513 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f5579674-2eaf-44d9-b407-585c871463c7-multus-conf-dir\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:53.000667 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.000481 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-host-run-netns\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:53.000667 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.000519 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-host-cni-bin\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:53.000667 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.000539 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a402afb6-d0bc-4139-8153-4e1463fb49cc-iptables-alerter-script\") pod \"iptables-alerter-6txjd\" (UID: \"a402afb6-d0bc-4139-8153-4e1463fb49cc\") " pod="openshift-network-operator/iptables-alerter-6txjd" May 26 15:20:53.000667 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.000541 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f5579674-2eaf-44d9-b407-585c871463c7-multus-socket-dir-parent\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:53.000667 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.000547 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f5579674-2eaf-44d9-b407-585c871463c7-host-run-netns\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:53.000667 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.000565 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f5579674-2eaf-44d9-b407-585c871463c7-multus-conf-dir\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:53.000667 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.000589 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/91d86e11-9ca2-4467-8929-484f81370ffe-sys-fs\") pod \"aws-ebs-csi-driver-node-mrrv8\" (UID: \"91d86e11-9ca2-4467-8929-484f81370ffe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrrv8" May 26 15:20:53.000667 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.000622 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/91d86e11-9ca2-4467-8929-484f81370ffe-registration-dir\") pod \"aws-ebs-csi-driver-node-mrrv8\" (UID: \"91d86e11-9ca2-4467-8929-484f81370ffe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrrv8" May 26 15:20:53.000667 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.000651 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/91d86e11-9ca2-4467-8929-484f81370ffe-device-dir\") pod \"aws-ebs-csi-driver-node-mrrv8\" (UID: \"91d86e11-9ca2-4467-8929-484f81370ffe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrrv8" May 26 15:20:53.000667 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.000669 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/91d86e11-9ca2-4467-8929-484f81370ffe-registration-dir\") pod \"aws-ebs-csi-driver-node-mrrv8\" (UID: \"91d86e11-9ca2-4467-8929-484f81370ffe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrrv8" May 26 15:20:53.001409 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.000676 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f5579674-2eaf-44d9-b407-585c871463c7-host-run-k8s-cni-cncf-io\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:53.001409 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.000711 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f5579674-2eaf-44d9-b407-585c871463c7-host-run-k8s-cni-cncf-io\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:53.001409 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.000715 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a402afb6-d0bc-4139-8153-4e1463fb49cc-host-slash\") pod \"iptables-alerter-6txjd\" (UID: \"a402afb6-d0bc-4139-8153-4e1463fb49cc\") " pod="openshift-network-operator/iptables-alerter-6txjd" May 26 15:20:53.001409 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.000723 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/91d86e11-9ca2-4467-8929-484f81370ffe-device-dir\") pod \"aws-ebs-csi-driver-node-mrrv8\" (UID: \"91d86e11-9ca2-4467-8929-484f81370ffe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrrv8" May 26 15:20:53.001409 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.000743 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-etc-openvswitch\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:53.001409 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.000756 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a402afb6-d0bc-4139-8153-4e1463fb49cc-host-slash\") pod \"iptables-alerter-6txjd\" (UID: \"a402afb6-d0bc-4139-8153-4e1463fb49cc\") " pod="openshift-network-operator/iptables-alerter-6txjd" May 26 15:20:53.001409 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.000770 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/beeec92c-3a10-4520-a63d-984e02c0bbcf-ovn-node-metrics-cert\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:53.001409 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.000780 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-etc-openvswitch\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:53.001409 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.000796 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/beeec92c-3a10-4520-a63d-984e02c0bbcf-ovnkube-script-lib\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:53.001409 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.000822 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/91d86e11-9ca2-4467-8929-484f81370ffe-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mrrv8\" (UID: \"91d86e11-9ca2-4467-8929-484f81370ffe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrrv8" May 26 15:20:53.001409 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.000848 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-systemd-units\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:53.001409 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.000873 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/beeec92c-3a10-4520-a63d-984e02c0bbcf-ovnkube-config\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:53.001409 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.000907 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/91d86e11-9ca2-4467-8929-484f81370ffe-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mrrv8\" (UID: \"91d86e11-9ca2-4467-8929-484f81370ffe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrrv8" May 26 15:20:53.001409 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.000917 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-systemd-units\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:53.001409 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.000970 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f5579674-2eaf-44d9-b407-585c871463c7-multus-cni-dir\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:53.001409 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001001 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-node-log\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:53.001409 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001040 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a402afb6-d0bc-4139-8153-4e1463fb49cc-iptables-alerter-script\") pod \"iptables-alerter-6txjd\" (UID: \"a402afb6-d0bc-4139-8153-4e1463fb49cc\") " pod="openshift-network-operator/iptables-alerter-6txjd" May 26 15:20:53.002211 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001080 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f5579674-2eaf-44d9-b407-585c871463c7-multus-cni-dir\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:53.002211 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001119 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/91d86e11-9ca2-4467-8929-484f81370ffe-etc-selinux\") pod \"aws-ebs-csi-driver-node-mrrv8\" (UID: \"91d86e11-9ca2-4467-8929-484f81370ffe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrrv8" May 26 15:20:53.002211 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001127 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-node-log\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:53.002211 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001161 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6zqc8\" (UniqueName: \"kubernetes.io/projected/91d86e11-9ca2-4467-8929-484f81370ffe-kube-api-access-6zqc8\") pod \"aws-ebs-csi-driver-node-mrrv8\" (UID: \"91d86e11-9ca2-4467-8929-484f81370ffe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrrv8" May 26 15:20:53.002211 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001201 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-host-kubelet\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:53.002211 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001211 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/91d86e11-9ca2-4467-8929-484f81370ffe-etc-selinux\") pod \"aws-ebs-csi-driver-node-mrrv8\" (UID: \"91d86e11-9ca2-4467-8929-484f81370ffe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrrv8" May 26 15:20:53.002211 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001228 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-host-slash\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:53.002211 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001253 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-host-kubelet\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:53.002211 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001273 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/91d86e11-9ca2-4467-8929-484f81370ffe-socket-dir\") pod \"aws-ebs-csi-driver-node-mrrv8\" (UID: \"91d86e11-9ca2-4467-8929-484f81370ffe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrrv8" May 26 15:20:53.002211 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001279 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-host-slash\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:53.002211 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001300 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f5579674-2eaf-44d9-b407-585c871463c7-cnibin\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:53.002211 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001325 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f5579674-2eaf-44d9-b407-585c871463c7-host-run-multus-certs\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:53.002211 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001362 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f5579674-2eaf-44d9-b407-585c871463c7-host-run-multus-certs\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:53.002211 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001362 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4lmmd\" (UniqueName: \"kubernetes.io/projected/033a1d5a-e393-4e8c-aa09-3786637f5a7e-kube-api-access-4lmmd\") pod \"network-check-target-kkq9m\" (UID: \"033a1d5a-e393-4e8c-aa09-3786637f5a7e\") " pod="openshift-network-diagnostics/network-check-target-kkq9m" May 26 15:20:53.002211 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001402 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f5579674-2eaf-44d9-b407-585c871463c7-cnibin\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:53.002211 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001411 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/beeec92c-3a10-4520-a63d-984e02c0bbcf-env-overrides\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:53.002211 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001428 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/91d86e11-9ca2-4467-8929-484f81370ffe-socket-dir\") pod \"aws-ebs-csi-driver-node-mrrv8\" (UID: \"91d86e11-9ca2-4467-8929-484f81370ffe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrrv8" May 26 15:20:53.002985 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001436 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/beeec92c-3a10-4520-a63d-984e02c0bbcf-ovnkube-config\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:53.002985 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001441 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/beeec92c-3a10-4520-a63d-984e02c0bbcf-ovnkube-script-lib\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:53.002985 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001454 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-var-lib-openvswitch\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:53.002985 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001486 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5nsn5\" (UniqueName: \"kubernetes.io/projected/beeec92c-3a10-4520-a63d-984e02c0bbcf-kube-api-access-5nsn5\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:53.002985 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001490 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-var-lib-openvswitch\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:53.002985 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001509 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f5579674-2eaf-44d9-b407-585c871463c7-host-var-lib-cni-multus\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:53.002985 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001528 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-log-socket\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:53.002985 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001562 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f5579674-2eaf-44d9-b407-585c871463c7-host-var-lib-kubelet\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:53.002985 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001572 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f5579674-2eaf-44d9-b407-585c871463c7-host-var-lib-cni-multus\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:53.002985 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001592 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f5579674-2eaf-44d9-b407-585c871463c7-host-var-lib-cni-bin\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:53.002985 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001613 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-log-socket\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:53.002985 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001627 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f5579674-2eaf-44d9-b407-585c871463c7-hostroot\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:53.002985 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001639 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f5579674-2eaf-44d9-b407-585c871463c7-host-var-lib-kubelet\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:53.002985 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001650 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f5579674-2eaf-44d9-b407-585c871463c7-multus-daemon-config\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:53.002985 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001674 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f5579674-2eaf-44d9-b407-585c871463c7-hostroot\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:53.002985 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001677 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f5579674-2eaf-44d9-b407-585c871463c7-host-var-lib-cni-bin\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:53.002985 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001681 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-run-systemd\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:53.002985 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001704 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-host-run-ovn-kubernetes\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:53.003515 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001715 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-run-systemd\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:53.003515 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001743 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f5579674-2eaf-44d9-b407-585c871463c7-cni-binary-copy\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:53.003515 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001763 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2dnzq\" (UniqueName: \"kubernetes.io/projected/a402afb6-d0bc-4139-8153-4e1463fb49cc-kube-api-access-2dnzq\") pod \"iptables-alerter-6txjd\" (UID: \"a402afb6-d0bc-4139-8153-4e1463fb49cc\") " pod="openshift-network-operator/iptables-alerter-6txjd" May 26 15:20:53.003515 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001761 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-host-run-ovn-kubernetes\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:53.003515 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001781 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-run-ovn\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:53.003515 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001798 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/beeec92c-3a10-4520-a63d-984e02c0bbcf-env-overrides\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:53.003515 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001816 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-host-cni-netd\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:53.003515 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001837 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f5579674-2eaf-44d9-b407-585c871463c7-system-cni-dir\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:53.003515 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001850 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-run-ovn\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:53.003515 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001854 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f5579674-2eaf-44d9-b407-585c871463c7-etc-kubernetes\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:53.003515 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001894 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/beeec92c-3a10-4520-a63d-984e02c0bbcf-host-cni-netd\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:53.003515 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001901 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f5579674-2eaf-44d9-b407-585c871463c7-system-cni-dir\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:53.003515 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.001916 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f5579674-2eaf-44d9-b407-585c871463c7-etc-kubernetes\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:53.003515 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.002183 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f5579674-2eaf-44d9-b407-585c871463c7-cni-binary-copy\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:53.003515 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.002199 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f5579674-2eaf-44d9-b407-585c871463c7-multus-daemon-config\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:53.003959 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.003905 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/beeec92c-3a10-4520-a63d-984e02c0bbcf-ovn-node-metrics-cert\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:53.007329 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:53.007311 2547 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 26 15:20:53.007329 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:53.007329 2547 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 26 15:20:53.007493 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:53.007340 2547 projected.go:194] Error preparing data for projected volume kube-api-access-4lmmd for pod openshift-network-diagnostics/network-check-target-kkq9m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 26 15:20:53.007493 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:53.007389 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/033a1d5a-e393-4e8c-aa09-3786637f5a7e-kube-api-access-4lmmd podName:033a1d5a-e393-4e8c-aa09-3786637f5a7e nodeName:}" failed. No retries permitted until 2026-05-26 15:20:53.507376377 +0000 UTC m=+2.234051293 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4lmmd" (UniqueName: "kubernetes.io/projected/033a1d5a-e393-4e8c-aa09-3786637f5a7e-kube-api-access-4lmmd") pod "network-check-target-kkq9m" (UID: "033a1d5a-e393-4e8c-aa09-3786637f5a7e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 26 15:20:53.008856 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.008828 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nsn5\" (UniqueName: \"kubernetes.io/projected/beeec92c-3a10-4520-a63d-984e02c0bbcf-kube-api-access-5nsn5\") pod \"ovnkube-node-vfdvw\" (UID: \"beeec92c-3a10-4520-a63d-984e02c0bbcf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:53.008928 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.008827 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2275m\" (UniqueName: \"kubernetes.io/projected/f5579674-2eaf-44d9-b407-585c871463c7-kube-api-access-2275m\") pod \"multus-66nvf\" (UID: \"f5579674-2eaf-44d9-b407-585c871463c7\") " pod="openshift-multus/multus-66nvf" May 26 15:20:53.009317 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.009302 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dnzq\" (UniqueName: \"kubernetes.io/projected/a402afb6-d0bc-4139-8153-4e1463fb49cc-kube-api-access-2dnzq\") pod \"iptables-alerter-6txjd\" (UID: \"a402afb6-d0bc-4139-8153-4e1463fb49cc\") " pod="openshift-network-operator/iptables-alerter-6txjd" May 26 15:20:53.009453 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.009439 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zqc8\" (UniqueName: \"kubernetes.io/projected/91d86e11-9ca2-4467-8929-484f81370ffe-kube-api-access-6zqc8\") pod \"aws-ebs-csi-driver-node-mrrv8\" (UID: \"91d86e11-9ca2-4467-8929-484f81370ffe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrrv8" May 26 15:20:53.110475 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.110417 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-knssg" May 26 15:20:53.117344 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:53.117316 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4d73d69_aaad_48c5_bfbc_75ac14352253.slice/crio-fdccf4e1e490cd8a8fa6455e565d45c1a8b88fd263d3b66b707ad06ae219a898 WatchSource:0}: Error finding container fdccf4e1e490cd8a8fa6455e565d45c1a8b88fd263d3b66b707ad06ae219a898: Status 404 returned error can't find the container with id fdccf4e1e490cd8a8fa6455e565d45c1a8b88fd263d3b66b707ad06ae219a898 May 26 15:20:53.122325 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.122309 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qcrp6" May 26 15:20:53.128355 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:53.128331 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeda06c33_9985_46af_ac9d_d006488f30ca.slice/crio-c2011c0d388e173a75b4c9a433bb399646703bf313193c74756f55771a0c5b88 WatchSource:0}: Error finding container c2011c0d388e173a75b4c9a433bb399646703bf313193c74756f55771a0c5b88: Status 404 returned error can't find the container with id c2011c0d388e173a75b4c9a433bb399646703bf313193c74756f55771a0c5b88 May 26 15:20:53.142294 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.142269 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2r56q" May 26 15:20:53.143742 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.143724 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-spgwf" May 26 15:20:53.150869 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:53.150840 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f6155d9_d73c_483b_ade7_8eecd0e99883.slice/crio-ca5bcdc9a90f89b6de6f29bb3af4099965be512dcfbdf2c9788443ef51dceb23 WatchSource:0}: Error finding container ca5bcdc9a90f89b6de6f29bb3af4099965be512dcfbdf2c9788443ef51dceb23: Status 404 returned error can't find the container with id ca5bcdc9a90f89b6de6f29bb3af4099965be512dcfbdf2c9788443ef51dceb23 May 26 15:20:53.151351 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:53.151311 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72e9400c_a618_45b9_8b31_3fa6b713a9a2.slice/crio-08642fb39b0f13dd5299eb6442325ed8efde346021e616dbda0957d98d2dc6ab WatchSource:0}: Error finding container 08642fb39b0f13dd5299eb6442325ed8efde346021e616dbda0957d98d2dc6ab: Status 404 returned error can't find the container with id 08642fb39b0f13dd5299eb6442325ed8efde346021e616dbda0957d98d2dc6ab May 26 15:20:53.166101 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.166085 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ddnhg" May 26 15:20:53.171493 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:53.171472 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1723cfd_d190_4d26_9964_3c378ba0cd3b.slice/crio-b606b188dd6261b6ee107255213d29b9359781887213dcb42d4a9514e0d00f08 WatchSource:0}: Error finding container b606b188dd6261b6ee107255213d29b9359781887213dcb42d4a9514e0d00f08: Status 404 returned error can't find the container with id b606b188dd6261b6ee107255213d29b9359781887213dcb42d4a9514e0d00f08 May 26 15:20:53.181769 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.181750 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6txjd" May 26 15:20:53.187181 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:53.187156 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda402afb6_d0bc_4139_8153_4e1463fb49cc.slice/crio-6479eb101f1bc8d96f8eb4861c368244f253a51a15590e2d6899c9d711075e60 WatchSource:0}: Error finding container 6479eb101f1bc8d96f8eb4861c368244f253a51a15590e2d6899c9d711075e60: Status 404 returned error can't find the container with id 6479eb101f1bc8d96f8eb4861c368244f253a51a15590e2d6899c9d711075e60 May 26 15:20:53.194194 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.194178 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:20:53.199386 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:53.199363 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbeeec92c_3a10_4520_a63d_984e02c0bbcf.slice/crio-2d01eca5b30ac8b2cd1bd87c388b24013b96d65cc49be5fe3290f10b9f3e936d WatchSource:0}: Error finding container 2d01eca5b30ac8b2cd1bd87c388b24013b96d65cc49be5fe3290f10b9f3e936d: Status 404 returned error can't find the container with id 2d01eca5b30ac8b2cd1bd87c388b24013b96d65cc49be5fe3290f10b9f3e936d May 26 15:20:53.208633 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.208615 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrrv8" May 26 15:20:53.213295 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.213271 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-66nvf" May 26 15:20:53.214358 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:53.214339 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91d86e11_9ca2_4467_8929_484f81370ffe.slice/crio-bec4e72bf187906263721e536a0ec7403606663e7cf82a0576b86b07bb98999a WatchSource:0}: Error finding container bec4e72bf187906263721e536a0ec7403606663e7cf82a0576b86b07bb98999a: Status 404 returned error can't find the container with id bec4e72bf187906263721e536a0ec7403606663e7cf82a0576b86b07bb98999a May 26 15:20:53.218992 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:20:53.218974 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5579674_2eaf_44d9_b407_585c871463c7.slice/crio-b04b110f7c84f594c4def20a6d6f9a6dbd7ad67fb18a45f696a0463bb41e70a5 WatchSource:0}: Error finding container b04b110f7c84f594c4def20a6d6f9a6dbd7ad67fb18a45f696a0463bb41e70a5: Status 404 returned error can't find the container with id b04b110f7c84f594c4def20a6d6f9a6dbd7ad67fb18a45f696a0463bb41e70a5 May 26 15:20:53.249347 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.248176 2547 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" May 26 15:20:53.405895 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.405830 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fac6c416-4fef-4b1b-846f-ea63a14c9efe-metrics-certs\") pod \"network-metrics-daemon-5xh82\" (UID: \"fac6c416-4fef-4b1b-846f-ea63a14c9efe\") " pod="openshift-multus/network-metrics-daemon-5xh82" May 26 15:20:53.406033 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:53.405957 2547 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 26 15:20:53.406033 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:53.406010 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fac6c416-4fef-4b1b-846f-ea63a14c9efe-metrics-certs podName:fac6c416-4fef-4b1b-846f-ea63a14c9efe nodeName:}" failed. No retries permitted until 2026-05-26 15:20:54.405992612 +0000 UTC m=+3.132667514 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fac6c416-4fef-4b1b-846f-ea63a14c9efe-metrics-certs") pod "network-metrics-daemon-5xh82" (UID: "fac6c416-4fef-4b1b-846f-ea63a14c9efe") : object "openshift-multus"/"metrics-daemon-secret" not registered May 26 15:20:53.608997 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.608807 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4lmmd\" (UniqueName: \"kubernetes.io/projected/033a1d5a-e393-4e8c-aa09-3786637f5a7e-kube-api-access-4lmmd\") pod \"network-check-target-kkq9m\" (UID: \"033a1d5a-e393-4e8c-aa09-3786637f5a7e\") " pod="openshift-network-diagnostics/network-check-target-kkq9m" May 26 15:20:53.608997 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:53.608955 2547 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 26 15:20:53.608997 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:53.608980 2547 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 26 15:20:53.608997 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:53.608994 2547 projected.go:194] Error preparing data for projected volume kube-api-access-4lmmd for pod openshift-network-diagnostics/network-check-target-kkq9m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 26 15:20:53.609285 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:53.609065 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/033a1d5a-e393-4e8c-aa09-3786637f5a7e-kube-api-access-4lmmd podName:033a1d5a-e393-4e8c-aa09-3786637f5a7e nodeName:}" failed. No retries permitted until 2026-05-26 15:20:54.609031698 +0000 UTC m=+3.335706614 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4lmmd" (UniqueName: "kubernetes.io/projected/033a1d5a-e393-4e8c-aa09-3786637f5a7e-kube-api-access-4lmmd") pod "network-check-target-kkq9m" (UID: "033a1d5a-e393-4e8c-aa09-3786637f5a7e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 26 15:20:53.646531 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.646274 2547 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" May 26 15:20:53.837802 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.837621 2547 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-25 15:15:52 +0000 UTC" deadline="2027-11-20 20:42:50.076744298 +0000 UTC" May 26 15:20:53.837802 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.837720 2547 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13037h21m56.239028619s" May 26 15:20:53.910084 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.909731 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xh82" May 26 15:20:53.910084 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.909769 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkq9m" May 26 15:20:53.910084 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:53.909882 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xh82" podUID="fac6c416-4fef-4b1b-846f-ea63a14c9efe" May 26 15:20:53.910084 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:53.910022 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkq9m" podUID="033a1d5a-e393-4e8c-aa09-3786637f5a7e" May 26 15:20:53.928074 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.927982 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-66nvf" event={"ID":"f5579674-2eaf-44d9-b407-585c871463c7","Type":"ContainerStarted","Data":"b04b110f7c84f594c4def20a6d6f9a6dbd7ad67fb18a45f696a0463bb41e70a5"} May 26 15:20:53.934351 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.934304 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ddnhg" event={"ID":"a1723cfd-d190-4d26-9964-3c378ba0cd3b","Type":"ContainerStarted","Data":"b606b188dd6261b6ee107255213d29b9359781887213dcb42d4a9514e0d00f08"} May 26 15:20:53.941373 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.941348 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-spgwf" event={"ID":"72e9400c-a618-45b9-8b31-3fa6b713a9a2","Type":"ContainerStarted","Data":"08642fb39b0f13dd5299eb6442325ed8efde346021e616dbda0957d98d2dc6ab"} May 26 15:20:53.950948 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.950921 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qcrp6" event={"ID":"eda06c33-9985-46af-ac9d-d006488f30ca","Type":"ContainerStarted","Data":"c2011c0d388e173a75b4c9a433bb399646703bf313193c74756f55771a0c5b88"} May 26 15:20:53.961019 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.960992 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-knssg" event={"ID":"c4d73d69-aaad-48c5-bfbc-75ac14352253","Type":"ContainerStarted","Data":"fdccf4e1e490cd8a8fa6455e565d45c1a8b88fd263d3b66b707ad06ae219a898"} May 26 15:20:53.972557 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.972531 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-198.ec2.internal" event={"ID":"10db4583199c612d03dabeed2e12e039","Type":"ContainerStarted","Data":"9e927a797d8bb01df2be3b0ffb9d4593c8ff0252692d8632337b05c3e72e6b00"} May 26 15:20:53.977087 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.975754 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-198.ec2.internal" event={"ID":"0640ae7f05c27f5c076d0840b04ed3a3","Type":"ContainerStarted","Data":"dcb9ab2475276cc8d2b192c75795c8321dcc9369f565ec9235e08df3129cf270"} May 26 15:20:53.982883 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.982859 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrrv8" event={"ID":"91d86e11-9ca2-4467-8929-484f81370ffe","Type":"ContainerStarted","Data":"bec4e72bf187906263721e536a0ec7403606663e7cf82a0576b86b07bb98999a"} May 26 15:20:53.992079 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:53.991911 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" event={"ID":"beeec92c-3a10-4520-a63d-984e02c0bbcf","Type":"ContainerStarted","Data":"2d01eca5b30ac8b2cd1bd87c388b24013b96d65cc49be5fe3290f10b9f3e936d"} May 26 15:20:54.010049 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:54.010021 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6txjd" event={"ID":"a402afb6-d0bc-4139-8153-4e1463fb49cc","Type":"ContainerStarted","Data":"6479eb101f1bc8d96f8eb4861c368244f253a51a15590e2d6899c9d711075e60"} May 26 15:20:54.017169 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:54.017144 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2r56q" event={"ID":"4f6155d9-d73c-483b-ade7-8eecd0e99883","Type":"ContainerStarted","Data":"ca5bcdc9a90f89b6de6f29bb3af4099965be512dcfbdf2c9788443ef51dceb23"} May 26 15:20:54.034968 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:54.034944 2547 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" May 26 15:20:54.416556 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:54.416517 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fac6c416-4fef-4b1b-846f-ea63a14c9efe-metrics-certs\") pod \"network-metrics-daemon-5xh82\" (UID: \"fac6c416-4fef-4b1b-846f-ea63a14c9efe\") " pod="openshift-multus/network-metrics-daemon-5xh82" May 26 15:20:54.416740 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:54.416709 2547 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 26 15:20:54.416815 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:54.416774 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fac6c416-4fef-4b1b-846f-ea63a14c9efe-metrics-certs podName:fac6c416-4fef-4b1b-846f-ea63a14c9efe nodeName:}" failed. No retries permitted until 2026-05-26 15:20:56.416755144 +0000 UTC m=+5.143430062 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fac6c416-4fef-4b1b-846f-ea63a14c9efe-metrics-certs") pod "network-metrics-daemon-5xh82" (UID: "fac6c416-4fef-4b1b-846f-ea63a14c9efe") : object "openshift-multus"/"metrics-daemon-secret" not registered May 26 15:20:54.618548 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:54.618503 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4lmmd\" (UniqueName: \"kubernetes.io/projected/033a1d5a-e393-4e8c-aa09-3786637f5a7e-kube-api-access-4lmmd\") pod \"network-check-target-kkq9m\" (UID: \"033a1d5a-e393-4e8c-aa09-3786637f5a7e\") " pod="openshift-network-diagnostics/network-check-target-kkq9m" May 26 15:20:54.618729 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:54.618703 2547 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 26 15:20:54.618791 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:54.618725 2547 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 26 15:20:54.618791 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:54.618750 2547 projected.go:194] Error preparing data for projected volume kube-api-access-4lmmd for pod openshift-network-diagnostics/network-check-target-kkq9m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 26 15:20:54.618887 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:54.618813 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/033a1d5a-e393-4e8c-aa09-3786637f5a7e-kube-api-access-4lmmd podName:033a1d5a-e393-4e8c-aa09-3786637f5a7e nodeName:}" failed. No retries permitted until 2026-05-26 15:20:56.618792278 +0000 UTC m=+5.345467200 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4lmmd" (UniqueName: "kubernetes.io/projected/033a1d5a-e393-4e8c-aa09-3786637f5a7e-kube-api-access-4lmmd") pod "network-check-target-kkq9m" (UID: "033a1d5a-e393-4e8c-aa09-3786637f5a7e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 26 15:20:54.838522 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:54.838433 2547 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-25 15:15:52 +0000 UTC" deadline="2027-11-14 08:16:16.501944437 +0000 UTC" May 26 15:20:54.838522 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:54.838473 2547 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12880h55m21.663475142s" May 26 15:20:55.909119 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:55.909085 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkq9m" May 26 15:20:55.909575 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:55.909141 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xh82" May 26 15:20:55.909575 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:55.909223 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkq9m" podUID="033a1d5a-e393-4e8c-aa09-3786637f5a7e" May 26 15:20:55.909575 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:55.909370 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xh82" podUID="fac6c416-4fef-4b1b-846f-ea63a14c9efe" May 26 15:20:56.437117 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:56.437084 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fac6c416-4fef-4b1b-846f-ea63a14c9efe-metrics-certs\") pod \"network-metrics-daemon-5xh82\" (UID: \"fac6c416-4fef-4b1b-846f-ea63a14c9efe\") " pod="openshift-multus/network-metrics-daemon-5xh82" May 26 15:20:56.437300 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:56.437249 2547 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 26 15:20:56.437368 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:56.437318 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fac6c416-4fef-4b1b-846f-ea63a14c9efe-metrics-certs podName:fac6c416-4fef-4b1b-846f-ea63a14c9efe nodeName:}" failed. No retries permitted until 2026-05-26 15:21:00.43729713 +0000 UTC m=+9.163972036 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fac6c416-4fef-4b1b-846f-ea63a14c9efe-metrics-certs") pod "network-metrics-daemon-5xh82" (UID: "fac6c416-4fef-4b1b-846f-ea63a14c9efe") : object "openshift-multus"/"metrics-daemon-secret" not registered May 26 15:20:56.638925 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:56.638888 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4lmmd\" (UniqueName: \"kubernetes.io/projected/033a1d5a-e393-4e8c-aa09-3786637f5a7e-kube-api-access-4lmmd\") pod \"network-check-target-kkq9m\" (UID: \"033a1d5a-e393-4e8c-aa09-3786637f5a7e\") " pod="openshift-network-diagnostics/network-check-target-kkq9m" May 26 15:20:56.639126 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:56.639110 2547 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 26 15:20:56.639194 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:56.639130 2547 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 26 15:20:56.639194 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:56.639161 2547 projected.go:194] Error preparing data for projected volume kube-api-access-4lmmd for pod openshift-network-diagnostics/network-check-target-kkq9m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 26 15:20:56.639295 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:56.639224 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/033a1d5a-e393-4e8c-aa09-3786637f5a7e-kube-api-access-4lmmd podName:033a1d5a-e393-4e8c-aa09-3786637f5a7e nodeName:}" failed. No retries permitted until 2026-05-26 15:21:00.639203881 +0000 UTC m=+9.365878787 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4lmmd" (UniqueName: "kubernetes.io/projected/033a1d5a-e393-4e8c-aa09-3786637f5a7e-kube-api-access-4lmmd") pod "network-check-target-kkq9m" (UID: "033a1d5a-e393-4e8c-aa09-3786637f5a7e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 26 15:20:57.909020 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:57.908988 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkq9m" May 26 15:20:57.909492 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:57.909139 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkq9m" podUID="033a1d5a-e393-4e8c-aa09-3786637f5a7e" May 26 15:20:57.909492 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:57.909168 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xh82" May 26 15:20:57.909492 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:57.909299 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xh82" podUID="fac6c416-4fef-4b1b-846f-ea63a14c9efe" May 26 15:20:59.909435 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:59.909308 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xh82" May 26 15:20:59.909909 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:59.909452 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xh82" podUID="fac6c416-4fef-4b1b-846f-ea63a14c9efe" May 26 15:20:59.909909 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:20:59.909562 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkq9m" May 26 15:20:59.909909 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:20:59.909648 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkq9m" podUID="033a1d5a-e393-4e8c-aa09-3786637f5a7e" May 26 15:21:00.471383 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:00.471348 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fac6c416-4fef-4b1b-846f-ea63a14c9efe-metrics-certs\") pod \"network-metrics-daemon-5xh82\" (UID: \"fac6c416-4fef-4b1b-846f-ea63a14c9efe\") " pod="openshift-multus/network-metrics-daemon-5xh82" May 26 15:21:00.471566 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:00.471460 2547 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 26 15:21:00.471566 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:00.471512 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fac6c416-4fef-4b1b-846f-ea63a14c9efe-metrics-certs podName:fac6c416-4fef-4b1b-846f-ea63a14c9efe nodeName:}" failed. No retries permitted until 2026-05-26 15:21:08.471498617 +0000 UTC m=+17.198173515 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fac6c416-4fef-4b1b-846f-ea63a14c9efe-metrics-certs") pod "network-metrics-daemon-5xh82" (UID: "fac6c416-4fef-4b1b-846f-ea63a14c9efe") : object "openshift-multus"/"metrics-daemon-secret" not registered May 26 15:21:00.673009 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:00.672970 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4lmmd\" (UniqueName: \"kubernetes.io/projected/033a1d5a-e393-4e8c-aa09-3786637f5a7e-kube-api-access-4lmmd\") pod \"network-check-target-kkq9m\" (UID: \"033a1d5a-e393-4e8c-aa09-3786637f5a7e\") " pod="openshift-network-diagnostics/network-check-target-kkq9m" May 26 15:21:00.673182 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:00.673160 2547 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 26 15:21:00.673237 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:00.673189 2547 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 26 15:21:00.673237 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:00.673203 2547 projected.go:194] Error preparing data for projected volume kube-api-access-4lmmd for pod openshift-network-diagnostics/network-check-target-kkq9m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 26 15:21:00.673338 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:00.673268 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/033a1d5a-e393-4e8c-aa09-3786637f5a7e-kube-api-access-4lmmd podName:033a1d5a-e393-4e8c-aa09-3786637f5a7e nodeName:}" failed. No retries permitted until 2026-05-26 15:21:08.673248267 +0000 UTC m=+17.399923175 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4lmmd" (UniqueName: "kubernetes.io/projected/033a1d5a-e393-4e8c-aa09-3786637f5a7e-kube-api-access-4lmmd") pod "network-check-target-kkq9m" (UID: "033a1d5a-e393-4e8c-aa09-3786637f5a7e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 26 15:21:01.910674 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:01.910630 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkq9m" May 26 15:21:01.911042 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:01.910751 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkq9m" podUID="033a1d5a-e393-4e8c-aa09-3786637f5a7e" May 26 15:21:01.911042 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:01.910762 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xh82" May 26 15:21:01.911042 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:01.910849 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xh82" podUID="fac6c416-4fef-4b1b-846f-ea63a14c9efe" May 26 15:21:03.909250 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:03.909218 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xh82" May 26 15:21:03.909250 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:03.909242 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkq9m" May 26 15:21:03.909709 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:03.909356 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xh82" podUID="fac6c416-4fef-4b1b-846f-ea63a14c9efe" May 26 15:21:03.909798 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:03.909770 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkq9m" podUID="033a1d5a-e393-4e8c-aa09-3786637f5a7e" May 26 15:21:05.909163 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:05.909120 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkq9m" May 26 15:21:05.909629 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:05.909177 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xh82" May 26 15:21:05.909629 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:05.909261 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkq9m" podUID="033a1d5a-e393-4e8c-aa09-3786637f5a7e" May 26 15:21:05.909629 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:05.909371 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xh82" podUID="fac6c416-4fef-4b1b-846f-ea63a14c9efe" May 26 15:21:07.909934 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:07.909903 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xh82" May 26 15:21:07.910490 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:07.909943 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkq9m" May 26 15:21:07.910490 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:07.910052 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xh82" podUID="fac6c416-4fef-4b1b-846f-ea63a14c9efe" May 26 15:21:07.910490 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:07.910196 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkq9m" podUID="033a1d5a-e393-4e8c-aa09-3786637f5a7e" May 26 15:21:08.534946 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:08.534904 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fac6c416-4fef-4b1b-846f-ea63a14c9efe-metrics-certs\") pod \"network-metrics-daemon-5xh82\" (UID: \"fac6c416-4fef-4b1b-846f-ea63a14c9efe\") " pod="openshift-multus/network-metrics-daemon-5xh82" May 26 15:21:08.535157 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:08.535041 2547 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 26 15:21:08.535157 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:08.535123 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fac6c416-4fef-4b1b-846f-ea63a14c9efe-metrics-certs podName:fac6c416-4fef-4b1b-846f-ea63a14c9efe nodeName:}" failed. No retries permitted until 2026-05-26 15:21:24.535102322 +0000 UTC m=+33.261777234 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fac6c416-4fef-4b1b-846f-ea63a14c9efe-metrics-certs") pod "network-metrics-daemon-5xh82" (UID: "fac6c416-4fef-4b1b-846f-ea63a14c9efe") : object "openshift-multus"/"metrics-daemon-secret" not registered May 26 15:21:08.736332 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:08.736289 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4lmmd\" (UniqueName: \"kubernetes.io/projected/033a1d5a-e393-4e8c-aa09-3786637f5a7e-kube-api-access-4lmmd\") pod \"network-check-target-kkq9m\" (UID: \"033a1d5a-e393-4e8c-aa09-3786637f5a7e\") " pod="openshift-network-diagnostics/network-check-target-kkq9m" May 26 15:21:08.736518 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:08.736476 2547 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 26 15:21:08.736518 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:08.736501 2547 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 26 15:21:08.736518 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:08.736515 2547 projected.go:194] Error preparing data for projected volume kube-api-access-4lmmd for pod openshift-network-diagnostics/network-check-target-kkq9m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 26 15:21:08.736688 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:08.736585 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/033a1d5a-e393-4e8c-aa09-3786637f5a7e-kube-api-access-4lmmd podName:033a1d5a-e393-4e8c-aa09-3786637f5a7e nodeName:}" failed. No retries permitted until 2026-05-26 15:21:24.736563899 +0000 UTC m=+33.463239024 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4lmmd" (UniqueName: "kubernetes.io/projected/033a1d5a-e393-4e8c-aa09-3786637f5a7e-kube-api-access-4lmmd") pod "network-check-target-kkq9m" (UID: "033a1d5a-e393-4e8c-aa09-3786637f5a7e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 26 15:21:09.909739 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:09.909527 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkq9m" May 26 15:21:09.910216 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:09.909536 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xh82" May 26 15:21:09.910216 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:09.909827 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkq9m" podUID="033a1d5a-e393-4e8c-aa09-3786637f5a7e" May 26 15:21:09.910216 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:09.909919 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xh82" podUID="fac6c416-4fef-4b1b-846f-ea63a14c9efe" May 26 15:21:11.909532 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:11.909509 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkq9m" May 26 15:21:11.909860 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:11.909603 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkq9m" podUID="033a1d5a-e393-4e8c-aa09-3786637f5a7e" May 26 15:21:11.909860 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:11.909656 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xh82" May 26 15:21:11.909860 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:11.909740 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xh82" podUID="fac6c416-4fef-4b1b-846f-ea63a14c9efe" May 26 15:21:13.056407 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:13.055924 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-66nvf" event={"ID":"f5579674-2eaf-44d9-b407-585c871463c7","Type":"ContainerStarted","Data":"5515038ec04e3acb13049bad45736e88b22c7957be3499756feff521cc0d2964"} May 26 15:21:13.057375 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:13.057352 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ddnhg" event={"ID":"a1723cfd-d190-4d26-9964-3c378ba0cd3b","Type":"ContainerStarted","Data":"1338a0bacab2e1688933c3562a07448bb4323171f5d28932e4e78324646fb60b"} May 26 15:21:13.058821 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:13.058799 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-spgwf" event={"ID":"72e9400c-a618-45b9-8b31-3fa6b713a9a2","Type":"ContainerStarted","Data":"044bdb6dd1846ed6db9f7b7f623f11b3a2f3824974fe004f61a5e15301918104"} May 26 15:21:13.060188 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:13.060161 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qcrp6" event={"ID":"eda06c33-9985-46af-ac9d-d006488f30ca","Type":"ContainerStarted","Data":"41b3736482a470d9e9040a85199c9acdde8c2f5c3dcc1879e1458dd6c57b9e87"} May 26 15:21:13.061624 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:13.061600 2547 generic.go:358] "Generic (PLEG): container finished" podID="c4d73d69-aaad-48c5-bfbc-75ac14352253" containerID="24d961af2f9a6c98ee5701933191e7b11da69344134e9e2d5631163c66238689" exitCode=0 May 26 15:21:13.061711 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:13.061630 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-knssg" event={"ID":"c4d73d69-aaad-48c5-bfbc-75ac14352253","Type":"ContainerDied","Data":"24d961af2f9a6c98ee5701933191e7b11da69344134e9e2d5631163c66238689"} May 26 15:21:13.063077 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:13.063030 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-198.ec2.internal" event={"ID":"10db4583199c612d03dabeed2e12e039","Type":"ContainerStarted","Data":"1bf2e36802407bbf91a0c07836953a1f28b1be2f327bf4f657f0da5528b8878e"} May 26 15:21:13.064545 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:13.064522 2547 generic.go:358] "Generic (PLEG): container finished" podID="0640ae7f05c27f5c076d0840b04ed3a3" containerID="039546d117f88b98d316e4bd3f9dc53f0aaa11751a9e9165e170efc1ed54e96e" exitCode=0 May 26 15:21:13.064632 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:13.064606 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-198.ec2.internal" event={"ID":"0640ae7f05c27f5c076d0840b04ed3a3","Type":"ContainerDied","Data":"039546d117f88b98d316e4bd3f9dc53f0aaa11751a9e9165e170efc1ed54e96e"} May 26 15:21:13.066847 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:13.066812 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrrv8" event={"ID":"91d86e11-9ca2-4467-8929-484f81370ffe","Type":"ContainerStarted","Data":"37ac91534beadad7e7e5f80c1d17243c0f87bb1a814b3005bc904b199291ce71"} May 26 15:21:13.069782 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:13.069752 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" event={"ID":"beeec92c-3a10-4520-a63d-984e02c0bbcf","Type":"ContainerStarted","Data":"3fa5a2ae629b54627af8d046912dab7ce592e193e1bc09278885d11b52097912"} May 26 15:21:13.069782 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:13.069781 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" event={"ID":"beeec92c-3a10-4520-a63d-984e02c0bbcf","Type":"ContainerStarted","Data":"068cdec27837fa88467c2040ca179f44024ffcbff65aab34c3aec221ac776ed3"} May 26 15:21:13.069924 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:13.069794 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" event={"ID":"beeec92c-3a10-4520-a63d-984e02c0bbcf","Type":"ContainerStarted","Data":"463bb87afa8032ecda08639ca65ea1a131d3b933b75abb264c7ed893902cefe8"} May 26 15:21:13.069924 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:13.069806 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" event={"ID":"beeec92c-3a10-4520-a63d-984e02c0bbcf","Type":"ContainerStarted","Data":"3e1f956993ec6d099b2d676370ffb77a9b1784c8e3db660b369bb272295d6001"} May 26 15:21:13.069924 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:13.069818 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" event={"ID":"beeec92c-3a10-4520-a63d-984e02c0bbcf","Type":"ContainerStarted","Data":"3d32f2f643b1720dbc6b589318b600f0697227b8b264838483c00fbfdb71b645"} May 26 15:21:13.069924 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:13.069830 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" event={"ID":"beeec92c-3a10-4520-a63d-984e02c0bbcf","Type":"ContainerStarted","Data":"91d3129fb13a9b1b13996553b28f32b83f79328d61b13c89a5c6bbeb653bfc3d"} May 26 15:21:13.071142 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:13.071117 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2r56q" event={"ID":"4f6155d9-d73c-483b-ade7-8eecd0e99883","Type":"ContainerStarted","Data":"4d75663d8d8350c39cef0907e530a7a5ccb9de3378971a1be29dc2d2031ff747"} May 26 15:21:13.088083 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:13.087981 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-66nvf" podStartSLOduration=2.328488292 podStartE2EDuration="21.08796605s" podCreationTimestamp="2026-05-26 15:20:52 +0000 UTC" firstStartedPulling="2026-05-26 15:20:53.220395413 +0000 UTC m=+1.947070312" lastFinishedPulling="2026-05-26 15:21:11.979873167 +0000 UTC m=+20.706548070" observedRunningTime="2026-05-26 15:21:13.073381852 +0000 UTC m=+21.800056777" watchObservedRunningTime="2026-05-26 15:21:13.08796605 +0000 UTC m=+21.814640972" May 26 15:21:13.088512 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:13.088468 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-qcrp6" podStartSLOduration=8.078682387 podStartE2EDuration="22.088454863s" podCreationTimestamp="2026-05-26 15:20:51 +0000 UTC" firstStartedPulling="2026-05-26 15:20:53.129896123 +0000 UTC m=+1.856571025" lastFinishedPulling="2026-05-26 15:21:07.139668589 +0000 UTC m=+15.866343501" observedRunningTime="2026-05-26 15:21:13.08701731 +0000 UTC m=+21.813692231" watchObservedRunningTime="2026-05-26 15:21:13.088454863 +0000 UTC m=+21.815129785" May 26 15:21:13.116170 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:13.116124 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-spgwf" podStartSLOduration=2.298377689 podStartE2EDuration="21.11611004s" podCreationTimestamp="2026-05-26 15:20:52 +0000 UTC" firstStartedPulling="2026-05-26 15:20:53.15297283 +0000 UTC m=+1.879647728" lastFinishedPulling="2026-05-26 15:21:11.970705162 +0000 UTC m=+20.697380079" observedRunningTime="2026-05-26 15:21:13.10320743 +0000 UTC m=+21.829882350" watchObservedRunningTime="2026-05-26 15:21:13.11611004 +0000 UTC m=+21.842784962" May 26 15:21:13.133794 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:13.133759 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-ddnhg" podStartSLOduration=2.418574498 podStartE2EDuration="21.133749204s" podCreationTimestamp="2026-05-26 15:20:52 +0000 UTC" firstStartedPulling="2026-05-26 15:20:53.172837451 +0000 UTC m=+1.899512350" lastFinishedPulling="2026-05-26 15:21:11.888012141 +0000 UTC m=+20.614687056" observedRunningTime="2026-05-26 15:21:13.116569362 +0000 UTC m=+21.843244289" watchObservedRunningTime="2026-05-26 15:21:13.133749204 +0000 UTC m=+21.860424136" May 26 15:21:13.145077 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:13.145027 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-198.ec2.internal" podStartSLOduration=21.145017911 podStartE2EDuration="21.145017911s" podCreationTimestamp="2026-05-26 15:20:52 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 15:21:13.144876205 +0000 UTC m=+21.871551128" watchObservedRunningTime="2026-05-26 15:21:13.145017911 +0000 UTC m=+21.871692831" May 26 15:21:13.181169 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:13.181122 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-2r56q" podStartSLOduration=3.364606556 podStartE2EDuration="22.181109333s" podCreationTimestamp="2026-05-26 15:20:51 +0000 UTC" firstStartedPulling="2026-05-26 15:20:53.15242537 +0000 UTC m=+1.879100272" lastFinishedPulling="2026-05-26 15:21:11.968928147 +0000 UTC m=+20.695603049" observedRunningTime="2026-05-26 15:21:13.180836794 +0000 UTC m=+21.907511715" watchObservedRunningTime="2026-05-26 15:21:13.181109333 +0000 UTC m=+21.907784254" May 26 15:21:13.548561 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:13.548527 2547 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" May 26 15:21:13.863294 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:13.863109 2547 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-26T15:21:13.548544468Z","UUID":"fb089b3f-f3a7-4e23-843b-37fc387895ab","Handler":null,"Name":"","Endpoint":""} May 26 15:21:13.866319 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:13.866294 2547 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 May 26 15:21:13.866319 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:13.866320 2547 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock May 26 15:21:13.909841 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:13.909809 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xh82" May 26 15:21:13.909980 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:13.909851 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkq9m" May 26 15:21:13.909980 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:13.909940 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xh82" podUID="fac6c416-4fef-4b1b-846f-ea63a14c9efe" May 26 15:21:13.910145 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:13.910108 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkq9m" podUID="033a1d5a-e393-4e8c-aa09-3786637f5a7e" May 26 15:21:14.076782 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:14.076743 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-198.ec2.internal" event={"ID":"0640ae7f05c27f5c076d0840b04ed3a3","Type":"ContainerStarted","Data":"2b8f13e98bc54948d35322e815f8e67f8e25d936cd737cc5832776d280dd2990"} May 26 15:21:14.078996 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:14.078968 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrrv8" event={"ID":"91d86e11-9ca2-4467-8929-484f81370ffe","Type":"ContainerStarted","Data":"f4cd550ab3d0b077543aa0b87825cc234cc1a66ffd385ee0620cb0ddb27b25bd"} May 26 15:21:14.080969 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:14.080505 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6txjd" event={"ID":"a402afb6-d0bc-4139-8153-4e1463fb49cc","Type":"ContainerStarted","Data":"72813c29ed61f8026f8d453850587aa372c8656ab0ff95b2251c6db8301f6d5c"} May 26 15:21:14.108136 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:14.108081 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-198.ec2.internal" podStartSLOduration=22.108042822 podStartE2EDuration="22.108042822s" podCreationTimestamp="2026-05-26 15:20:52 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 15:21:14.091280406 +0000 UTC m=+22.817955326" watchObservedRunningTime="2026-05-26 15:21:14.108042822 +0000 UTC m=+22.834717744" May 26 15:21:14.108566 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:14.108499 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-6txjd" podStartSLOduration=3.32537492 podStartE2EDuration="22.10848888s" podCreationTimestamp="2026-05-26 15:20:52 +0000 UTC" firstStartedPulling="2026-05-26 15:20:53.190175591 +0000 UTC m=+1.916850489" lastFinishedPulling="2026-05-26 15:21:11.973289536 +0000 UTC m=+20.699964449" observedRunningTime="2026-05-26 15:21:14.107840498 +0000 UTC m=+22.834515419" watchObservedRunningTime="2026-05-26 15:21:14.10848888 +0000 UTC m=+22.835163804" May 26 15:21:15.084172 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:15.084121 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrrv8" event={"ID":"91d86e11-9ca2-4467-8929-484f81370ffe","Type":"ContainerStarted","Data":"4d996e34005c71527908b61ddac8539735aa1a662a99733f9c4f0c5fc4925b3f"} May 26 15:21:15.087858 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:15.087788 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" event={"ID":"beeec92c-3a10-4520-a63d-984e02c0bbcf","Type":"ContainerStarted","Data":"5606f3df7e7a7d099d59108ef06c4dfe1fabccc0fb8ec6260e84274bdeb601c2"} May 26 15:21:15.103033 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:15.102983 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mrrv8" podStartSLOduration=1.991532223 podStartE2EDuration="23.102970707s" podCreationTimestamp="2026-05-26 15:20:52 +0000 UTC" firstStartedPulling="2026-05-26 15:20:53.216261375 +0000 UTC m=+1.942936273" lastFinishedPulling="2026-05-26 15:21:14.327699841 +0000 UTC m=+23.054374757" observedRunningTime="2026-05-26 15:21:15.102277057 +0000 UTC m=+23.828951992" watchObservedRunningTime="2026-05-26 15:21:15.102970707 +0000 UTC m=+23.829645629" May 26 15:21:15.909014 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:15.908982 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xh82" May 26 15:21:15.909188 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:15.909035 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkq9m" May 26 15:21:15.909188 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:15.909129 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xh82" podUID="fac6c416-4fef-4b1b-846f-ea63a14c9efe" May 26 15:21:15.909324 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:15.909255 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkq9m" podUID="033a1d5a-e393-4e8c-aa09-3786637f5a7e" May 26 15:21:17.597105 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:17.596877 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-qcrp6" May 26 15:21:17.597750 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:17.597730 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-qcrp6" May 26 15:21:17.909698 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:17.909614 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkq9m" May 26 15:21:17.909844 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:17.909614 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xh82" May 26 15:21:17.909844 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:17.909714 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkq9m" podUID="033a1d5a-e393-4e8c-aa09-3786637f5a7e" May 26 15:21:17.909844 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:17.909790 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xh82" podUID="fac6c416-4fef-4b1b-846f-ea63a14c9efe" May 26 15:21:18.094794 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:18.094761 2547 generic.go:358] "Generic (PLEG): container finished" podID="c4d73d69-aaad-48c5-bfbc-75ac14352253" containerID="5b843703a2aed85b2d1dc0bb9f85deb3bdbc5a5747b25b0754f88c605e61bdee" exitCode=0 May 26 15:21:18.094945 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:18.094836 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-knssg" event={"ID":"c4d73d69-aaad-48c5-bfbc-75ac14352253","Type":"ContainerDied","Data":"5b843703a2aed85b2d1dc0bb9f85deb3bdbc5a5747b25b0754f88c605e61bdee"} May 26 15:21:18.098016 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:18.097985 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" event={"ID":"beeec92c-3a10-4520-a63d-984e02c0bbcf","Type":"ContainerStarted","Data":"8f509cbffd06d353249f8a04238c12699d547a0b619a677a48c851478ab667ae"} May 26 15:21:18.098173 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:18.098158 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-qcrp6" May 26 15:21:18.098369 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:18.098348 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:21:18.098467 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:18.098383 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:21:18.098848 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:18.098831 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qcrp6" May 26 15:21:18.114093 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:18.114076 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:21:18.137652 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:18.137612 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" podStartSLOduration=7.063224092 podStartE2EDuration="26.137600493s" podCreationTimestamp="2026-05-26 15:20:52 +0000 UTC" firstStartedPulling="2026-05-26 15:20:53.200807868 +0000 UTC m=+1.927482767" lastFinishedPulling="2026-05-26 15:21:12.275184269 +0000 UTC m=+21.001859168" observedRunningTime="2026-05-26 15:21:18.137400725 +0000 UTC m=+26.864075724" watchObservedRunningTime="2026-05-26 15:21:18.137600493 +0000 UTC m=+26.864275419" May 26 15:21:19.101831 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:19.101496 2547 generic.go:358] "Generic (PLEG): container finished" podID="c4d73d69-aaad-48c5-bfbc-75ac14352253" containerID="f035fab0cac5ecf96e996f96ee429d57790d2327796f15baa0b75f2f9527d87e" exitCode=0 May 26 15:21:19.101831 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:19.101579 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-knssg" event={"ID":"c4d73d69-aaad-48c5-bfbc-75ac14352253","Type":"ContainerDied","Data":"f035fab0cac5ecf96e996f96ee429d57790d2327796f15baa0b75f2f9527d87e"} May 26 15:21:19.103038 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:19.103019 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:21:19.120844 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:19.120820 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kkq9m"] May 26 15:21:19.120945 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:19.120933 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkq9m" May 26 15:21:19.121044 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:19.121021 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkq9m" podUID="033a1d5a-e393-4e8c-aa09-3786637f5a7e" May 26 15:21:19.121771 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:19.121752 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:21:19.126622 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:19.126604 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5xh82"] May 26 15:21:19.126693 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:19.126687 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xh82" May 26 15:21:19.126792 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:19.126766 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xh82" podUID="fac6c416-4fef-4b1b-846f-ea63a14c9efe" May 26 15:21:20.105687 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:20.105656 2547 generic.go:358] "Generic (PLEG): container finished" podID="c4d73d69-aaad-48c5-bfbc-75ac14352253" containerID="5755dc9c8ed267a348830544bdcafb9d230936cb773fa1e257b04f5500391746" exitCode=0 May 26 15:21:20.106111 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:20.105744 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-knssg" event={"ID":"c4d73d69-aaad-48c5-bfbc-75ac14352253","Type":"ContainerDied","Data":"5755dc9c8ed267a348830544bdcafb9d230936cb773fa1e257b04f5500391746"} May 26 15:21:20.909256 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:20.909214 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkq9m" May 26 15:21:20.909424 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:20.909339 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkq9m" podUID="033a1d5a-e393-4e8c-aa09-3786637f5a7e" May 26 15:21:20.909424 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:20.909225 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xh82" May 26 15:21:20.909539 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:20.909514 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xh82" podUID="fac6c416-4fef-4b1b-846f-ea63a14c9efe" May 26 15:21:22.909361 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:22.909329 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xh82" May 26 15:21:22.909749 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:22.909330 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkq9m" May 26 15:21:22.909749 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:22.909458 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xh82" podUID="fac6c416-4fef-4b1b-846f-ea63a14c9efe" May 26 15:21:22.909749 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:22.909531 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkq9m" podUID="033a1d5a-e393-4e8c-aa09-3786637f5a7e" May 26 15:21:24.551877 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:24.551839 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fac6c416-4fef-4b1b-846f-ea63a14c9efe-metrics-certs\") pod \"network-metrics-daemon-5xh82\" (UID: \"fac6c416-4fef-4b1b-846f-ea63a14c9efe\") " pod="openshift-multus/network-metrics-daemon-5xh82" May 26 15:21:24.552361 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:24.551993 2547 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 26 15:21:24.552361 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:24.552087 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fac6c416-4fef-4b1b-846f-ea63a14c9efe-metrics-certs podName:fac6c416-4fef-4b1b-846f-ea63a14c9efe nodeName:}" failed. No retries permitted until 2026-05-26 15:21:56.552047906 +0000 UTC m=+65.278722807 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fac6c416-4fef-4b1b-846f-ea63a14c9efe-metrics-certs") pod "network-metrics-daemon-5xh82" (UID: "fac6c416-4fef-4b1b-846f-ea63a14c9efe") : object "openshift-multus"/"metrics-daemon-secret" not registered May 26 15:21:24.753456 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:24.753236 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4lmmd\" (UniqueName: \"kubernetes.io/projected/033a1d5a-e393-4e8c-aa09-3786637f5a7e-kube-api-access-4lmmd\") pod \"network-check-target-kkq9m\" (UID: \"033a1d5a-e393-4e8c-aa09-3786637f5a7e\") " pod="openshift-network-diagnostics/network-check-target-kkq9m" May 26 15:21:24.753629 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:24.753375 2547 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 26 15:21:24.753629 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:24.753532 2547 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 26 15:21:24.753629 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:24.753549 2547 projected.go:194] Error preparing data for projected volume kube-api-access-4lmmd for pod openshift-network-diagnostics/network-check-target-kkq9m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 26 15:21:24.753629 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:24.753607 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/033a1d5a-e393-4e8c-aa09-3786637f5a7e-kube-api-access-4lmmd podName:033a1d5a-e393-4e8c-aa09-3786637f5a7e nodeName:}" failed. No retries permitted until 2026-05-26 15:21:56.75359378 +0000 UTC m=+65.480268680 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4lmmd" (UniqueName: "kubernetes.io/projected/033a1d5a-e393-4e8c-aa09-3786637f5a7e-kube-api-access-4lmmd") pod "network-check-target-kkq9m" (UID: "033a1d5a-e393-4e8c-aa09-3786637f5a7e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 26 15:21:24.909341 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:24.909271 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xh82" May 26 15:21:24.909495 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:24.909271 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkq9m" May 26 15:21:24.909495 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:24.909411 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5xh82" podUID="fac6c416-4fef-4b1b-846f-ea63a14c9efe" May 26 15:21:24.909589 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:24.909492 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkq9m" podUID="033a1d5a-e393-4e8c-aa09-3786637f5a7e" May 26 15:21:25.122381 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:25.122353 2547 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-198.ec2.internal" event="NodeReady" May 26 15:21:25.122587 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:25.122497 2547 kubelet_node_status.go:550] "Fast updating node status as it just became ready" May 26 15:21:25.164411 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:25.164347 2547 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-dzpbp"] May 26 15:21:25.191913 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:25.191887 2547 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-8sxg9"] May 26 15:21:25.192089 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:25.192046 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dzpbp" May 26 15:21:25.194288 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:25.194265 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" May 26 15:21:25.194420 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:25.194328 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-smt4m\"" May 26 15:21:25.194420 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:25.194327 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" May 26 15:21:25.205942 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:25.205924 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dzpbp"] May 26 15:21:25.206043 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:25.205946 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8sxg9"] May 26 15:21:25.206043 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:25.206025 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8sxg9" May 26 15:21:25.208023 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:25.208001 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" May 26 15:21:25.208129 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:25.208026 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-pt65f\"" May 26 15:21:25.208338 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:25.208319 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" May 26 15:21:25.208338 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:25.208327 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" May 26 15:21:25.357746 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:25.357716 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b1e268b-9140-4f73-8abf-c1c30fc706bc-cert\") pod \"ingress-canary-8sxg9\" (UID: \"7b1e268b-9140-4f73-8abf-c1c30fc706bc\") " pod="openshift-ingress-canary/ingress-canary-8sxg9" May 26 15:21:25.357746 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:25.357759 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91f902eb-040a-46bf-a271-e75d7e98479f-tmp-dir\") pod \"dns-default-dzpbp\" (UID: \"91f902eb-040a-46bf-a271-e75d7e98479f\") " pod="openshift-dns/dns-default-dzpbp" May 26 15:21:25.358048 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:25.357788 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/91f902eb-040a-46bf-a271-e75d7e98479f-config-volume\") pod \"dns-default-dzpbp\" (UID: \"91f902eb-040a-46bf-a271-e75d7e98479f\") " pod="openshift-dns/dns-default-dzpbp" May 26 15:21:25.358048 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:25.357816 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vmkx\" (UniqueName: \"kubernetes.io/projected/91f902eb-040a-46bf-a271-e75d7e98479f-kube-api-access-4vmkx\") pod \"dns-default-dzpbp\" (UID: \"91f902eb-040a-46bf-a271-e75d7e98479f\") " pod="openshift-dns/dns-default-dzpbp" May 26 15:21:25.358048 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:25.357905 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mm2l4\" (UniqueName: \"kubernetes.io/projected/7b1e268b-9140-4f73-8abf-c1c30fc706bc-kube-api-access-mm2l4\") pod \"ingress-canary-8sxg9\" (UID: \"7b1e268b-9140-4f73-8abf-c1c30fc706bc\") " pod="openshift-ingress-canary/ingress-canary-8sxg9" May 26 15:21:25.358048 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:25.357986 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/91f902eb-040a-46bf-a271-e75d7e98479f-metrics-tls\") pod \"dns-default-dzpbp\" (UID: \"91f902eb-040a-46bf-a271-e75d7e98479f\") " pod="openshift-dns/dns-default-dzpbp" May 26 15:21:25.459023 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:25.458989 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b1e268b-9140-4f73-8abf-c1c30fc706bc-cert\") pod \"ingress-canary-8sxg9\" (UID: \"7b1e268b-9140-4f73-8abf-c1c30fc706bc\") " pod="openshift-ingress-canary/ingress-canary-8sxg9" May 26 15:21:25.459229 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:25.459034 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91f902eb-040a-46bf-a271-e75d7e98479f-tmp-dir\") pod \"dns-default-dzpbp\" (UID: \"91f902eb-040a-46bf-a271-e75d7e98479f\") " pod="openshift-dns/dns-default-dzpbp" May 26 15:21:25.459229 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:25.459074 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/91f902eb-040a-46bf-a271-e75d7e98479f-config-volume\") pod \"dns-default-dzpbp\" (UID: \"91f902eb-040a-46bf-a271-e75d7e98479f\") " pod="openshift-dns/dns-default-dzpbp" May 26 15:21:25.459229 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:25.459102 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4vmkx\" (UniqueName: \"kubernetes.io/projected/91f902eb-040a-46bf-a271-e75d7e98479f-kube-api-access-4vmkx\") pod \"dns-default-dzpbp\" (UID: \"91f902eb-040a-46bf-a271-e75d7e98479f\") " pod="openshift-dns/dns-default-dzpbp" May 26 15:21:25.459229 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:25.459129 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mm2l4\" (UniqueName: \"kubernetes.io/projected/7b1e268b-9140-4f73-8abf-c1c30fc706bc-kube-api-access-mm2l4\") pod \"ingress-canary-8sxg9\" (UID: \"7b1e268b-9140-4f73-8abf-c1c30fc706bc\") " pod="openshift-ingress-canary/ingress-canary-8sxg9" May 26 15:21:25.459229 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:25.459185 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/91f902eb-040a-46bf-a271-e75d7e98479f-metrics-tls\") pod \"dns-default-dzpbp\" (UID: \"91f902eb-040a-46bf-a271-e75d7e98479f\") " pod="openshift-dns/dns-default-dzpbp" May 26 15:21:25.459229 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:25.459196 2547 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 26 15:21:25.459555 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:25.459278 2547 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 26 15:21:25.459555 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:25.459282 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b1e268b-9140-4f73-8abf-c1c30fc706bc-cert podName:7b1e268b-9140-4f73-8abf-c1c30fc706bc nodeName:}" failed. No retries permitted until 2026-05-26 15:21:25.959259942 +0000 UTC m=+34.685934857 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7b1e268b-9140-4f73-8abf-c1c30fc706bc-cert") pod "ingress-canary-8sxg9" (UID: "7b1e268b-9140-4f73-8abf-c1c30fc706bc") : secret "canary-serving-cert" not found May 26 15:21:25.459555 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:25.459376 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91f902eb-040a-46bf-a271-e75d7e98479f-metrics-tls podName:91f902eb-040a-46bf-a271-e75d7e98479f nodeName:}" failed. No retries permitted until 2026-05-26 15:21:25.959364103 +0000 UTC m=+34.686039007 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/91f902eb-040a-46bf-a271-e75d7e98479f-metrics-tls") pod "dns-default-dzpbp" (UID: "91f902eb-040a-46bf-a271-e75d7e98479f") : secret "dns-default-metrics-tls" not found May 26 15:21:25.459555 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:25.459426 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91f902eb-040a-46bf-a271-e75d7e98479f-tmp-dir\") pod \"dns-default-dzpbp\" (UID: \"91f902eb-040a-46bf-a271-e75d7e98479f\") " pod="openshift-dns/dns-default-dzpbp" May 26 15:21:25.459915 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:25.459884 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/91f902eb-040a-46bf-a271-e75d7e98479f-config-volume\") pod \"dns-default-dzpbp\" (UID: \"91f902eb-040a-46bf-a271-e75d7e98479f\") " pod="openshift-dns/dns-default-dzpbp" May 26 15:21:25.469850 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:25.469824 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vmkx\" (UniqueName: \"kubernetes.io/projected/91f902eb-040a-46bf-a271-e75d7e98479f-kube-api-access-4vmkx\") pod \"dns-default-dzpbp\" (UID: \"91f902eb-040a-46bf-a271-e75d7e98479f\") " pod="openshift-dns/dns-default-dzpbp" May 26 15:21:25.469992 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:25.469834 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mm2l4\" (UniqueName: \"kubernetes.io/projected/7b1e268b-9140-4f73-8abf-c1c30fc706bc-kube-api-access-mm2l4\") pod \"ingress-canary-8sxg9\" (UID: \"7b1e268b-9140-4f73-8abf-c1c30fc706bc\") " pod="openshift-ingress-canary/ingress-canary-8sxg9" May 26 15:21:25.964095 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:25.964049 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/91f902eb-040a-46bf-a271-e75d7e98479f-metrics-tls\") pod \"dns-default-dzpbp\" (UID: \"91f902eb-040a-46bf-a271-e75d7e98479f\") " pod="openshift-dns/dns-default-dzpbp" May 26 15:21:25.964599 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:25.964117 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b1e268b-9140-4f73-8abf-c1c30fc706bc-cert\") pod \"ingress-canary-8sxg9\" (UID: \"7b1e268b-9140-4f73-8abf-c1c30fc706bc\") " pod="openshift-ingress-canary/ingress-canary-8sxg9" May 26 15:21:25.964599 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:25.964206 2547 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 26 15:21:25.964599 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:25.964216 2547 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 26 15:21:25.964599 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:25.964272 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91f902eb-040a-46bf-a271-e75d7e98479f-metrics-tls podName:91f902eb-040a-46bf-a271-e75d7e98479f nodeName:}" failed. No retries permitted until 2026-05-26 15:21:26.964254759 +0000 UTC m=+35.690929659 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/91f902eb-040a-46bf-a271-e75d7e98479f-metrics-tls") pod "dns-default-dzpbp" (UID: "91f902eb-040a-46bf-a271-e75d7e98479f") : secret "dns-default-metrics-tls" not found May 26 15:21:25.964599 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:25.964291 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b1e268b-9140-4f73-8abf-c1c30fc706bc-cert podName:7b1e268b-9140-4f73-8abf-c1c30fc706bc nodeName:}" failed. No retries permitted until 2026-05-26 15:21:26.964285245 +0000 UTC m=+35.690960144 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7b1e268b-9140-4f73-8abf-c1c30fc706bc-cert") pod "ingress-canary-8sxg9" (UID: "7b1e268b-9140-4f73-8abf-c1c30fc706bc") : secret "canary-serving-cert" not found May 26 15:21:26.909498 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:26.909467 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xh82" May 26 15:21:26.909652 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:26.909467 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkq9m" May 26 15:21:26.911973 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:26.911950 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" May 26 15:21:26.912119 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:26.912018 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pgdkb\"" May 26 15:21:26.912119 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:26.911956 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" May 26 15:21:26.914970 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:26.912471 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" May 26 15:21:26.914970 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:26.912771 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dzb4k\"" May 26 15:21:26.970522 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:26.970497 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b1e268b-9140-4f73-8abf-c1c30fc706bc-cert\") pod \"ingress-canary-8sxg9\" (UID: \"7b1e268b-9140-4f73-8abf-c1c30fc706bc\") " pod="openshift-ingress-canary/ingress-canary-8sxg9" May 26 15:21:26.970865 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:26.970558 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/91f902eb-040a-46bf-a271-e75d7e98479f-metrics-tls\") pod \"dns-default-dzpbp\" (UID: \"91f902eb-040a-46bf-a271-e75d7e98479f\") " pod="openshift-dns/dns-default-dzpbp" May 26 15:21:26.970865 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:26.970634 2547 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 26 15:21:26.970865 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:26.970634 2547 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 26 15:21:26.970865 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:26.970679 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91f902eb-040a-46bf-a271-e75d7e98479f-metrics-tls podName:91f902eb-040a-46bf-a271-e75d7e98479f nodeName:}" failed. No retries permitted until 2026-05-26 15:21:28.970666397 +0000 UTC m=+37.697341296 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/91f902eb-040a-46bf-a271-e75d7e98479f-metrics-tls") pod "dns-default-dzpbp" (UID: "91f902eb-040a-46bf-a271-e75d7e98479f") : secret "dns-default-metrics-tls" not found May 26 15:21:26.970865 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:26.970691 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b1e268b-9140-4f73-8abf-c1c30fc706bc-cert podName:7b1e268b-9140-4f73-8abf-c1c30fc706bc nodeName:}" failed. No retries permitted until 2026-05-26 15:21:28.970685781 +0000 UTC m=+37.697360680 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7b1e268b-9140-4f73-8abf-c1c30fc706bc-cert") pod "ingress-canary-8sxg9" (UID: "7b1e268b-9140-4f73-8abf-c1c30fc706bc") : secret "canary-serving-cert" not found May 26 15:21:27.120864 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:27.120839 2547 generic.go:358] "Generic (PLEG): container finished" podID="c4d73d69-aaad-48c5-bfbc-75ac14352253" containerID="7f1f16c4e553cb5a4a59e6c37ba2e7024090413b3b200ed5b0421936035e15eb" exitCode=0 May 26 15:21:27.120988 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:27.120897 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-knssg" event={"ID":"c4d73d69-aaad-48c5-bfbc-75ac14352253","Type":"ContainerDied","Data":"7f1f16c4e553cb5a4a59e6c37ba2e7024090413b3b200ed5b0421936035e15eb"} May 26 15:21:28.125104 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:28.125076 2547 generic.go:358] "Generic (PLEG): container finished" podID="c4d73d69-aaad-48c5-bfbc-75ac14352253" containerID="d5cff7b7cb6b96024613ec86dd9ffa2832457007045a141c2ac1490a4b2af582" exitCode=0 May 26 15:21:28.125656 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:28.125135 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-knssg" event={"ID":"c4d73d69-aaad-48c5-bfbc-75ac14352253","Type":"ContainerDied","Data":"d5cff7b7cb6b96024613ec86dd9ffa2832457007045a141c2ac1490a4b2af582"} May 26 15:21:28.984767 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:28.984735 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b1e268b-9140-4f73-8abf-c1c30fc706bc-cert\") pod \"ingress-canary-8sxg9\" (UID: \"7b1e268b-9140-4f73-8abf-c1c30fc706bc\") " pod="openshift-ingress-canary/ingress-canary-8sxg9" May 26 15:21:28.984939 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:28.984806 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/91f902eb-040a-46bf-a271-e75d7e98479f-metrics-tls\") pod \"dns-default-dzpbp\" (UID: \"91f902eb-040a-46bf-a271-e75d7e98479f\") " pod="openshift-dns/dns-default-dzpbp" May 26 15:21:28.984939 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:28.984898 2547 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 26 15:21:28.985009 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:28.984968 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b1e268b-9140-4f73-8abf-c1c30fc706bc-cert podName:7b1e268b-9140-4f73-8abf-c1c30fc706bc nodeName:}" failed. No retries permitted until 2026-05-26 15:21:32.984951873 +0000 UTC m=+41.711626772 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7b1e268b-9140-4f73-8abf-c1c30fc706bc-cert") pod "ingress-canary-8sxg9" (UID: "7b1e268b-9140-4f73-8abf-c1c30fc706bc") : secret "canary-serving-cert" not found May 26 15:21:28.985009 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:28.984904 2547 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 26 15:21:28.985113 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:28.985042 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91f902eb-040a-46bf-a271-e75d7e98479f-metrics-tls podName:91f902eb-040a-46bf-a271-e75d7e98479f nodeName:}" failed. No retries permitted until 2026-05-26 15:21:32.985028953 +0000 UTC m=+41.711703853 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/91f902eb-040a-46bf-a271-e75d7e98479f-metrics-tls") pod "dns-default-dzpbp" (UID: "91f902eb-040a-46bf-a271-e75d7e98479f") : secret "dns-default-metrics-tls" not found May 26 15:21:29.130659 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:29.130617 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-knssg" event={"ID":"c4d73d69-aaad-48c5-bfbc-75ac14352253","Type":"ContainerStarted","Data":"5adf177c7673eba71008bd236c9f945647d6bfbcc79c27cf1dce68ff5d620870"} May 26 15:21:29.152698 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:29.152653 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-knssg" podStartSLOduration=5.063119761 podStartE2EDuration="38.152639846s" podCreationTimestamp="2026-05-26 15:20:51 +0000 UTC" firstStartedPulling="2026-05-26 15:20:53.118924923 +0000 UTC m=+1.845599822" lastFinishedPulling="2026-05-26 15:21:26.208444994 +0000 UTC m=+34.935119907" observedRunningTime="2026-05-26 15:21:29.151625288 +0000 UTC m=+37.878300221" watchObservedRunningTime="2026-05-26 15:21:29.152639846 +0000 UTC m=+37.879314780" May 26 15:21:33.012723 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:33.012533 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b1e268b-9140-4f73-8abf-c1c30fc706bc-cert\") pod \"ingress-canary-8sxg9\" (UID: \"7b1e268b-9140-4f73-8abf-c1c30fc706bc\") " pod="openshift-ingress-canary/ingress-canary-8sxg9" May 26 15:21:33.013115 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:33.012762 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/91f902eb-040a-46bf-a271-e75d7e98479f-metrics-tls\") pod \"dns-default-dzpbp\" (UID: \"91f902eb-040a-46bf-a271-e75d7e98479f\") " pod="openshift-dns/dns-default-dzpbp" May 26 15:21:33.013115 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:33.012679 2547 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 26 15:21:33.013115 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:33.012836 2547 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 26 15:21:33.013115 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:33.012848 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b1e268b-9140-4f73-8abf-c1c30fc706bc-cert podName:7b1e268b-9140-4f73-8abf-c1c30fc706bc nodeName:}" failed. No retries permitted until 2026-05-26 15:21:41.012830411 +0000 UTC m=+49.739505330 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7b1e268b-9140-4f73-8abf-c1c30fc706bc-cert") pod "ingress-canary-8sxg9" (UID: "7b1e268b-9140-4f73-8abf-c1c30fc706bc") : secret "canary-serving-cert" not found May 26 15:21:33.013115 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:33.012874 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91f902eb-040a-46bf-a271-e75d7e98479f-metrics-tls podName:91f902eb-040a-46bf-a271-e75d7e98479f nodeName:}" failed. No retries permitted until 2026-05-26 15:21:41.012863581 +0000 UTC m=+49.739538480 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/91f902eb-040a-46bf-a271-e75d7e98479f-metrics-tls") pod "dns-default-dzpbp" (UID: "91f902eb-040a-46bf-a271-e75d7e98479f") : secret "dns-default-metrics-tls" not found May 26 15:21:41.072138 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:41.072098 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/91f902eb-040a-46bf-a271-e75d7e98479f-metrics-tls\") pod \"dns-default-dzpbp\" (UID: \"91f902eb-040a-46bf-a271-e75d7e98479f\") " pod="openshift-dns/dns-default-dzpbp" May 26 15:21:41.072658 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:41.072148 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b1e268b-9140-4f73-8abf-c1c30fc706bc-cert\") pod \"ingress-canary-8sxg9\" (UID: \"7b1e268b-9140-4f73-8abf-c1c30fc706bc\") " pod="openshift-ingress-canary/ingress-canary-8sxg9" May 26 15:21:41.072658 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:41.072252 2547 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 26 15:21:41.072658 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:41.072262 2547 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 26 15:21:41.072658 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:41.072313 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b1e268b-9140-4f73-8abf-c1c30fc706bc-cert podName:7b1e268b-9140-4f73-8abf-c1c30fc706bc nodeName:}" failed. No retries permitted until 2026-05-26 15:21:57.072298187 +0000 UTC m=+65.798973087 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7b1e268b-9140-4f73-8abf-c1c30fc706bc-cert") pod "ingress-canary-8sxg9" (UID: "7b1e268b-9140-4f73-8abf-c1c30fc706bc") : secret "canary-serving-cert" not found May 26 15:21:41.072658 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:41.072340 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91f902eb-040a-46bf-a271-e75d7e98479f-metrics-tls podName:91f902eb-040a-46bf-a271-e75d7e98479f nodeName:}" failed. No retries permitted until 2026-05-26 15:21:57.07232107 +0000 UTC m=+65.798995993 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/91f902eb-040a-46bf-a271-e75d7e98479f-metrics-tls") pod "dns-default-dzpbp" (UID: "91f902eb-040a-46bf-a271-e75d7e98479f") : secret "dns-default-metrics-tls" not found May 26 15:21:51.118257 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:51.118226 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vfdvw" May 26 15:21:56.571107 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:56.571070 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fac6c416-4fef-4b1b-846f-ea63a14c9efe-metrics-certs\") pod \"network-metrics-daemon-5xh82\" (UID: \"fac6c416-4fef-4b1b-846f-ea63a14c9efe\") " pod="openshift-multus/network-metrics-daemon-5xh82" May 26 15:21:56.573867 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:56.573848 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" May 26 15:21:56.581448 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:56.581432 2547 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found May 26 15:21:56.581510 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:56.581500 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fac6c416-4fef-4b1b-846f-ea63a14c9efe-metrics-certs podName:fac6c416-4fef-4b1b-846f-ea63a14c9efe nodeName:}" failed. No retries permitted until 2026-05-26 15:23:00.581480151 +0000 UTC m=+129.308155057 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fac6c416-4fef-4b1b-846f-ea63a14c9efe-metrics-certs") pod "network-metrics-daemon-5xh82" (UID: "fac6c416-4fef-4b1b-846f-ea63a14c9efe") : secret "metrics-daemon-secret" not found May 26 15:21:56.773098 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:56.773051 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4lmmd\" (UniqueName: \"kubernetes.io/projected/033a1d5a-e393-4e8c-aa09-3786637f5a7e-kube-api-access-4lmmd\") pod \"network-check-target-kkq9m\" (UID: \"033a1d5a-e393-4e8c-aa09-3786637f5a7e\") " pod="openshift-network-diagnostics/network-check-target-kkq9m" May 26 15:21:56.776165 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:56.776149 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" May 26 15:21:56.786048 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:56.786031 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" May 26 15:21:56.797525 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:56.797507 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lmmd\" (UniqueName: \"kubernetes.io/projected/033a1d5a-e393-4e8c-aa09-3786637f5a7e-kube-api-access-4lmmd\") pod \"network-check-target-kkq9m\" (UID: \"033a1d5a-e393-4e8c-aa09-3786637f5a7e\") " pod="openshift-network-diagnostics/network-check-target-kkq9m" May 26 15:21:56.928299 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:56.928280 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dzb4k\"" May 26 15:21:56.936285 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:56.936268 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkq9m" May 26 15:21:57.075659 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:57.075632 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/91f902eb-040a-46bf-a271-e75d7e98479f-metrics-tls\") pod \"dns-default-dzpbp\" (UID: \"91f902eb-040a-46bf-a271-e75d7e98479f\") " pod="openshift-dns/dns-default-dzpbp" May 26 15:21:57.075796 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:57.075676 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b1e268b-9140-4f73-8abf-c1c30fc706bc-cert\") pod \"ingress-canary-8sxg9\" (UID: \"7b1e268b-9140-4f73-8abf-c1c30fc706bc\") " pod="openshift-ingress-canary/ingress-canary-8sxg9" May 26 15:21:57.075796 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:57.075779 2547 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 26 15:21:57.075867 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:57.075834 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b1e268b-9140-4f73-8abf-c1c30fc706bc-cert podName:7b1e268b-9140-4f73-8abf-c1c30fc706bc nodeName:}" failed. No retries permitted until 2026-05-26 15:22:29.07582102 +0000 UTC m=+97.802495920 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7b1e268b-9140-4f73-8abf-c1c30fc706bc-cert") pod "ingress-canary-8sxg9" (UID: "7b1e268b-9140-4f73-8abf-c1c30fc706bc") : secret "canary-serving-cert" not found May 26 15:21:57.075867 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:57.075779 2547 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 26 15:21:57.075941 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:21:57.075897 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91f902eb-040a-46bf-a271-e75d7e98479f-metrics-tls podName:91f902eb-040a-46bf-a271-e75d7e98479f nodeName:}" failed. No retries permitted until 2026-05-26 15:22:29.075885757 +0000 UTC m=+97.802560666 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/91f902eb-040a-46bf-a271-e75d7e98479f-metrics-tls") pod "dns-default-dzpbp" (UID: "91f902eb-040a-46bf-a271-e75d7e98479f") : secret "dns-default-metrics-tls" not found May 26 15:21:57.097619 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:57.097578 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kkq9m"] May 26 15:21:57.101301 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:21:57.101272 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod033a1d5a_e393_4e8c_aa09_3786637f5a7e.slice/crio-6693eb2982c7bf29d22a48edc520caf1abddc8fcf94df1526100a4d03b108c15 WatchSource:0}: Error finding container 6693eb2982c7bf29d22a48edc520caf1abddc8fcf94df1526100a4d03b108c15: Status 404 returned error can't find the container with id 6693eb2982c7bf29d22a48edc520caf1abddc8fcf94df1526100a4d03b108c15 May 26 15:21:57.181199 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:21:57.181126 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kkq9m" event={"ID":"033a1d5a-e393-4e8c-aa09-3786637f5a7e","Type":"ContainerStarted","Data":"6693eb2982c7bf29d22a48edc520caf1abddc8fcf94df1526100a4d03b108c15"} May 26 15:22:00.188496 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:22:00.188460 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kkq9m" event={"ID":"033a1d5a-e393-4e8c-aa09-3786637f5a7e","Type":"ContainerStarted","Data":"6112ce9eb89820c232f6e531653f4f536f1d847ba724edabf3f6f9ba80d4a0b4"} May 26 15:22:00.188863 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:22:00.188620 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-kkq9m" May 26 15:22:29.086998 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:22:29.086962 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b1e268b-9140-4f73-8abf-c1c30fc706bc-cert\") pod \"ingress-canary-8sxg9\" (UID: \"7b1e268b-9140-4f73-8abf-c1c30fc706bc\") " pod="openshift-ingress-canary/ingress-canary-8sxg9" May 26 15:22:29.087375 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:22:29.087037 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/91f902eb-040a-46bf-a271-e75d7e98479f-metrics-tls\") pod \"dns-default-dzpbp\" (UID: \"91f902eb-040a-46bf-a271-e75d7e98479f\") " pod="openshift-dns/dns-default-dzpbp" May 26 15:22:29.087375 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:22:29.087132 2547 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 26 15:22:29.087375 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:22:29.087189 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91f902eb-040a-46bf-a271-e75d7e98479f-metrics-tls podName:91f902eb-040a-46bf-a271-e75d7e98479f nodeName:}" failed. No retries permitted until 2026-05-26 15:23:33.087174592 +0000 UTC m=+161.813849491 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/91f902eb-040a-46bf-a271-e75d7e98479f-metrics-tls") pod "dns-default-dzpbp" (UID: "91f902eb-040a-46bf-a271-e75d7e98479f") : secret "dns-default-metrics-tls" not found May 26 15:22:29.087375 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:22:29.087132 2547 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 26 15:22:29.087375 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:22:29.087299 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b1e268b-9140-4f73-8abf-c1c30fc706bc-cert podName:7b1e268b-9140-4f73-8abf-c1c30fc706bc nodeName:}" failed. No retries permitted until 2026-05-26 15:23:33.087285668 +0000 UTC m=+161.813960585 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7b1e268b-9140-4f73-8abf-c1c30fc706bc-cert") pod "ingress-canary-8sxg9" (UID: "7b1e268b-9140-4f73-8abf-c1c30fc706bc") : secret "canary-serving-cert" not found May 26 15:22:31.192503 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:22:31.192473 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-kkq9m" May 26 15:22:31.209274 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:22:31.206891 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-kkq9m" podStartSLOduration=96.494824545 podStartE2EDuration="1m39.206874973s" podCreationTimestamp="2026-05-26 15:20:52 +0000 UTC" firstStartedPulling="2026-05-26 15:21:57.103150788 +0000 UTC m=+65.829825701" lastFinishedPulling="2026-05-26 15:21:59.815201216 +0000 UTC m=+68.541876129" observedRunningTime="2026-05-26 15:22:00.204422115 +0000 UTC m=+68.931097036" watchObservedRunningTime="2026-05-26 15:22:31.206874973 +0000 UTC m=+99.933549894" May 26 15:23:00.594407 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:00.594362 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fac6c416-4fef-4b1b-846f-ea63a14c9efe-metrics-certs\") pod \"network-metrics-daemon-5xh82\" (UID: \"fac6c416-4fef-4b1b-846f-ea63a14c9efe\") " pod="openshift-multus/network-metrics-daemon-5xh82" May 26 15:23:00.594903 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:23:00.594489 2547 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found May 26 15:23:00.594903 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:23:00.594544 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fac6c416-4fef-4b1b-846f-ea63a14c9efe-metrics-certs podName:fac6c416-4fef-4b1b-846f-ea63a14c9efe nodeName:}" failed. No retries permitted until 2026-05-26 15:25:02.594527947 +0000 UTC m=+251.321202846 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fac6c416-4fef-4b1b-846f-ea63a14c9efe-metrics-certs") pod "network-metrics-daemon-5xh82" (UID: "fac6c416-4fef-4b1b-846f-ea63a14c9efe") : secret "metrics-daemon-secret" not found May 26 15:23:13.830832 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.830796 2547 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-76dff48d59-qd2tx"] May 26 15:23:13.833726 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.833704 2547 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-klhzf"] May 26 15:23:13.833883 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.833862 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-76dff48d59-qd2tx" May 26 15:23:13.835813 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.835785 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" May 26 15:23:13.835922 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.835888 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" May 26 15:23:13.836269 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.836253 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" May 26 15:23:13.836338 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.836302 2547 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-mkzgw"] May 26 15:23:13.836396 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.836378 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" May 26 15:23:13.836464 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.836446 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-klhzf" May 26 15:23:13.836529 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.836301 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-r5rdx\"" May 26 15:23:13.838539 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.838520 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" May 26 15:23:13.838813 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.838781 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-822c6\"" May 26 15:23:13.838911 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.838817 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" May 26 15:23:13.838984 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.838907 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-mkzgw" May 26 15:23:13.841187 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.840776 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" May 26 15:23:13.841187 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.840864 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" May 26 15:23:13.841712 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.841595 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-99kps\"" May 26 15:23:13.842035 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.842013 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" May 26 15:23:13.842294 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.842276 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-klhzf"] May 26 15:23:13.846902 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.846883 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-mkzgw"] May 26 15:23:13.848015 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.847996 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-76dff48d59-qd2tx"] May 26 15:23:13.882828 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.882790 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dd56d3c2-38ff-4bd8-9893-6b643455a976-config\") pod \"console-operator-76dff48d59-qd2tx\" (UID: \"dd56d3c2-38ff-4bd8-9893-6b643455a976\") " pod="openshift-console-operator/console-operator-76dff48d59-qd2tx" May 26 15:23:13.882937 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.882843 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-mkzgw\" (UID: \"a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-mkzgw" May 26 15:23:13.882937 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.882908 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dd56d3c2-38ff-4bd8-9893-6b643455a976-trusted-ca\") pod \"console-operator-76dff48d59-qd2tx\" (UID: \"dd56d3c2-38ff-4bd8-9893-6b643455a976\") " pod="openshift-console-operator/console-operator-76dff48d59-qd2tx" May 26 15:23:13.883010 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.882968 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dd56d3c2-38ff-4bd8-9893-6b643455a976-serving-cert\") pod \"console-operator-76dff48d59-qd2tx\" (UID: \"dd56d3c2-38ff-4bd8-9893-6b643455a976\") " pod="openshift-console-operator/console-operator-76dff48d59-qd2tx" May 26 15:23:13.883046 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.883011 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwpxm\" (UniqueName: \"kubernetes.io/projected/463f7c87-7dc8-4b09-a855-6565923f3a8d-kube-api-access-zwpxm\") pod \"volume-data-source-validator-5cf8c89d64-klhzf\" (UID: \"463f7c87-7dc8-4b09-a855-6565923f3a8d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-klhzf" May 26 15:23:13.883046 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.883035 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-mkzgw\" (UID: \"a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-mkzgw" May 26 15:23:13.883160 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.883073 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hz7p\" (UniqueName: \"kubernetes.io/projected/dd56d3c2-38ff-4bd8-9893-6b643455a976-kube-api-access-5hz7p\") pod \"console-operator-76dff48d59-qd2tx\" (UID: \"dd56d3c2-38ff-4bd8-9893-6b643455a976\") " pod="openshift-console-operator/console-operator-76dff48d59-qd2tx" May 26 15:23:13.920254 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.920231 2547 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57b5b68b8-mrfrn"] May 26 15:23:13.922966 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.922943 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-mrfrn" May 26 15:23:13.930042 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.930026 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" May 26 15:23:13.930423 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.930408 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" May 26 15:23:13.931193 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.931177 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-rzj5f\"" May 26 15:23:13.932222 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.932208 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" May 26 15:23:13.941175 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.941156 2547 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7f44b6dd87-j9222"] May 26 15:23:13.944620 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.944439 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-j9222" May 26 15:23:13.951510 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.951487 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-vcdqv\"" May 26 15:23:13.955192 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.955173 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57b5b68b8-mrfrn"] May 26 15:23:13.970289 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.970268 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" May 26 15:23:13.980423 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.980389 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7f44b6dd87-j9222"] May 26 15:23:13.983777 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.983754 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dd56d3c2-38ff-4bd8-9893-6b643455a976-trusted-ca\") pod \"console-operator-76dff48d59-qd2tx\" (UID: \"dd56d3c2-38ff-4bd8-9893-6b643455a976\") " pod="openshift-console-operator/console-operator-76dff48d59-qd2tx" May 26 15:23:13.983845 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.983785 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c380575f-335e-4023-9b3a-277b8b5fcbbf-serving-cert\") pod \"service-ca-operator-57b5b68b8-mrfrn\" (UID: \"c380575f-335e-4023-9b3a-277b8b5fcbbf\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-mrfrn" May 26 15:23:13.983845 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.983809 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dd56d3c2-38ff-4bd8-9893-6b643455a976-serving-cert\") pod \"console-operator-76dff48d59-qd2tx\" (UID: \"dd56d3c2-38ff-4bd8-9893-6b643455a976\") " pod="openshift-console-operator/console-operator-76dff48d59-qd2tx" May 26 15:23:13.983845 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.983827 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrj8w\" (UniqueName: \"kubernetes.io/projected/ce0ae8cb-e08f-491d-a674-b60312b575d7-kube-api-access-vrj8w\") pod \"network-check-source-7f44b6dd87-j9222\" (UID: \"ce0ae8cb-e08f-491d-a674-b60312b575d7\") " pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-j9222" May 26 15:23:13.983982 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.983957 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnv6m\" (UniqueName: \"kubernetes.io/projected/c380575f-335e-4023-9b3a-277b8b5fcbbf-kube-api-access-jnv6m\") pod \"service-ca-operator-57b5b68b8-mrfrn\" (UID: \"c380575f-335e-4023-9b3a-277b8b5fcbbf\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-mrfrn" May 26 15:23:13.984016 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.984006 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zwpxm\" (UniqueName: \"kubernetes.io/projected/463f7c87-7dc8-4b09-a855-6565923f3a8d-kube-api-access-zwpxm\") pod \"volume-data-source-validator-5cf8c89d64-klhzf\" (UID: \"463f7c87-7dc8-4b09-a855-6565923f3a8d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-klhzf" May 26 15:23:13.984051 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.984036 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-mkzgw\" (UID: \"a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-mkzgw" May 26 15:23:13.984107 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.984084 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5hz7p\" (UniqueName: \"kubernetes.io/projected/dd56d3c2-38ff-4bd8-9893-6b643455a976-kube-api-access-5hz7p\") pod \"console-operator-76dff48d59-qd2tx\" (UID: \"dd56d3c2-38ff-4bd8-9893-6b643455a976\") " pod="openshift-console-operator/console-operator-76dff48d59-qd2tx" May 26 15:23:13.984140 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.984117 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c380575f-335e-4023-9b3a-277b8b5fcbbf-config\") pod \"service-ca-operator-57b5b68b8-mrfrn\" (UID: \"c380575f-335e-4023-9b3a-277b8b5fcbbf\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-mrfrn" May 26 15:23:13.984188 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.984174 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dd56d3c2-38ff-4bd8-9893-6b643455a976-config\") pod \"console-operator-76dff48d59-qd2tx\" (UID: \"dd56d3c2-38ff-4bd8-9893-6b643455a976\") " pod="openshift-console-operator/console-operator-76dff48d59-qd2tx" May 26 15:23:13.984239 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.984211 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-mkzgw\" (UID: \"a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-mkzgw" May 26 15:23:13.984286 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:23:13.984247 2547 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found May 26 15:23:13.984357 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:23:13.984343 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41-networking-console-plugin-cert podName:a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41 nodeName:}" failed. No retries permitted until 2026-05-26 15:23:14.484325356 +0000 UTC m=+143.211000258 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-mkzgw" (UID: "a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41") : secret "networking-console-plugin-cert" not found May 26 15:23:13.984856 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.984836 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dd56d3c2-38ff-4bd8-9893-6b643455a976-trusted-ca\") pod \"console-operator-76dff48d59-qd2tx\" (UID: \"dd56d3c2-38ff-4bd8-9893-6b643455a976\") " pod="openshift-console-operator/console-operator-76dff48d59-qd2tx" May 26 15:23:13.984921 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.984864 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dd56d3c2-38ff-4bd8-9893-6b643455a976-config\") pod \"console-operator-76dff48d59-qd2tx\" (UID: \"dd56d3c2-38ff-4bd8-9893-6b643455a976\") " pod="openshift-console-operator/console-operator-76dff48d59-qd2tx" May 26 15:23:13.984956 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.984928 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-mkzgw\" (UID: \"a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-mkzgw" May 26 15:23:13.986383 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.986362 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dd56d3c2-38ff-4bd8-9893-6b643455a976-serving-cert\") pod \"console-operator-76dff48d59-qd2tx\" (UID: \"dd56d3c2-38ff-4bd8-9893-6b643455a976\") " pod="openshift-console-operator/console-operator-76dff48d59-qd2tx" May 26 15:23:13.996586 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.996560 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwpxm\" (UniqueName: \"kubernetes.io/projected/463f7c87-7dc8-4b09-a855-6565923f3a8d-kube-api-access-zwpxm\") pod \"volume-data-source-validator-5cf8c89d64-klhzf\" (UID: \"463f7c87-7dc8-4b09-a855-6565923f3a8d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-klhzf" May 26 15:23:13.996832 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:13.996816 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hz7p\" (UniqueName: \"kubernetes.io/projected/dd56d3c2-38ff-4bd8-9893-6b643455a976-kube-api-access-5hz7p\") pod \"console-operator-76dff48d59-qd2tx\" (UID: \"dd56d3c2-38ff-4bd8-9893-6b643455a976\") " pod="openshift-console-operator/console-operator-76dff48d59-qd2tx" May 26 15:23:14.019005 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.018981 2547 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-78bdfb888c-ht62j"] May 26 15:23:14.022925 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.022908 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-78bdfb888c-ht62j" May 26 15:23:14.024838 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.024819 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-vjsf8\"" May 26 15:23:14.024922 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.024846 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" May 26 15:23:14.024922 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.024905 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" May 26 15:23:14.024922 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.024916 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" May 26 15:23:14.025148 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.025132 2547 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-69965b4854-82hkh"] May 26 15:23:14.025296 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.025282 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" May 26 15:23:14.028204 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.028190 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-69965b4854-82hkh" May 26 15:23:14.029649 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.029628 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" May 26 15:23:14.030471 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.030451 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" May 26 15:23:14.030471 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.030463 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" May 26 15:23:14.030649 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.030527 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" May 26 15:23:14.030649 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.030545 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" May 26 15:23:14.030649 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.030608 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-chnn7\"" May 26 15:23:14.030649 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.030457 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" May 26 15:23:14.030874 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.030720 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" May 26 15:23:14.031440 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.031420 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-78bdfb888c-ht62j"] May 26 15:23:14.036092 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.036071 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-69965b4854-82hkh"] May 26 15:23:14.084804 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.084747 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vrj8w\" (UniqueName: \"kubernetes.io/projected/ce0ae8cb-e08f-491d-a674-b60312b575d7-kube-api-access-vrj8w\") pod \"network-check-source-7f44b6dd87-j9222\" (UID: \"ce0ae8cb-e08f-491d-a674-b60312b575d7\") " pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-j9222" May 26 15:23:14.084804 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.084787 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49tqw\" (UniqueName: \"kubernetes.io/projected/90be60c1-e9f6-476d-9a27-f050b27cff15-kube-api-access-49tqw\") pod \"insights-operator-78bdfb888c-ht62j\" (UID: \"90be60c1-e9f6-476d-9a27-f050b27cff15\") " pod="openshift-insights/insights-operator-78bdfb888c-ht62j" May 26 15:23:14.084972 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.084840 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jnv6m\" (UniqueName: \"kubernetes.io/projected/c380575f-335e-4023-9b3a-277b8b5fcbbf-kube-api-access-jnv6m\") pod \"service-ca-operator-57b5b68b8-mrfrn\" (UID: \"c380575f-335e-4023-9b3a-277b8b5fcbbf\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-mrfrn" May 26 15:23:14.084972 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.084866 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/90be60c1-e9f6-476d-9a27-f050b27cff15-serving-cert\") pod \"insights-operator-78bdfb888c-ht62j\" (UID: \"90be60c1-e9f6-476d-9a27-f050b27cff15\") " pod="openshift-insights/insights-operator-78bdfb888c-ht62j" May 26 15:23:14.084972 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.084892 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f9b2322f-cdfd-43b7-8481-fe9c08706d91-default-certificate\") pod \"router-default-69965b4854-82hkh\" (UID: \"f9b2322f-cdfd-43b7-8481-fe9c08706d91\") " pod="openshift-ingress/router-default-69965b4854-82hkh" May 26 15:23:14.085163 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.085071 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f9b2322f-cdfd-43b7-8481-fe9c08706d91-stats-auth\") pod \"router-default-69965b4854-82hkh\" (UID: \"f9b2322f-cdfd-43b7-8481-fe9c08706d91\") " pod="openshift-ingress/router-default-69965b4854-82hkh" May 26 15:23:14.085163 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.085122 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90be60c1-e9f6-476d-9a27-f050b27cff15-tmp\") pod \"insights-operator-78bdfb888c-ht62j\" (UID: \"90be60c1-e9f6-476d-9a27-f050b27cff15\") " pod="openshift-insights/insights-operator-78bdfb888c-ht62j" May 26 15:23:14.085163 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.085141 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90be60c1-e9f6-476d-9a27-f050b27cff15-service-ca-bundle\") pod \"insights-operator-78bdfb888c-ht62j\" (UID: \"90be60c1-e9f6-476d-9a27-f050b27cff15\") " pod="openshift-insights/insights-operator-78bdfb888c-ht62j" May 26 15:23:14.085163 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.085158 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9b2322f-cdfd-43b7-8481-fe9c08706d91-service-ca-bundle\") pod \"router-default-69965b4854-82hkh\" (UID: \"f9b2322f-cdfd-43b7-8481-fe9c08706d91\") " pod="openshift-ingress/router-default-69965b4854-82hkh" May 26 15:23:14.085329 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.085184 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c380575f-335e-4023-9b3a-277b8b5fcbbf-config\") pod \"service-ca-operator-57b5b68b8-mrfrn\" (UID: \"c380575f-335e-4023-9b3a-277b8b5fcbbf\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-mrfrn" May 26 15:23:14.085329 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.085221 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdbfn\" (UniqueName: \"kubernetes.io/projected/f9b2322f-cdfd-43b7-8481-fe9c08706d91-kube-api-access-fdbfn\") pod \"router-default-69965b4854-82hkh\" (UID: \"f9b2322f-cdfd-43b7-8481-fe9c08706d91\") " pod="openshift-ingress/router-default-69965b4854-82hkh" May 26 15:23:14.085329 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.085251 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/90be60c1-e9f6-476d-9a27-f050b27cff15-snapshots\") pod \"insights-operator-78bdfb888c-ht62j\" (UID: \"90be60c1-e9f6-476d-9a27-f050b27cff15\") " pod="openshift-insights/insights-operator-78bdfb888c-ht62j" May 26 15:23:14.085329 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.085302 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90be60c1-e9f6-476d-9a27-f050b27cff15-trusted-ca-bundle\") pod \"insights-operator-78bdfb888c-ht62j\" (UID: \"90be60c1-e9f6-476d-9a27-f050b27cff15\") " pod="openshift-insights/insights-operator-78bdfb888c-ht62j" May 26 15:23:14.085489 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.085350 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c380575f-335e-4023-9b3a-277b8b5fcbbf-serving-cert\") pod \"service-ca-operator-57b5b68b8-mrfrn\" (UID: \"c380575f-335e-4023-9b3a-277b8b5fcbbf\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-mrfrn" May 26 15:23:14.085489 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.085383 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9b2322f-cdfd-43b7-8481-fe9c08706d91-metrics-certs\") pod \"router-default-69965b4854-82hkh\" (UID: \"f9b2322f-cdfd-43b7-8481-fe9c08706d91\") " pod="openshift-ingress/router-default-69965b4854-82hkh" May 26 15:23:14.085675 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.085660 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c380575f-335e-4023-9b3a-277b8b5fcbbf-config\") pod \"service-ca-operator-57b5b68b8-mrfrn\" (UID: \"c380575f-335e-4023-9b3a-277b8b5fcbbf\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-mrfrn" May 26 15:23:14.087294 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.087276 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c380575f-335e-4023-9b3a-277b8b5fcbbf-serving-cert\") pod \"service-ca-operator-57b5b68b8-mrfrn\" (UID: \"c380575f-335e-4023-9b3a-277b8b5fcbbf\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-mrfrn" May 26 15:23:14.092111 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.092094 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrj8w\" (UniqueName: \"kubernetes.io/projected/ce0ae8cb-e08f-491d-a674-b60312b575d7-kube-api-access-vrj8w\") pod \"network-check-source-7f44b6dd87-j9222\" (UID: \"ce0ae8cb-e08f-491d-a674-b60312b575d7\") " pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-j9222" May 26 15:23:14.092186 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.092144 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnv6m\" (UniqueName: \"kubernetes.io/projected/c380575f-335e-4023-9b3a-277b8b5fcbbf-kube-api-access-jnv6m\") pod \"service-ca-operator-57b5b68b8-mrfrn\" (UID: \"c380575f-335e-4023-9b3a-277b8b5fcbbf\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-mrfrn" May 26 15:23:14.145194 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.145175 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-76dff48d59-qd2tx" May 26 15:23:14.151825 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.151802 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-klhzf" May 26 15:23:14.186496 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.186463 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/90be60c1-e9f6-476d-9a27-f050b27cff15-serving-cert\") pod \"insights-operator-78bdfb888c-ht62j\" (UID: \"90be60c1-e9f6-476d-9a27-f050b27cff15\") " pod="openshift-insights/insights-operator-78bdfb888c-ht62j" May 26 15:23:14.186666 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.186510 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f9b2322f-cdfd-43b7-8481-fe9c08706d91-default-certificate\") pod \"router-default-69965b4854-82hkh\" (UID: \"f9b2322f-cdfd-43b7-8481-fe9c08706d91\") " pod="openshift-ingress/router-default-69965b4854-82hkh" May 26 15:23:14.186666 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.186544 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f9b2322f-cdfd-43b7-8481-fe9c08706d91-stats-auth\") pod \"router-default-69965b4854-82hkh\" (UID: \"f9b2322f-cdfd-43b7-8481-fe9c08706d91\") " pod="openshift-ingress/router-default-69965b4854-82hkh" May 26 15:23:14.186666 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.186584 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90be60c1-e9f6-476d-9a27-f050b27cff15-tmp\") pod \"insights-operator-78bdfb888c-ht62j\" (UID: \"90be60c1-e9f6-476d-9a27-f050b27cff15\") " pod="openshift-insights/insights-operator-78bdfb888c-ht62j" May 26 15:23:14.186666 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.186611 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90be60c1-e9f6-476d-9a27-f050b27cff15-service-ca-bundle\") pod \"insights-operator-78bdfb888c-ht62j\" (UID: \"90be60c1-e9f6-476d-9a27-f050b27cff15\") " pod="openshift-insights/insights-operator-78bdfb888c-ht62j" May 26 15:23:14.186666 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.186635 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9b2322f-cdfd-43b7-8481-fe9c08706d91-service-ca-bundle\") pod \"router-default-69965b4854-82hkh\" (UID: \"f9b2322f-cdfd-43b7-8481-fe9c08706d91\") " pod="openshift-ingress/router-default-69965b4854-82hkh" May 26 15:23:14.186917 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.186675 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fdbfn\" (UniqueName: \"kubernetes.io/projected/f9b2322f-cdfd-43b7-8481-fe9c08706d91-kube-api-access-fdbfn\") pod \"router-default-69965b4854-82hkh\" (UID: \"f9b2322f-cdfd-43b7-8481-fe9c08706d91\") " pod="openshift-ingress/router-default-69965b4854-82hkh" May 26 15:23:14.186917 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.186726 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/90be60c1-e9f6-476d-9a27-f050b27cff15-snapshots\") pod \"insights-operator-78bdfb888c-ht62j\" (UID: \"90be60c1-e9f6-476d-9a27-f050b27cff15\") " pod="openshift-insights/insights-operator-78bdfb888c-ht62j" May 26 15:23:14.186917 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.186766 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90be60c1-e9f6-476d-9a27-f050b27cff15-trusted-ca-bundle\") pod \"insights-operator-78bdfb888c-ht62j\" (UID: \"90be60c1-e9f6-476d-9a27-f050b27cff15\") " pod="openshift-insights/insights-operator-78bdfb888c-ht62j" May 26 15:23:14.186917 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.186812 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9b2322f-cdfd-43b7-8481-fe9c08706d91-metrics-certs\") pod \"router-default-69965b4854-82hkh\" (UID: \"f9b2322f-cdfd-43b7-8481-fe9c08706d91\") " pod="openshift-ingress/router-default-69965b4854-82hkh" May 26 15:23:14.186917 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.186849 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-49tqw\" (UniqueName: \"kubernetes.io/projected/90be60c1-e9f6-476d-9a27-f050b27cff15-kube-api-access-49tqw\") pod \"insights-operator-78bdfb888c-ht62j\" (UID: \"90be60c1-e9f6-476d-9a27-f050b27cff15\") " pod="openshift-insights/insights-operator-78bdfb888c-ht62j" May 26 15:23:14.189004 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:23:14.187360 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f9b2322f-cdfd-43b7-8481-fe9c08706d91-service-ca-bundle podName:f9b2322f-cdfd-43b7-8481-fe9c08706d91 nodeName:}" failed. No retries permitted until 2026-05-26 15:23:14.687339418 +0000 UTC m=+143.414014333 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f9b2322f-cdfd-43b7-8481-fe9c08706d91-service-ca-bundle") pod "router-default-69965b4854-82hkh" (UID: "f9b2322f-cdfd-43b7-8481-fe9c08706d91") : configmap references non-existent config key: service-ca.crt May 26 15:23:14.189004 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:23:14.187805 2547 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 26 15:23:14.189004 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.187838 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90be60c1-e9f6-476d-9a27-f050b27cff15-service-ca-bundle\") pod \"insights-operator-78bdfb888c-ht62j\" (UID: \"90be60c1-e9f6-476d-9a27-f050b27cff15\") " pod="openshift-insights/insights-operator-78bdfb888c-ht62j" May 26 15:23:14.189004 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:23:14.187871 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9b2322f-cdfd-43b7-8481-fe9c08706d91-metrics-certs podName:f9b2322f-cdfd-43b7-8481-fe9c08706d91 nodeName:}" failed. No retries permitted until 2026-05-26 15:23:14.687854334 +0000 UTC m=+143.414529250 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f9b2322f-cdfd-43b7-8481-fe9c08706d91-metrics-certs") pod "router-default-69965b4854-82hkh" (UID: "f9b2322f-cdfd-43b7-8481-fe9c08706d91") : secret "router-metrics-certs-default" not found May 26 15:23:14.189004 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.188390 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/90be60c1-e9f6-476d-9a27-f050b27cff15-snapshots\") pod \"insights-operator-78bdfb888c-ht62j\" (UID: \"90be60c1-e9f6-476d-9a27-f050b27cff15\") " pod="openshift-insights/insights-operator-78bdfb888c-ht62j" May 26 15:23:14.189004 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.188424 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90be60c1-e9f6-476d-9a27-f050b27cff15-trusted-ca-bundle\") pod \"insights-operator-78bdfb888c-ht62j\" (UID: \"90be60c1-e9f6-476d-9a27-f050b27cff15\") " pod="openshift-insights/insights-operator-78bdfb888c-ht62j" May 26 15:23:14.190917 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.190896 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/90be60c1-e9f6-476d-9a27-f050b27cff15-serving-cert\") pod \"insights-operator-78bdfb888c-ht62j\" (UID: \"90be60c1-e9f6-476d-9a27-f050b27cff15\") " pod="openshift-insights/insights-operator-78bdfb888c-ht62j" May 26 15:23:14.191265 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.191246 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90be60c1-e9f6-476d-9a27-f050b27cff15-tmp\") pod \"insights-operator-78bdfb888c-ht62j\" (UID: \"90be60c1-e9f6-476d-9a27-f050b27cff15\") " pod="openshift-insights/insights-operator-78bdfb888c-ht62j" May 26 15:23:14.192287 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.192243 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f9b2322f-cdfd-43b7-8481-fe9c08706d91-default-certificate\") pod \"router-default-69965b4854-82hkh\" (UID: \"f9b2322f-cdfd-43b7-8481-fe9c08706d91\") " pod="openshift-ingress/router-default-69965b4854-82hkh" May 26 15:23:14.192420 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.192395 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f9b2322f-cdfd-43b7-8481-fe9c08706d91-stats-auth\") pod \"router-default-69965b4854-82hkh\" (UID: \"f9b2322f-cdfd-43b7-8481-fe9c08706d91\") " pod="openshift-ingress/router-default-69965b4854-82hkh" May 26 15:23:14.194790 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.194772 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49tqw\" (UniqueName: \"kubernetes.io/projected/90be60c1-e9f6-476d-9a27-f050b27cff15-kube-api-access-49tqw\") pod \"insights-operator-78bdfb888c-ht62j\" (UID: \"90be60c1-e9f6-476d-9a27-f050b27cff15\") " pod="openshift-insights/insights-operator-78bdfb888c-ht62j" May 26 15:23:14.195265 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.195248 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdbfn\" (UniqueName: \"kubernetes.io/projected/f9b2322f-cdfd-43b7-8481-fe9c08706d91-kube-api-access-fdbfn\") pod \"router-default-69965b4854-82hkh\" (UID: \"f9b2322f-cdfd-43b7-8481-fe9c08706d91\") " pod="openshift-ingress/router-default-69965b4854-82hkh" May 26 15:23:14.230846 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.230818 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-mrfrn" May 26 15:23:14.253449 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.253032 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-j9222" May 26 15:23:14.271129 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.271081 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-76dff48d59-qd2tx"] May 26 15:23:14.275590 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:23:14.275558 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd56d3c2_38ff_4bd8_9893_6b643455a976.slice/crio-5fef8abf489db174d1bef8a4755dd0f8affd04df206b158200a752debf486dce WatchSource:0}: Error finding container 5fef8abf489db174d1bef8a4755dd0f8affd04df206b158200a752debf486dce: Status 404 returned error can't find the container with id 5fef8abf489db174d1bef8a4755dd0f8affd04df206b158200a752debf486dce May 26 15:23:14.289410 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.289341 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-klhzf"] May 26 15:23:14.294594 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:23:14.294559 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod463f7c87_7dc8_4b09_a855_6565923f3a8d.slice/crio-13f45e3ae950de27268e6d5d33f454664d764f56931be54e0d8ce884d055da96 WatchSource:0}: Error finding container 13f45e3ae950de27268e6d5d33f454664d764f56931be54e0d8ce884d055da96: Status 404 returned error can't find the container with id 13f45e3ae950de27268e6d5d33f454664d764f56931be54e0d8ce884d055da96 May 26 15:23:14.324784 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.324749 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-klhzf" event={"ID":"463f7c87-7dc8-4b09-a855-6565923f3a8d","Type":"ContainerStarted","Data":"13f45e3ae950de27268e6d5d33f454664d764f56931be54e0d8ce884d055da96"} May 26 15:23:14.325852 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.325827 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-qd2tx" event={"ID":"dd56d3c2-38ff-4bd8-9893-6b643455a976","Type":"ContainerStarted","Data":"5fef8abf489db174d1bef8a4755dd0f8affd04df206b158200a752debf486dce"} May 26 15:23:14.335216 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.335158 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-78bdfb888c-ht62j" May 26 15:23:14.365420 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.365392 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57b5b68b8-mrfrn"] May 26 15:23:14.368233 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:23:14.368198 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc380575f_335e_4023_9b3a_277b8b5fcbbf.slice/crio-db485c7ca46b596140ea1dcaff72fc4a1e31501483f89708977409f999c2fa8f WatchSource:0}: Error finding container db485c7ca46b596140ea1dcaff72fc4a1e31501483f89708977409f999c2fa8f: Status 404 returned error can't find the container with id db485c7ca46b596140ea1dcaff72fc4a1e31501483f89708977409f999c2fa8f May 26 15:23:14.386175 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.386135 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7f44b6dd87-j9222"] May 26 15:23:14.390114 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:23:14.390085 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce0ae8cb_e08f_491d_a674_b60312b575d7.slice/crio-95fe8dde034dd4cd034ada271739af8410c56b7fc704ef1c06fde06d42a0b4f9 WatchSource:0}: Error finding container 95fe8dde034dd4cd034ada271739af8410c56b7fc704ef1c06fde06d42a0b4f9: Status 404 returned error can't find the container with id 95fe8dde034dd4cd034ada271739af8410c56b7fc704ef1c06fde06d42a0b4f9 May 26 15:23:14.456414 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.456375 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-78bdfb888c-ht62j"] May 26 15:23:14.458682 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:23:14.458611 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90be60c1_e9f6_476d_9a27_f050b27cff15.slice/crio-a113bdfa05a707b7ee3bfea83f659bdd1fdc4a705b08e3b30b612f3a6854ac21 WatchSource:0}: Error finding container a113bdfa05a707b7ee3bfea83f659bdd1fdc4a705b08e3b30b612f3a6854ac21: Status 404 returned error can't find the container with id a113bdfa05a707b7ee3bfea83f659bdd1fdc4a705b08e3b30b612f3a6854ac21 May 26 15:23:14.489822 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.489794 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-mkzgw\" (UID: \"a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-mkzgw" May 26 15:23:14.489961 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:23:14.489942 2547 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found May 26 15:23:14.490013 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:23:14.490006 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41-networking-console-plugin-cert podName:a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41 nodeName:}" failed. No retries permitted until 2026-05-26 15:23:15.489992259 +0000 UTC m=+144.216667158 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-mkzgw" (UID: "a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41") : secret "networking-console-plugin-cert" not found May 26 15:23:14.692528 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.692503 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9b2322f-cdfd-43b7-8481-fe9c08706d91-metrics-certs\") pod \"router-default-69965b4854-82hkh\" (UID: \"f9b2322f-cdfd-43b7-8481-fe9c08706d91\") " pod="openshift-ingress/router-default-69965b4854-82hkh" May 26 15:23:14.692684 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:14.692565 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9b2322f-cdfd-43b7-8481-fe9c08706d91-service-ca-bundle\") pod \"router-default-69965b4854-82hkh\" (UID: \"f9b2322f-cdfd-43b7-8481-fe9c08706d91\") " pod="openshift-ingress/router-default-69965b4854-82hkh" May 26 15:23:14.692684 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:23:14.692658 2547 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 26 15:23:14.692784 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:23:14.692667 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f9b2322f-cdfd-43b7-8481-fe9c08706d91-service-ca-bundle podName:f9b2322f-cdfd-43b7-8481-fe9c08706d91 nodeName:}" failed. No retries permitted until 2026-05-26 15:23:15.692654264 +0000 UTC m=+144.419329163 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f9b2322f-cdfd-43b7-8481-fe9c08706d91-service-ca-bundle") pod "router-default-69965b4854-82hkh" (UID: "f9b2322f-cdfd-43b7-8481-fe9c08706d91") : configmap references non-existent config key: service-ca.crt May 26 15:23:14.692784 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:23:14.692728 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9b2322f-cdfd-43b7-8481-fe9c08706d91-metrics-certs podName:f9b2322f-cdfd-43b7-8481-fe9c08706d91 nodeName:}" failed. No retries permitted until 2026-05-26 15:23:15.692710095 +0000 UTC m=+144.419384994 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f9b2322f-cdfd-43b7-8481-fe9c08706d91-metrics-certs") pod "router-default-69965b4854-82hkh" (UID: "f9b2322f-cdfd-43b7-8481-fe9c08706d91") : secret "router-metrics-certs-default" not found May 26 15:23:15.329595 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:15.329530 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-mrfrn" event={"ID":"c380575f-335e-4023-9b3a-277b8b5fcbbf","Type":"ContainerStarted","Data":"db485c7ca46b596140ea1dcaff72fc4a1e31501483f89708977409f999c2fa8f"} May 26 15:23:15.332564 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:15.332533 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-j9222" event={"ID":"ce0ae8cb-e08f-491d-a674-b60312b575d7","Type":"ContainerStarted","Data":"c0077c6a921327876e9cb860d550b9b599a225d43038152586c8ae4d5ab32c1e"} May 26 15:23:15.332687 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:15.332574 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-j9222" event={"ID":"ce0ae8cb-e08f-491d-a674-b60312b575d7","Type":"ContainerStarted","Data":"95fe8dde034dd4cd034ada271739af8410c56b7fc704ef1c06fde06d42a0b4f9"} May 26 15:23:15.335456 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:15.335432 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-78bdfb888c-ht62j" event={"ID":"90be60c1-e9f6-476d-9a27-f050b27cff15","Type":"ContainerStarted","Data":"a113bdfa05a707b7ee3bfea83f659bdd1fdc4a705b08e3b30b612f3a6854ac21"} May 26 15:23:15.350153 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:15.350102 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-j9222" podStartSLOduration=2.35008488 podStartE2EDuration="2.35008488s" podCreationTimestamp="2026-05-26 15:23:13 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 15:23:15.348097552 +0000 UTC m=+144.074772476" watchObservedRunningTime="2026-05-26 15:23:15.35008488 +0000 UTC m=+144.076759810" May 26 15:23:15.499902 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:15.499229 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-mkzgw\" (UID: \"a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-mkzgw" May 26 15:23:15.499902 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:23:15.499444 2547 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found May 26 15:23:15.499902 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:23:15.499512 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41-networking-console-plugin-cert podName:a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41 nodeName:}" failed. No retries permitted until 2026-05-26 15:23:17.499490158 +0000 UTC m=+146.226165076 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-mkzgw" (UID: "a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41") : secret "networking-console-plugin-cert" not found May 26 15:23:15.701104 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:15.700699 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9b2322f-cdfd-43b7-8481-fe9c08706d91-metrics-certs\") pod \"router-default-69965b4854-82hkh\" (UID: \"f9b2322f-cdfd-43b7-8481-fe9c08706d91\") " pod="openshift-ingress/router-default-69965b4854-82hkh" May 26 15:23:15.701104 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:15.700792 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9b2322f-cdfd-43b7-8481-fe9c08706d91-service-ca-bundle\") pod \"router-default-69965b4854-82hkh\" (UID: \"f9b2322f-cdfd-43b7-8481-fe9c08706d91\") " pod="openshift-ingress/router-default-69965b4854-82hkh" May 26 15:23:15.701104 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:23:15.700947 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f9b2322f-cdfd-43b7-8481-fe9c08706d91-service-ca-bundle podName:f9b2322f-cdfd-43b7-8481-fe9c08706d91 nodeName:}" failed. No retries permitted until 2026-05-26 15:23:17.700927267 +0000 UTC m=+146.427602172 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f9b2322f-cdfd-43b7-8481-fe9c08706d91-service-ca-bundle") pod "router-default-69965b4854-82hkh" (UID: "f9b2322f-cdfd-43b7-8481-fe9c08706d91") : configmap references non-existent config key: service-ca.crt May 26 15:23:15.701104 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:23:15.701032 2547 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 26 15:23:15.701104 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:23:15.701085 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9b2322f-cdfd-43b7-8481-fe9c08706d91-metrics-certs podName:f9b2322f-cdfd-43b7-8481-fe9c08706d91 nodeName:}" failed. No retries permitted until 2026-05-26 15:23:17.701071775 +0000 UTC m=+146.427746681 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f9b2322f-cdfd-43b7-8481-fe9c08706d91-metrics-certs") pod "router-default-69965b4854-82hkh" (UID: "f9b2322f-cdfd-43b7-8481-fe9c08706d91") : secret "router-metrics-certs-default" not found May 26 15:23:16.338839 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:16.338749 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-klhzf" event={"ID":"463f7c87-7dc8-4b09-a855-6565923f3a8d","Type":"ContainerStarted","Data":"7f9e62aa045255a804daefdb3fc3a8f05eb2711ef66ea7490c0dcb60b9f22e0a"} May 26 15:23:16.352878 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:16.352833 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-klhzf" podStartSLOduration=1.558712815 podStartE2EDuration="3.352817281s" podCreationTimestamp="2026-05-26 15:23:13 +0000 UTC" firstStartedPulling="2026-05-26 15:23:14.297155078 +0000 UTC m=+143.023829978" lastFinishedPulling="2026-05-26 15:23:16.09125953 +0000 UTC m=+144.817934444" observedRunningTime="2026-05-26 15:23:16.352121017 +0000 UTC m=+145.078795941" watchObservedRunningTime="2026-05-26 15:23:16.352817281 +0000 UTC m=+145.079492204" May 26 15:23:17.516626 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:17.516583 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-mkzgw\" (UID: \"a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-mkzgw" May 26 15:23:17.517073 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:23:17.516751 2547 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found May 26 15:23:17.517073 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:23:17.516831 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41-networking-console-plugin-cert podName:a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41 nodeName:}" failed. No retries permitted until 2026-05-26 15:23:21.516809941 +0000 UTC m=+150.243484865 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-mkzgw" (UID: "a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41") : secret "networking-console-plugin-cert" not found May 26 15:23:17.717618 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:17.717594 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9b2322f-cdfd-43b7-8481-fe9c08706d91-metrics-certs\") pod \"router-default-69965b4854-82hkh\" (UID: \"f9b2322f-cdfd-43b7-8481-fe9c08706d91\") " pod="openshift-ingress/router-default-69965b4854-82hkh" May 26 15:23:17.717706 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:17.717649 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9b2322f-cdfd-43b7-8481-fe9c08706d91-service-ca-bundle\") pod \"router-default-69965b4854-82hkh\" (UID: \"f9b2322f-cdfd-43b7-8481-fe9c08706d91\") " pod="openshift-ingress/router-default-69965b4854-82hkh" May 26 15:23:17.717742 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:23:17.717721 2547 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 26 15:23:17.717775 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:23:17.717760 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f9b2322f-cdfd-43b7-8481-fe9c08706d91-service-ca-bundle podName:f9b2322f-cdfd-43b7-8481-fe9c08706d91 nodeName:}" failed. No retries permitted until 2026-05-26 15:23:21.717747353 +0000 UTC m=+150.444422252 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f9b2322f-cdfd-43b7-8481-fe9c08706d91-service-ca-bundle") pod "router-default-69965b4854-82hkh" (UID: "f9b2322f-cdfd-43b7-8481-fe9c08706d91") : configmap references non-existent config key: service-ca.crt May 26 15:23:17.717775 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:23:17.717774 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9b2322f-cdfd-43b7-8481-fe9c08706d91-metrics-certs podName:f9b2322f-cdfd-43b7-8481-fe9c08706d91 nodeName:}" failed. No retries permitted until 2026-05-26 15:23:21.717767853 +0000 UTC m=+150.444442752 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f9b2322f-cdfd-43b7-8481-fe9c08706d91-metrics-certs") pod "router-default-69965b4854-82hkh" (UID: "f9b2322f-cdfd-43b7-8481-fe9c08706d91") : secret "router-metrics-certs-default" not found May 26 15:23:18.343965 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:18.343927 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-78bdfb888c-ht62j" event={"ID":"90be60c1-e9f6-476d-9a27-f050b27cff15","Type":"ContainerStarted","Data":"9732e6b113a75c3305ea05d3fbac1b4d9cb33e17007e130709e966adf64e0d48"} May 26 15:23:18.345217 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:18.345183 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-mrfrn" event={"ID":"c380575f-335e-4023-9b3a-277b8b5fcbbf","Type":"ContainerStarted","Data":"6cbf6430d0f50255a4ec3e4e1095bf5272a5fd9d5e7ffbc97405514bdd4f6809"} May 26 15:23:18.346717 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:18.346696 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-qd2tx_dd56d3c2-38ff-4bd8-9893-6b643455a976/console-operator/0.log" May 26 15:23:18.346836 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:18.346737 2547 generic.go:358] "Generic (PLEG): container finished" podID="dd56d3c2-38ff-4bd8-9893-6b643455a976" containerID="6b04871c6eca2b5a45c920efe60b23dfc6f65127283c04a26a03f440caa6e6d0" exitCode=255 May 26 15:23:18.346836 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:18.346764 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-qd2tx" event={"ID":"dd56d3c2-38ff-4bd8-9893-6b643455a976","Type":"ContainerDied","Data":"6b04871c6eca2b5a45c920efe60b23dfc6f65127283c04a26a03f440caa6e6d0"} May 26 15:23:18.347022 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:18.347006 2547 scope.go:117] "RemoveContainer" containerID="6b04871c6eca2b5a45c920efe60b23dfc6f65127283c04a26a03f440caa6e6d0" May 26 15:23:18.359035 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:18.358993 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-operator-78bdfb888c-ht62j" podStartSLOduration=1.168860627 podStartE2EDuration="4.358978305s" podCreationTimestamp="2026-05-26 15:23:14 +0000 UTC" firstStartedPulling="2026-05-26 15:23:14.460540932 +0000 UTC m=+143.187215836" lastFinishedPulling="2026-05-26 15:23:17.6506586 +0000 UTC m=+146.377333514" observedRunningTime="2026-05-26 15:23:18.35822807 +0000 UTC m=+147.084902979" watchObservedRunningTime="2026-05-26 15:23:18.358978305 +0000 UTC m=+147.085653225" May 26 15:23:18.388194 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:18.388121 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-mrfrn" podStartSLOduration=2.104434603 podStartE2EDuration="5.388105392s" podCreationTimestamp="2026-05-26 15:23:13 +0000 UTC" firstStartedPulling="2026-05-26 15:23:14.370138992 +0000 UTC m=+143.096813905" lastFinishedPulling="2026-05-26 15:23:17.653809786 +0000 UTC m=+146.380484694" observedRunningTime="2026-05-26 15:23:18.38750968 +0000 UTC m=+147.114184602" watchObservedRunningTime="2026-05-26 15:23:18.388105392 +0000 UTC m=+147.114780316" May 26 15:23:19.350773 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:19.350747 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-qd2tx_dd56d3c2-38ff-4bd8-9893-6b643455a976/console-operator/1.log" May 26 15:23:19.351244 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:19.351130 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-qd2tx_dd56d3c2-38ff-4bd8-9893-6b643455a976/console-operator/0.log" May 26 15:23:19.351244 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:19.351164 2547 generic.go:358] "Generic (PLEG): container finished" podID="dd56d3c2-38ff-4bd8-9893-6b643455a976" containerID="7096dd2ac8ad23027ca790c8e94ab63fe370f92adc921488c0745f283b5bbe48" exitCode=255 May 26 15:23:19.351316 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:19.351252 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-qd2tx" event={"ID":"dd56d3c2-38ff-4bd8-9893-6b643455a976","Type":"ContainerDied","Data":"7096dd2ac8ad23027ca790c8e94ab63fe370f92adc921488c0745f283b5bbe48"} May 26 15:23:19.351316 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:19.351304 2547 scope.go:117] "RemoveContainer" containerID="6b04871c6eca2b5a45c920efe60b23dfc6f65127283c04a26a03f440caa6e6d0" May 26 15:23:19.351470 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:19.351456 2547 scope.go:117] "RemoveContainer" containerID="7096dd2ac8ad23027ca790c8e94ab63fe370f92adc921488c0745f283b5bbe48" May 26 15:23:19.351646 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:23:19.351624 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-76dff48d59-qd2tx_openshift-console-operator(dd56d3c2-38ff-4bd8-9893-6b643455a976)\"" pod="openshift-console-operator/console-operator-76dff48d59-qd2tx" podUID="dd56d3c2-38ff-4bd8-9893-6b643455a976" May 26 15:23:20.355251 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:20.355216 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-qd2tx_dd56d3c2-38ff-4bd8-9893-6b643455a976/console-operator/1.log" May 26 15:23:20.355612 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:20.355536 2547 scope.go:117] "RemoveContainer" containerID="7096dd2ac8ad23027ca790c8e94ab63fe370f92adc921488c0745f283b5bbe48" May 26 15:23:20.355708 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:23:20.355692 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-76dff48d59-qd2tx_openshift-console-operator(dd56d3c2-38ff-4bd8-9893-6b643455a976)\"" pod="openshift-console-operator/console-operator-76dff48d59-qd2tx" podUID="dd56d3c2-38ff-4bd8-9893-6b643455a976" May 26 15:23:21.080101 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:21.080049 2547 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-67d667c7d-6wfrt"] May 26 15:23:21.084332 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:21.084312 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-67d667c7d-6wfrt" May 26 15:23:21.086434 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:21.086414 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" May 26 15:23:21.086532 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:21.086449 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" May 26 15:23:21.086989 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:21.086968 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-gmgtv\"" May 26 15:23:21.087047 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:21.086996 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" May 26 15:23:21.087131 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:21.087004 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" May 26 15:23:21.089459 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:21.089436 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-67d667c7d-6wfrt"] May 26 15:23:21.144593 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:21.144570 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/58a88959-ac35-4c77-bb73-f6e2c0c705e0-signing-cabundle\") pod \"service-ca-67d667c7d-6wfrt\" (UID: \"58a88959-ac35-4c77-bb73-f6e2c0c705e0\") " pod="openshift-service-ca/service-ca-67d667c7d-6wfrt" May 26 15:23:21.144717 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:21.144703 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klj6g\" (UniqueName: \"kubernetes.io/projected/58a88959-ac35-4c77-bb73-f6e2c0c705e0-kube-api-access-klj6g\") pod \"service-ca-67d667c7d-6wfrt\" (UID: \"58a88959-ac35-4c77-bb73-f6e2c0c705e0\") " pod="openshift-service-ca/service-ca-67d667c7d-6wfrt" May 26 15:23:21.144769 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:21.144729 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/58a88959-ac35-4c77-bb73-f6e2c0c705e0-signing-key\") pod \"service-ca-67d667c7d-6wfrt\" (UID: \"58a88959-ac35-4c77-bb73-f6e2c0c705e0\") " pod="openshift-service-ca/service-ca-67d667c7d-6wfrt" May 26 15:23:21.245580 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:21.245554 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-klj6g\" (UniqueName: \"kubernetes.io/projected/58a88959-ac35-4c77-bb73-f6e2c0c705e0-kube-api-access-klj6g\") pod \"service-ca-67d667c7d-6wfrt\" (UID: \"58a88959-ac35-4c77-bb73-f6e2c0c705e0\") " pod="openshift-service-ca/service-ca-67d667c7d-6wfrt" May 26 15:23:21.245675 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:21.245583 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/58a88959-ac35-4c77-bb73-f6e2c0c705e0-signing-key\") pod \"service-ca-67d667c7d-6wfrt\" (UID: \"58a88959-ac35-4c77-bb73-f6e2c0c705e0\") " pod="openshift-service-ca/service-ca-67d667c7d-6wfrt" May 26 15:23:21.245675 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:21.245611 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/58a88959-ac35-4c77-bb73-f6e2c0c705e0-signing-cabundle\") pod \"service-ca-67d667c7d-6wfrt\" (UID: \"58a88959-ac35-4c77-bb73-f6e2c0c705e0\") " pod="openshift-service-ca/service-ca-67d667c7d-6wfrt" May 26 15:23:21.246171 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:21.246154 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/58a88959-ac35-4c77-bb73-f6e2c0c705e0-signing-cabundle\") pod \"service-ca-67d667c7d-6wfrt\" (UID: \"58a88959-ac35-4c77-bb73-f6e2c0c705e0\") " pod="openshift-service-ca/service-ca-67d667c7d-6wfrt" May 26 15:23:21.247781 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:21.247758 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/58a88959-ac35-4c77-bb73-f6e2c0c705e0-signing-key\") pod \"service-ca-67d667c7d-6wfrt\" (UID: \"58a88959-ac35-4c77-bb73-f6e2c0c705e0\") " pod="openshift-service-ca/service-ca-67d667c7d-6wfrt" May 26 15:23:21.253371 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:21.253347 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-klj6g\" (UniqueName: \"kubernetes.io/projected/58a88959-ac35-4c77-bb73-f6e2c0c705e0-kube-api-access-klj6g\") pod \"service-ca-67d667c7d-6wfrt\" (UID: \"58a88959-ac35-4c77-bb73-f6e2c0c705e0\") " pod="openshift-service-ca/service-ca-67d667c7d-6wfrt" May 26 15:23:21.393689 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:21.393628 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-67d667c7d-6wfrt" May 26 15:23:21.506777 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:21.506747 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-67d667c7d-6wfrt"] May 26 15:23:21.509880 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:23:21.509848 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58a88959_ac35_4c77_bb73_f6e2c0c705e0.slice/crio-c1173aa42a7a4268fa9044a8cb94c0a135f3a874e6900f48a985e9bfcc907c8b WatchSource:0}: Error finding container c1173aa42a7a4268fa9044a8cb94c0a135f3a874e6900f48a985e9bfcc907c8b: Status 404 returned error can't find the container with id c1173aa42a7a4268fa9044a8cb94c0a135f3a874e6900f48a985e9bfcc907c8b May 26 15:23:21.548236 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:21.548210 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-mkzgw\" (UID: \"a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-mkzgw" May 26 15:23:21.548367 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:23:21.548353 2547 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found May 26 15:23:21.548423 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:23:21.548413 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41-networking-console-plugin-cert podName:a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41 nodeName:}" failed. No retries permitted until 2026-05-26 15:23:29.548398119 +0000 UTC m=+158.275073018 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-mkzgw" (UID: "a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41") : secret "networking-console-plugin-cert" not found May 26 15:23:21.710935 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:21.710910 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ddnhg_a1723cfd-d190-4d26-9964-3c378ba0cd3b/dns-node-resolver/0.log" May 26 15:23:21.750343 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:21.750317 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9b2322f-cdfd-43b7-8481-fe9c08706d91-service-ca-bundle\") pod \"router-default-69965b4854-82hkh\" (UID: \"f9b2322f-cdfd-43b7-8481-fe9c08706d91\") " pod="openshift-ingress/router-default-69965b4854-82hkh" May 26 15:23:21.750470 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:21.750367 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9b2322f-cdfd-43b7-8481-fe9c08706d91-metrics-certs\") pod \"router-default-69965b4854-82hkh\" (UID: \"f9b2322f-cdfd-43b7-8481-fe9c08706d91\") " pod="openshift-ingress/router-default-69965b4854-82hkh" May 26 15:23:21.750470 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:23:21.750462 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f9b2322f-cdfd-43b7-8481-fe9c08706d91-service-ca-bundle podName:f9b2322f-cdfd-43b7-8481-fe9c08706d91 nodeName:}" failed. No retries permitted until 2026-05-26 15:23:29.750447277 +0000 UTC m=+158.477122175 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f9b2322f-cdfd-43b7-8481-fe9c08706d91-service-ca-bundle") pod "router-default-69965b4854-82hkh" (UID: "f9b2322f-cdfd-43b7-8481-fe9c08706d91") : configmap references non-existent config key: service-ca.crt May 26 15:23:21.750565 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:23:21.750491 2547 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 26 15:23:21.750565 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:23:21.750535 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9b2322f-cdfd-43b7-8481-fe9c08706d91-metrics-certs podName:f9b2322f-cdfd-43b7-8481-fe9c08706d91 nodeName:}" failed. No retries permitted until 2026-05-26 15:23:29.750520784 +0000 UTC m=+158.477195683 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f9b2322f-cdfd-43b7-8481-fe9c08706d91-metrics-certs") pod "router-default-69965b4854-82hkh" (UID: "f9b2322f-cdfd-43b7-8481-fe9c08706d91") : secret "router-metrics-certs-default" not found May 26 15:23:22.361812 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:22.361769 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-67d667c7d-6wfrt" event={"ID":"58a88959-ac35-4c77-bb73-f6e2c0c705e0","Type":"ContainerStarted","Data":"6a850f549ae7f769b57c4041c87845a94e2b1721b65b691f52e198cb76e46ca2"} May 26 15:23:22.361812 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:22.361811 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-67d667c7d-6wfrt" event={"ID":"58a88959-ac35-4c77-bb73-f6e2c0c705e0","Type":"ContainerStarted","Data":"c1173aa42a7a4268fa9044a8cb94c0a135f3a874e6900f48a985e9bfcc907c8b"} May 26 15:23:22.378004 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:22.377964 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca/service-ca-67d667c7d-6wfrt" podStartSLOduration=1.3779520490000001 podStartE2EDuration="1.377952049s" podCreationTimestamp="2026-05-26 15:23:21 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 15:23:22.377579405 +0000 UTC m=+151.104254328" watchObservedRunningTime="2026-05-26 15:23:22.377952049 +0000 UTC m=+151.104626969" May 26 15:23:22.712019 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:22.711995 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2r56q_4f6155d9-d73c-483b-ade7-8eecd0e99883/node-ca/0.log" May 26 15:23:24.145514 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:24.145480 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-76dff48d59-qd2tx" May 26 15:23:24.145514 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:24.145512 2547 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-76dff48d59-qd2tx" May 26 15:23:24.145932 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:24.145860 2547 scope.go:117] "RemoveContainer" containerID="7096dd2ac8ad23027ca790c8e94ab63fe370f92adc921488c0745f283b5bbe48" May 26 15:23:24.146098 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:23:24.146075 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-76dff48d59-qd2tx_openshift-console-operator(dd56d3c2-38ff-4bd8-9893-6b643455a976)\"" pod="openshift-console-operator/console-operator-76dff48d59-qd2tx" podUID="dd56d3c2-38ff-4bd8-9893-6b643455a976" May 26 15:23:28.202531 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:23:28.202478 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-dzpbp" podUID="91f902eb-040a-46bf-a271-e75d7e98479f" May 26 15:23:28.214913 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:23:28.214880 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-8sxg9" podUID="7b1e268b-9140-4f73-8abf-c1c30fc706bc" May 26 15:23:28.376631 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:28.376594 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dzpbp" May 26 15:23:29.613070 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:29.613021 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-mkzgw\" (UID: \"a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-mkzgw" May 26 15:23:29.613427 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:23:29.613169 2547 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found May 26 15:23:29.613427 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:23:29.613240 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41-networking-console-plugin-cert podName:a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41 nodeName:}" failed. No retries permitted until 2026-05-26 15:23:45.613224484 +0000 UTC m=+174.339899384 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-mkzgw" (UID: "a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41") : secret "networking-console-plugin-cert" not found May 26 15:23:29.815349 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:29.815308 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9b2322f-cdfd-43b7-8481-fe9c08706d91-service-ca-bundle\") pod \"router-default-69965b4854-82hkh\" (UID: \"f9b2322f-cdfd-43b7-8481-fe9c08706d91\") " pod="openshift-ingress/router-default-69965b4854-82hkh" May 26 15:23:29.815534 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:29.815392 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9b2322f-cdfd-43b7-8481-fe9c08706d91-metrics-certs\") pod \"router-default-69965b4854-82hkh\" (UID: \"f9b2322f-cdfd-43b7-8481-fe9c08706d91\") " pod="openshift-ingress/router-default-69965b4854-82hkh" May 26 15:23:29.815906 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:29.815872 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9b2322f-cdfd-43b7-8481-fe9c08706d91-service-ca-bundle\") pod \"router-default-69965b4854-82hkh\" (UID: \"f9b2322f-cdfd-43b7-8481-fe9c08706d91\") " pod="openshift-ingress/router-default-69965b4854-82hkh" May 26 15:23:29.817762 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:29.817742 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9b2322f-cdfd-43b7-8481-fe9c08706d91-metrics-certs\") pod \"router-default-69965b4854-82hkh\" (UID: \"f9b2322f-cdfd-43b7-8481-fe9c08706d91\") " pod="openshift-ingress/router-default-69965b4854-82hkh" May 26 15:23:29.921019 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:23:29.920946 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-5xh82" podUID="fac6c416-4fef-4b1b-846f-ea63a14c9efe" May 26 15:23:29.941392 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:29.941365 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-69965b4854-82hkh" May 26 15:23:30.064766 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:30.064732 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-69965b4854-82hkh"] May 26 15:23:30.067789 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:23:30.067759 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9b2322f_cdfd_43b7_8481_fe9c08706d91.slice/crio-83bd4375fc1ae7ec8c57b0dc54d116a7371f85b7cfed64f3de5c6b8e93a8d86a WatchSource:0}: Error finding container 83bd4375fc1ae7ec8c57b0dc54d116a7371f85b7cfed64f3de5c6b8e93a8d86a: Status 404 returned error can't find the container with id 83bd4375fc1ae7ec8c57b0dc54d116a7371f85b7cfed64f3de5c6b8e93a8d86a May 26 15:23:30.383261 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:30.383224 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-69965b4854-82hkh" event={"ID":"f9b2322f-cdfd-43b7-8481-fe9c08706d91","Type":"ContainerStarted","Data":"42fd0bdffbfc6b7719133672abeeb49aa44333dda3d55b7941bd3d4035edead5"} May 26 15:23:30.383261 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:30.383262 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-69965b4854-82hkh" event={"ID":"f9b2322f-cdfd-43b7-8481-fe9c08706d91","Type":"ContainerStarted","Data":"83bd4375fc1ae7ec8c57b0dc54d116a7371f85b7cfed64f3de5c6b8e93a8d86a"} May 26 15:23:30.400994 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:30.400948 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/router-default-69965b4854-82hkh" podStartSLOduration=16.400933511 podStartE2EDuration="16.400933511s" podCreationTimestamp="2026-05-26 15:23:14 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 15:23:30.400029909 +0000 UTC m=+159.126704831" watchObservedRunningTime="2026-05-26 15:23:30.400933511 +0000 UTC m=+159.127608483" May 26 15:23:30.942488 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:30.942453 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-69965b4854-82hkh" May 26 15:23:30.944717 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:30.944693 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-69965b4854-82hkh" May 26 15:23:31.386645 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:31.386568 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-69965b4854-82hkh" May 26 15:23:31.387815 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:31.387793 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-69965b4854-82hkh" May 26 15:23:33.141321 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:33.141276 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/91f902eb-040a-46bf-a271-e75d7e98479f-metrics-tls\") pod \"dns-default-dzpbp\" (UID: \"91f902eb-040a-46bf-a271-e75d7e98479f\") " pod="openshift-dns/dns-default-dzpbp" May 26 15:23:33.141709 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:33.141365 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b1e268b-9140-4f73-8abf-c1c30fc706bc-cert\") pod \"ingress-canary-8sxg9\" (UID: \"7b1e268b-9140-4f73-8abf-c1c30fc706bc\") " pod="openshift-ingress-canary/ingress-canary-8sxg9" May 26 15:23:33.143614 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:33.143584 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/91f902eb-040a-46bf-a271-e75d7e98479f-metrics-tls\") pod \"dns-default-dzpbp\" (UID: \"91f902eb-040a-46bf-a271-e75d7e98479f\") " pod="openshift-dns/dns-default-dzpbp" May 26 15:23:33.143721 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:33.143645 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b1e268b-9140-4f73-8abf-c1c30fc706bc-cert\") pod \"ingress-canary-8sxg9\" (UID: \"7b1e268b-9140-4f73-8abf-c1c30fc706bc\") " pod="openshift-ingress-canary/ingress-canary-8sxg9" May 26 15:23:33.179335 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:33.179307 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-smt4m\"" May 26 15:23:33.187729 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:33.187705 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dzpbp" May 26 15:23:33.305795 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:33.305763 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dzpbp"] May 26 15:23:33.308684 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:23:33.308652 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91f902eb_040a_46bf_a271_e75d7e98479f.slice/crio-c5debd92d51303a0fba59a855a655d510de20c96f88be72508554874344f9b1d WatchSource:0}: Error finding container c5debd92d51303a0fba59a855a655d510de20c96f88be72508554874344f9b1d: Status 404 returned error can't find the container with id c5debd92d51303a0fba59a855a655d510de20c96f88be72508554874344f9b1d May 26 15:23:33.392020 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:33.391951 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dzpbp" event={"ID":"91f902eb-040a-46bf-a271-e75d7e98479f","Type":"ContainerStarted","Data":"c5debd92d51303a0fba59a855a655d510de20c96f88be72508554874344f9b1d"} May 26 15:23:35.398846 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:35.398809 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dzpbp" event={"ID":"91f902eb-040a-46bf-a271-e75d7e98479f","Type":"ContainerStarted","Data":"0d18ab2b6613d1a81d78b86ed4dbbc637deec2ff02be40704066c77cf951099d"} May 26 15:23:35.398846 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:35.398844 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dzpbp" event={"ID":"91f902eb-040a-46bf-a271-e75d7e98479f","Type":"ContainerStarted","Data":"501ba1e30d35afda796aa314b986cb3713a1dbf90bb2ce420241840b91424800"} May 26 15:23:35.399259 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:35.398938 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-dzpbp" May 26 15:23:35.414635 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:35.414591 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-dzpbp" podStartSLOduration=129.211017547 podStartE2EDuration="2m10.414580819s" podCreationTimestamp="2026-05-26 15:21:25 +0000 UTC" firstStartedPulling="2026-05-26 15:23:33.310474521 +0000 UTC m=+162.037149422" lastFinishedPulling="2026-05-26 15:23:34.51403778 +0000 UTC m=+163.240712694" observedRunningTime="2026-05-26 15:23:35.413982435 +0000 UTC m=+164.140657358" watchObservedRunningTime="2026-05-26 15:23:35.414580819 +0000 UTC m=+164.141255752" May 26 15:23:38.910089 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:38.909997 2547 scope.go:117] "RemoveContainer" containerID="7096dd2ac8ad23027ca790c8e94ab63fe370f92adc921488c0745f283b5bbe48" May 26 15:23:39.410141 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:39.410110 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-qd2tx_dd56d3c2-38ff-4bd8-9893-6b643455a976/console-operator/2.log" May 26 15:23:39.410450 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:39.410435 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-qd2tx_dd56d3c2-38ff-4bd8-9893-6b643455a976/console-operator/1.log" May 26 15:23:39.410497 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:39.410466 2547 generic.go:358] "Generic (PLEG): container finished" podID="dd56d3c2-38ff-4bd8-9893-6b643455a976" containerID="ba1f1c3faf37af59cd908b38d94e6701bd2e4738cecead8b89a1317caed08299" exitCode=255 May 26 15:23:39.410542 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:39.410505 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-qd2tx" event={"ID":"dd56d3c2-38ff-4bd8-9893-6b643455a976","Type":"ContainerDied","Data":"ba1f1c3faf37af59cd908b38d94e6701bd2e4738cecead8b89a1317caed08299"} May 26 15:23:39.410542 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:39.410534 2547 scope.go:117] "RemoveContainer" containerID="7096dd2ac8ad23027ca790c8e94ab63fe370f92adc921488c0745f283b5bbe48" May 26 15:23:39.410831 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:39.410813 2547 scope.go:117] "RemoveContainer" containerID="ba1f1c3faf37af59cd908b38d94e6701bd2e4738cecead8b89a1317caed08299" May 26 15:23:39.411036 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:23:39.411018 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-76dff48d59-qd2tx_openshift-console-operator(dd56d3c2-38ff-4bd8-9893-6b643455a976)\"" pod="openshift-console-operator/console-operator-76dff48d59-qd2tx" podUID="dd56d3c2-38ff-4bd8-9893-6b643455a976" May 26 15:23:40.413968 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:40.413940 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-qd2tx_dd56d3c2-38ff-4bd8-9893-6b643455a976/console-operator/2.log" May 26 15:23:41.444466 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.444433 2547 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6b645d86d4-g9z92"] May 26 15:23:41.448745 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.448724 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b645d86d4-g9z92" May 26 15:23:41.451138 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.451119 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" May 26 15:23:41.451251 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.451179 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gj9mv\"" May 26 15:23:41.451312 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.451277 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" May 26 15:23:41.451679 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.451655 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" May 26 15:23:41.457663 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.457645 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" May 26 15:23:41.462622 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.462603 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b645d86d4-g9z92"] May 26 15:23:41.472959 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.472939 2547 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mkhbh"] May 26 15:23:41.475941 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.475923 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mkhbh" May 26 15:23:41.478023 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.478002 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" May 26 15:23:41.478137 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.478009 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" May 26 15:23:41.478309 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.478294 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nz49q\"" May 26 15:23:41.486527 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.486506 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mkhbh"] May 26 15:23:41.608404 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.608375 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/125e1723-0c7c-4afd-b344-a05746f62eb7-installation-pull-secrets\") pod \"image-registry-6b645d86d4-g9z92\" (UID: \"125e1723-0c7c-4afd-b344-a05746f62eb7\") " pod="openshift-image-registry/image-registry-6b645d86d4-g9z92" May 26 15:23:41.608534 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.608411 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/12d2a1e5-82ec-4b4d-9eb1-fac4406e2005-crio-socket\") pod \"insights-runtime-extractor-mkhbh\" (UID: \"12d2a1e5-82ec-4b4d-9eb1-fac4406e2005\") " pod="openshift-insights/insights-runtime-extractor-mkhbh" May 26 15:23:41.608534 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.608482 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/125e1723-0c7c-4afd-b344-a05746f62eb7-ca-trust-extracted\") pod \"image-registry-6b645d86d4-g9z92\" (UID: \"125e1723-0c7c-4afd-b344-a05746f62eb7\") " pod="openshift-image-registry/image-registry-6b645d86d4-g9z92" May 26 15:23:41.608534 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.608515 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/125e1723-0c7c-4afd-b344-a05746f62eb7-registry-certificates\") pod \"image-registry-6b645d86d4-g9z92\" (UID: \"125e1723-0c7c-4afd-b344-a05746f62eb7\") " pod="openshift-image-registry/image-registry-6b645d86d4-g9z92" May 26 15:23:41.608649 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.608550 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/12d2a1e5-82ec-4b4d-9eb1-fac4406e2005-data-volume\") pod \"insights-runtime-extractor-mkhbh\" (UID: \"12d2a1e5-82ec-4b4d-9eb1-fac4406e2005\") " pod="openshift-insights/insights-runtime-extractor-mkhbh" May 26 15:23:41.608649 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.608579 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/12d2a1e5-82ec-4b4d-9eb1-fac4406e2005-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mkhbh\" (UID: \"12d2a1e5-82ec-4b4d-9eb1-fac4406e2005\") " pod="openshift-insights/insights-runtime-extractor-mkhbh" May 26 15:23:41.608649 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.608609 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/12d2a1e5-82ec-4b4d-9eb1-fac4406e2005-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mkhbh\" (UID: \"12d2a1e5-82ec-4b4d-9eb1-fac4406e2005\") " pod="openshift-insights/insights-runtime-extractor-mkhbh" May 26 15:23:41.608649 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.608637 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/125e1723-0c7c-4afd-b344-a05746f62eb7-bound-sa-token\") pod \"image-registry-6b645d86d4-g9z92\" (UID: \"125e1723-0c7c-4afd-b344-a05746f62eb7\") " pod="openshift-image-registry/image-registry-6b645d86d4-g9z92" May 26 15:23:41.608779 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.608659 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/125e1723-0c7c-4afd-b344-a05746f62eb7-image-registry-private-configuration\") pod \"image-registry-6b645d86d4-g9z92\" (UID: \"125e1723-0c7c-4afd-b344-a05746f62eb7\") " pod="openshift-image-registry/image-registry-6b645d86d4-g9z92" May 26 15:23:41.608779 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.608680 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ch98z\" (UniqueName: \"kubernetes.io/projected/12d2a1e5-82ec-4b4d-9eb1-fac4406e2005-kube-api-access-ch98z\") pod \"insights-runtime-extractor-mkhbh\" (UID: \"12d2a1e5-82ec-4b4d-9eb1-fac4406e2005\") " pod="openshift-insights/insights-runtime-extractor-mkhbh" May 26 15:23:41.608779 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.608719 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/125e1723-0c7c-4afd-b344-a05746f62eb7-trusted-ca\") pod \"image-registry-6b645d86d4-g9z92\" (UID: \"125e1723-0c7c-4afd-b344-a05746f62eb7\") " pod="openshift-image-registry/image-registry-6b645d86d4-g9z92" May 26 15:23:41.608779 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.608739 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w527r\" (UniqueName: \"kubernetes.io/projected/125e1723-0c7c-4afd-b344-a05746f62eb7-kube-api-access-w527r\") pod \"image-registry-6b645d86d4-g9z92\" (UID: \"125e1723-0c7c-4afd-b344-a05746f62eb7\") " pod="openshift-image-registry/image-registry-6b645d86d4-g9z92" May 26 15:23:41.608779 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.608768 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/125e1723-0c7c-4afd-b344-a05746f62eb7-registry-tls\") pod \"image-registry-6b645d86d4-g9z92\" (UID: \"125e1723-0c7c-4afd-b344-a05746f62eb7\") " pod="openshift-image-registry/image-registry-6b645d86d4-g9z92" May 26 15:23:41.709635 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.709605 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/125e1723-0c7c-4afd-b344-a05746f62eb7-registry-tls\") pod \"image-registry-6b645d86d4-g9z92\" (UID: \"125e1723-0c7c-4afd-b344-a05746f62eb7\") " pod="openshift-image-registry/image-registry-6b645d86d4-g9z92" May 26 15:23:41.709755 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.709647 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/125e1723-0c7c-4afd-b344-a05746f62eb7-installation-pull-secrets\") pod \"image-registry-6b645d86d4-g9z92\" (UID: \"125e1723-0c7c-4afd-b344-a05746f62eb7\") " pod="openshift-image-registry/image-registry-6b645d86d4-g9z92" May 26 15:23:41.709755 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.709704 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/12d2a1e5-82ec-4b4d-9eb1-fac4406e2005-crio-socket\") pod \"insights-runtime-extractor-mkhbh\" (UID: \"12d2a1e5-82ec-4b4d-9eb1-fac4406e2005\") " pod="openshift-insights/insights-runtime-extractor-mkhbh" May 26 15:23:41.709866 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.709756 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/125e1723-0c7c-4afd-b344-a05746f62eb7-ca-trust-extracted\") pod \"image-registry-6b645d86d4-g9z92\" (UID: \"125e1723-0c7c-4afd-b344-a05746f62eb7\") " pod="openshift-image-registry/image-registry-6b645d86d4-g9z92" May 26 15:23:41.709866 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.709786 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/125e1723-0c7c-4afd-b344-a05746f62eb7-registry-certificates\") pod \"image-registry-6b645d86d4-g9z92\" (UID: \"125e1723-0c7c-4afd-b344-a05746f62eb7\") " pod="openshift-image-registry/image-registry-6b645d86d4-g9z92" May 26 15:23:41.709973 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.709913 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/12d2a1e5-82ec-4b4d-9eb1-fac4406e2005-data-volume\") pod \"insights-runtime-extractor-mkhbh\" (UID: \"12d2a1e5-82ec-4b4d-9eb1-fac4406e2005\") " pod="openshift-insights/insights-runtime-extractor-mkhbh" May 26 15:23:41.710023 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.709829 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/12d2a1e5-82ec-4b4d-9eb1-fac4406e2005-crio-socket\") pod \"insights-runtime-extractor-mkhbh\" (UID: \"12d2a1e5-82ec-4b4d-9eb1-fac4406e2005\") " pod="openshift-insights/insights-runtime-extractor-mkhbh" May 26 15:23:41.710110 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.709959 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/12d2a1e5-82ec-4b4d-9eb1-fac4406e2005-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mkhbh\" (UID: \"12d2a1e5-82ec-4b4d-9eb1-fac4406e2005\") " pod="openshift-insights/insights-runtime-extractor-mkhbh" May 26 15:23:41.710110 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.710086 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/12d2a1e5-82ec-4b4d-9eb1-fac4406e2005-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mkhbh\" (UID: \"12d2a1e5-82ec-4b4d-9eb1-fac4406e2005\") " pod="openshift-insights/insights-runtime-extractor-mkhbh" May 26 15:23:41.710203 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.710121 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/125e1723-0c7c-4afd-b344-a05746f62eb7-bound-sa-token\") pod \"image-registry-6b645d86d4-g9z92\" (UID: \"125e1723-0c7c-4afd-b344-a05746f62eb7\") " pod="openshift-image-registry/image-registry-6b645d86d4-g9z92" May 26 15:23:41.710203 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.710154 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/125e1723-0c7c-4afd-b344-a05746f62eb7-image-registry-private-configuration\") pod \"image-registry-6b645d86d4-g9z92\" (UID: \"125e1723-0c7c-4afd-b344-a05746f62eb7\") " pod="openshift-image-registry/image-registry-6b645d86d4-g9z92" May 26 15:23:41.710294 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.710186 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ch98z\" (UniqueName: \"kubernetes.io/projected/12d2a1e5-82ec-4b4d-9eb1-fac4406e2005-kube-api-access-ch98z\") pod \"insights-runtime-extractor-mkhbh\" (UID: \"12d2a1e5-82ec-4b4d-9eb1-fac4406e2005\") " pod="openshift-insights/insights-runtime-extractor-mkhbh" May 26 15:23:41.710350 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.710294 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/125e1723-0c7c-4afd-b344-a05746f62eb7-trusted-ca\") pod \"image-registry-6b645d86d4-g9z92\" (UID: \"125e1723-0c7c-4afd-b344-a05746f62eb7\") " pod="openshift-image-registry/image-registry-6b645d86d4-g9z92" May 26 15:23:41.710350 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.710326 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w527r\" (UniqueName: \"kubernetes.io/projected/125e1723-0c7c-4afd-b344-a05746f62eb7-kube-api-access-w527r\") pod \"image-registry-6b645d86d4-g9z92\" (UID: \"125e1723-0c7c-4afd-b344-a05746f62eb7\") " pod="openshift-image-registry/image-registry-6b645d86d4-g9z92" May 26 15:23:41.710350 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.710337 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/12d2a1e5-82ec-4b4d-9eb1-fac4406e2005-data-volume\") pod \"insights-runtime-extractor-mkhbh\" (UID: \"12d2a1e5-82ec-4b4d-9eb1-fac4406e2005\") " pod="openshift-insights/insights-runtime-extractor-mkhbh" May 26 15:23:41.710844 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.710822 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/125e1723-0c7c-4afd-b344-a05746f62eb7-ca-trust-extracted\") pod \"image-registry-6b645d86d4-g9z92\" (UID: \"125e1723-0c7c-4afd-b344-a05746f62eb7\") " pod="openshift-image-registry/image-registry-6b645d86d4-g9z92" May 26 15:23:41.710970 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.710837 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/125e1723-0c7c-4afd-b344-a05746f62eb7-registry-certificates\") pod \"image-registry-6b645d86d4-g9z92\" (UID: \"125e1723-0c7c-4afd-b344-a05746f62eb7\") " pod="openshift-image-registry/image-registry-6b645d86d4-g9z92" May 26 15:23:41.711161 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.711137 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/12d2a1e5-82ec-4b4d-9eb1-fac4406e2005-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mkhbh\" (UID: \"12d2a1e5-82ec-4b4d-9eb1-fac4406e2005\") " pod="openshift-insights/insights-runtime-extractor-mkhbh" May 26 15:23:41.711347 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.711331 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/125e1723-0c7c-4afd-b344-a05746f62eb7-trusted-ca\") pod \"image-registry-6b645d86d4-g9z92\" (UID: \"125e1723-0c7c-4afd-b344-a05746f62eb7\") " pod="openshift-image-registry/image-registry-6b645d86d4-g9z92" May 26 15:23:41.712720 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.712699 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/12d2a1e5-82ec-4b4d-9eb1-fac4406e2005-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mkhbh\" (UID: \"12d2a1e5-82ec-4b4d-9eb1-fac4406e2005\") " pod="openshift-insights/insights-runtime-extractor-mkhbh" May 26 15:23:41.712925 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.712901 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/125e1723-0c7c-4afd-b344-a05746f62eb7-installation-pull-secrets\") pod \"image-registry-6b645d86d4-g9z92\" (UID: \"125e1723-0c7c-4afd-b344-a05746f62eb7\") " pod="openshift-image-registry/image-registry-6b645d86d4-g9z92" May 26 15:23:41.713011 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.712941 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/125e1723-0c7c-4afd-b344-a05746f62eb7-registry-tls\") pod \"image-registry-6b645d86d4-g9z92\" (UID: \"125e1723-0c7c-4afd-b344-a05746f62eb7\") " pod="openshift-image-registry/image-registry-6b645d86d4-g9z92" May 26 15:23:41.713118 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.713099 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/125e1723-0c7c-4afd-b344-a05746f62eb7-image-registry-private-configuration\") pod \"image-registry-6b645d86d4-g9z92\" (UID: \"125e1723-0c7c-4afd-b344-a05746f62eb7\") " pod="openshift-image-registry/image-registry-6b645d86d4-g9z92" May 26 15:23:41.720466 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.720447 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ch98z\" (UniqueName: \"kubernetes.io/projected/12d2a1e5-82ec-4b4d-9eb1-fac4406e2005-kube-api-access-ch98z\") pod \"insights-runtime-extractor-mkhbh\" (UID: \"12d2a1e5-82ec-4b4d-9eb1-fac4406e2005\") " pod="openshift-insights/insights-runtime-extractor-mkhbh" May 26 15:23:41.720549 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.720484 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/125e1723-0c7c-4afd-b344-a05746f62eb7-bound-sa-token\") pod \"image-registry-6b645d86d4-g9z92\" (UID: \"125e1723-0c7c-4afd-b344-a05746f62eb7\") " pod="openshift-image-registry/image-registry-6b645d86d4-g9z92" May 26 15:23:41.720663 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.720647 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w527r\" (UniqueName: \"kubernetes.io/projected/125e1723-0c7c-4afd-b344-a05746f62eb7-kube-api-access-w527r\") pod \"image-registry-6b645d86d4-g9z92\" (UID: \"125e1723-0c7c-4afd-b344-a05746f62eb7\") " pod="openshift-image-registry/image-registry-6b645d86d4-g9z92" May 26 15:23:41.758439 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.758420 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b645d86d4-g9z92" May 26 15:23:41.784358 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.784334 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mkhbh" May 26 15:23:41.916403 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.916385 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b645d86d4-g9z92"] May 26 15:23:41.919387 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:23:41.919359 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod125e1723_0c7c_4afd_b344_a05746f62eb7.slice/crio-3e96656f98914e61b4f345b165e93c8944ca8fb244a977cd7d5394217f913f55 WatchSource:0}: Error finding container 3e96656f98914e61b4f345b165e93c8944ca8fb244a977cd7d5394217f913f55: Status 404 returned error can't find the container with id 3e96656f98914e61b4f345b165e93c8944ca8fb244a977cd7d5394217f913f55 May 26 15:23:41.933460 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:41.932317 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mkhbh"] May 26 15:23:42.421401 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:42.421306 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mkhbh" event={"ID":"12d2a1e5-82ec-4b4d-9eb1-fac4406e2005","Type":"ContainerStarted","Data":"2e5b75fa86c3c5986ee9d2085c513cc71816a4f400344892455111ceae4c0e4b"} May 26 15:23:42.421401 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:42.421350 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mkhbh" event={"ID":"12d2a1e5-82ec-4b4d-9eb1-fac4406e2005","Type":"ContainerStarted","Data":"a2e183a53c2584f90082e6d3e2505a05da7ad8588c366dc0d91a34303ce39d02"} May 26 15:23:42.422565 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:42.422537 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b645d86d4-g9z92" event={"ID":"125e1723-0c7c-4afd-b344-a05746f62eb7","Type":"ContainerStarted","Data":"bb16287f30e0bc5d60705932554a057d27461e2a7883c8ab998d25c0da8652fc"} May 26 15:23:42.422565 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:42.422569 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b645d86d4-g9z92" event={"ID":"125e1723-0c7c-4afd-b344-a05746f62eb7","Type":"ContainerStarted","Data":"3e96656f98914e61b4f345b165e93c8944ca8fb244a977cd7d5394217f913f55"} May 26 15:23:42.422738 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:42.422693 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6b645d86d4-g9z92" May 26 15:23:42.440703 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:42.440656 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6b645d86d4-g9z92" podStartSLOduration=1.44064279 podStartE2EDuration="1.44064279s" podCreationTimestamp="2026-05-26 15:23:41 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 15:23:42.440218509 +0000 UTC m=+171.166893430" watchObservedRunningTime="2026-05-26 15:23:42.44064279 +0000 UTC m=+171.167317755" May 26 15:23:42.909787 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:42.909757 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8sxg9" May 26 15:23:42.911989 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:42.911972 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-pt65f\"" May 26 15:23:42.920451 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:42.920435 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8sxg9" May 26 15:23:43.037970 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:43.037943 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8sxg9"] May 26 15:23:43.041825 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:23:43.041794 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b1e268b_9140_4f73_8abf_c1c30fc706bc.slice/crio-1692c79b3b5029464adb38ea8859fa57c7c55358414e0576d9a8757906c99411 WatchSource:0}: Error finding container 1692c79b3b5029464adb38ea8859fa57c7c55358414e0576d9a8757906c99411: Status 404 returned error can't find the container with id 1692c79b3b5029464adb38ea8859fa57c7c55358414e0576d9a8757906c99411 May 26 15:23:43.426397 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:43.426364 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mkhbh" event={"ID":"12d2a1e5-82ec-4b4d-9eb1-fac4406e2005","Type":"ContainerStarted","Data":"690d4c896bc48af224213f51db655beca137602b106b2ddd5c5d0d30eaea833e"} May 26 15:23:43.427344 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:43.427309 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8sxg9" event={"ID":"7b1e268b-9140-4f73-8abf-c1c30fc706bc","Type":"ContainerStarted","Data":"1692c79b3b5029464adb38ea8859fa57c7c55358414e0576d9a8757906c99411"} May 26 15:23:44.145283 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:44.145260 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-76dff48d59-qd2tx" May 26 15:23:44.145670 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:44.145290 2547 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-76dff48d59-qd2tx" May 26 15:23:44.145670 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:44.145659 2547 scope.go:117] "RemoveContainer" containerID="ba1f1c3faf37af59cd908b38d94e6701bd2e4738cecead8b89a1317caed08299" May 26 15:23:44.145854 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:23:44.145836 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-76dff48d59-qd2tx_openshift-console-operator(dd56d3c2-38ff-4bd8-9893-6b643455a976)\"" pod="openshift-console-operator/console-operator-76dff48d59-qd2tx" podUID="dd56d3c2-38ff-4bd8-9893-6b643455a976" May 26 15:23:44.432468 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:44.432420 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mkhbh" event={"ID":"12d2a1e5-82ec-4b4d-9eb1-fac4406e2005","Type":"ContainerStarted","Data":"452b255976c1f35fc9b130a20af877ac341ea94f572240ae4665def43258df7f"} May 26 15:23:44.449251 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:44.449200 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-mkhbh" podStartSLOduration=1.286067235 podStartE2EDuration="3.44918563s" podCreationTimestamp="2026-05-26 15:23:41 +0000 UTC" firstStartedPulling="2026-05-26 15:23:41.982119083 +0000 UTC m=+170.708793988" lastFinishedPulling="2026-05-26 15:23:44.145237479 +0000 UTC m=+172.871912383" observedRunningTime="2026-05-26 15:23:44.447986747 +0000 UTC m=+173.174661669" watchObservedRunningTime="2026-05-26 15:23:44.44918563 +0000 UTC m=+173.175860556" May 26 15:23:44.909177 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:44.909157 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xh82" May 26 15:23:45.403484 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:45.403449 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-dzpbp" May 26 15:23:45.439922 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:45.439881 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8sxg9" event={"ID":"7b1e268b-9140-4f73-8abf-c1c30fc706bc","Type":"ContainerStarted","Data":"fe944106a08e147bf38237ecbbf201da418552f8d5967d90a198a5f07b232446"} May 26 15:23:45.454072 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:45.454016 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-8sxg9" podStartSLOduration=138.636206044 podStartE2EDuration="2m20.453998519s" podCreationTimestamp="2026-05-26 15:21:25 +0000 UTC" firstStartedPulling="2026-05-26 15:23:43.044148733 +0000 UTC m=+171.770823632" lastFinishedPulling="2026-05-26 15:23:44.861941194 +0000 UTC m=+173.588616107" observedRunningTime="2026-05-26 15:23:45.453596373 +0000 UTC m=+174.180271295" watchObservedRunningTime="2026-05-26 15:23:45.453998519 +0000 UTC m=+174.180673439" May 26 15:23:45.646460 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:45.646420 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-mkzgw\" (UID: \"a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-mkzgw" May 26 15:23:45.648830 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:45.648813 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-mkzgw\" (UID: \"a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-mkzgw" May 26 15:23:45.657694 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:45.657645 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-mkzgw" May 26 15:23:45.777780 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:45.777228 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-mkzgw"] May 26 15:23:46.443493 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:46.443460 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-mkzgw" event={"ID":"a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41","Type":"ContainerStarted","Data":"3b56cf71bd66574d5f5a18e4ce245da2933777545d6dadee1d6009f453c92327"} May 26 15:23:47.447256 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:47.447225 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-mkzgw" event={"ID":"a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41","Type":"ContainerStarted","Data":"f1262fa2140edd1be33a767a9ed9d845bba980a43c592c819774225f18d1054a"} May 26 15:23:56.249953 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.249886 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-mkzgw" podStartSLOduration=41.809257271 podStartE2EDuration="43.249867794s" podCreationTimestamp="2026-05-26 15:23:13 +0000 UTC" firstStartedPulling="2026-05-26 15:23:45.782214039 +0000 UTC m=+174.508888952" lastFinishedPulling="2026-05-26 15:23:47.222824556 +0000 UTC m=+175.949499475" observedRunningTime="2026-05-26 15:23:47.460967327 +0000 UTC m=+176.187642247" watchObservedRunningTime="2026-05-26 15:23:56.249867794 +0000 UTC m=+184.976542716" May 26 15:23:56.250600 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.250574 2547 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-54fd6bc5c4-k8flp"] May 26 15:23:56.252906 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.252883 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-k8flp" May 26 15:23:56.255254 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.255229 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" May 26 15:23:56.255748 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.255728 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" May 26 15:23:56.256034 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.256016 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" May 26 15:23:56.256174 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.256149 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" May 26 15:23:56.256256 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.256022 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" May 26 15:23:56.256375 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.256359 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-fj4gw\"" May 26 15:23:56.256439 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.256022 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" May 26 15:23:56.264473 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.264450 2547 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-j87gv"] May 26 15:23:56.266954 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.266936 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-54fd6bc5c4-k8flp"] May 26 15:23:56.267094 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.267080 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-j87gv" May 26 15:23:56.269298 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.269278 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" May 26 15:23:56.269429 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.269375 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" May 26 15:23:56.270320 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.270302 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5nr68\"" May 26 15:23:56.271333 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.271315 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" May 26 15:23:56.322322 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.322294 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a16892d4-26db-4f38-a60b-7a92ae470ae1-volume-directive-shadow\") pod \"kube-state-metrics-54fd6bc5c4-k8flp\" (UID: \"a16892d4-26db-4f38-a60b-7a92ae470ae1\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-k8flp" May 26 15:23:56.322452 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.322329 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcp4l\" (UniqueName: \"kubernetes.io/projected/a16892d4-26db-4f38-a60b-7a92ae470ae1-kube-api-access-zcp4l\") pod \"kube-state-metrics-54fd6bc5c4-k8flp\" (UID: \"a16892d4-26db-4f38-a60b-7a92ae470ae1\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-k8flp" May 26 15:23:56.322452 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.322379 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08d8a00f-3b89-4724-bce7-d3a7f395346b-metrics-client-ca\") pod \"node-exporter-j87gv\" (UID: \"08d8a00f-3b89-4724-bce7-d3a7f395346b\") " pod="openshift-monitoring/node-exporter-j87gv" May 26 15:23:56.322452 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.322444 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2cd9\" (UniqueName: \"kubernetes.io/projected/08d8a00f-3b89-4724-bce7-d3a7f395346b-kube-api-access-l2cd9\") pod \"node-exporter-j87gv\" (UID: \"08d8a00f-3b89-4724-bce7-d3a7f395346b\") " pod="openshift-monitoring/node-exporter-j87gv" May 26 15:23:56.322619 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.322475 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a16892d4-26db-4f38-a60b-7a92ae470ae1-metrics-client-ca\") pod \"kube-state-metrics-54fd6bc5c4-k8flp\" (UID: \"a16892d4-26db-4f38-a60b-7a92ae470ae1\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-k8flp" May 26 15:23:56.322619 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.322528 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/08d8a00f-3b89-4724-bce7-d3a7f395346b-node-exporter-wtmp\") pod \"node-exporter-j87gv\" (UID: \"08d8a00f-3b89-4724-bce7-d3a7f395346b\") " pod="openshift-monitoring/node-exporter-j87gv" May 26 15:23:56.322619 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.322557 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/08d8a00f-3b89-4724-bce7-d3a7f395346b-node-exporter-tls\") pod \"node-exporter-j87gv\" (UID: \"08d8a00f-3b89-4724-bce7-d3a7f395346b\") " pod="openshift-monitoring/node-exporter-j87gv" May 26 15:23:56.322619 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.322605 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a16892d4-26db-4f38-a60b-7a92ae470ae1-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-54fd6bc5c4-k8flp\" (UID: \"a16892d4-26db-4f38-a60b-7a92ae470ae1\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-k8flp" May 26 15:23:56.322779 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.322645 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/08d8a00f-3b89-4724-bce7-d3a7f395346b-node-exporter-textfile\") pod \"node-exporter-j87gv\" (UID: \"08d8a00f-3b89-4724-bce7-d3a7f395346b\") " pod="openshift-monitoring/node-exporter-j87gv" May 26 15:23:56.322779 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.322691 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/08d8a00f-3b89-4724-bce7-d3a7f395346b-node-exporter-accelerators-collector-config\") pod \"node-exporter-j87gv\" (UID: \"08d8a00f-3b89-4724-bce7-d3a7f395346b\") " pod="openshift-monitoring/node-exporter-j87gv" May 26 15:23:56.322779 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.322725 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/08d8a00f-3b89-4724-bce7-d3a7f395346b-root\") pod \"node-exporter-j87gv\" (UID: \"08d8a00f-3b89-4724-bce7-d3a7f395346b\") " pod="openshift-monitoring/node-exporter-j87gv" May 26 15:23:56.322892 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.322797 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a16892d4-26db-4f38-a60b-7a92ae470ae1-kube-state-metrics-tls\") pod \"kube-state-metrics-54fd6bc5c4-k8flp\" (UID: \"a16892d4-26db-4f38-a60b-7a92ae470ae1\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-k8flp" May 26 15:23:56.322892 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.322833 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a16892d4-26db-4f38-a60b-7a92ae470ae1-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-54fd6bc5c4-k8flp\" (UID: \"a16892d4-26db-4f38-a60b-7a92ae470ae1\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-k8flp" May 26 15:23:56.322892 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.322859 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08d8a00f-3b89-4724-bce7-d3a7f395346b-sys\") pod \"node-exporter-j87gv\" (UID: \"08d8a00f-3b89-4724-bce7-d3a7f395346b\") " pod="openshift-monitoring/node-exporter-j87gv" May 26 15:23:56.322998 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.322897 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/08d8a00f-3b89-4724-bce7-d3a7f395346b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-j87gv\" (UID: \"08d8a00f-3b89-4724-bce7-d3a7f395346b\") " pod="openshift-monitoring/node-exporter-j87gv" May 26 15:23:56.424006 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.423969 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/08d8a00f-3b89-4724-bce7-d3a7f395346b-node-exporter-tls\") pod \"node-exporter-j87gv\" (UID: \"08d8a00f-3b89-4724-bce7-d3a7f395346b\") " pod="openshift-monitoring/node-exporter-j87gv" May 26 15:23:56.424176 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.424022 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a16892d4-26db-4f38-a60b-7a92ae470ae1-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-54fd6bc5c4-k8flp\" (UID: \"a16892d4-26db-4f38-a60b-7a92ae470ae1\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-k8flp" May 26 15:23:56.424176 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.424076 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/08d8a00f-3b89-4724-bce7-d3a7f395346b-node-exporter-textfile\") pod \"node-exporter-j87gv\" (UID: \"08d8a00f-3b89-4724-bce7-d3a7f395346b\") " pod="openshift-monitoring/node-exporter-j87gv" May 26 15:23:56.424176 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.424103 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/08d8a00f-3b89-4724-bce7-d3a7f395346b-node-exporter-accelerators-collector-config\") pod \"node-exporter-j87gv\" (UID: \"08d8a00f-3b89-4724-bce7-d3a7f395346b\") " pod="openshift-monitoring/node-exporter-j87gv" May 26 15:23:56.424596 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.424573 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/08d8a00f-3b89-4724-bce7-d3a7f395346b-root\") pod \"node-exporter-j87gv\" (UID: \"08d8a00f-3b89-4724-bce7-d3a7f395346b\") " pod="openshift-monitoring/node-exporter-j87gv" May 26 15:23:56.424808 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.424773 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/08d8a00f-3b89-4724-bce7-d3a7f395346b-node-exporter-textfile\") pod \"node-exporter-j87gv\" (UID: \"08d8a00f-3b89-4724-bce7-d3a7f395346b\") " pod="openshift-monitoring/node-exporter-j87gv" May 26 15:23:56.424961 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.424775 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a16892d4-26db-4f38-a60b-7a92ae470ae1-kube-state-metrics-tls\") pod \"kube-state-metrics-54fd6bc5c4-k8flp\" (UID: \"a16892d4-26db-4f38-a60b-7a92ae470ae1\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-k8flp" May 26 15:23:56.425139 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.425107 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a16892d4-26db-4f38-a60b-7a92ae470ae1-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-54fd6bc5c4-k8flp\" (UID: \"a16892d4-26db-4f38-a60b-7a92ae470ae1\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-k8flp" May 26 15:23:56.425263 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.425239 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/08d8a00f-3b89-4724-bce7-d3a7f395346b-node-exporter-accelerators-collector-config\") pod \"node-exporter-j87gv\" (UID: \"08d8a00f-3b89-4724-bce7-d3a7f395346b\") " pod="openshift-monitoring/node-exporter-j87gv" May 26 15:23:56.426639 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.425250 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08d8a00f-3b89-4724-bce7-d3a7f395346b-sys\") pod \"node-exporter-j87gv\" (UID: \"08d8a00f-3b89-4724-bce7-d3a7f395346b\") " pod="openshift-monitoring/node-exporter-j87gv" May 26 15:23:56.426639 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.425418 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/08d8a00f-3b89-4724-bce7-d3a7f395346b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-j87gv\" (UID: \"08d8a00f-3b89-4724-bce7-d3a7f395346b\") " pod="openshift-monitoring/node-exporter-j87gv" May 26 15:23:56.426639 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.425465 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a16892d4-26db-4f38-a60b-7a92ae470ae1-volume-directive-shadow\") pod \"kube-state-metrics-54fd6bc5c4-k8flp\" (UID: \"a16892d4-26db-4f38-a60b-7a92ae470ae1\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-k8flp" May 26 15:23:56.426639 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.425497 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zcp4l\" (UniqueName: \"kubernetes.io/projected/a16892d4-26db-4f38-a60b-7a92ae470ae1-kube-api-access-zcp4l\") pod \"kube-state-metrics-54fd6bc5c4-k8flp\" (UID: \"a16892d4-26db-4f38-a60b-7a92ae470ae1\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-k8flp" May 26 15:23:56.426639 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.425550 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08d8a00f-3b89-4724-bce7-d3a7f395346b-metrics-client-ca\") pod \"node-exporter-j87gv\" (UID: \"08d8a00f-3b89-4724-bce7-d3a7f395346b\") " pod="openshift-monitoring/node-exporter-j87gv" May 26 15:23:56.426639 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.425589 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-l2cd9\" (UniqueName: \"kubernetes.io/projected/08d8a00f-3b89-4724-bce7-d3a7f395346b-kube-api-access-l2cd9\") pod \"node-exporter-j87gv\" (UID: \"08d8a00f-3b89-4724-bce7-d3a7f395346b\") " pod="openshift-monitoring/node-exporter-j87gv" May 26 15:23:56.426639 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.425660 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a16892d4-26db-4f38-a60b-7a92ae470ae1-metrics-client-ca\") pod \"kube-state-metrics-54fd6bc5c4-k8flp\" (UID: \"a16892d4-26db-4f38-a60b-7a92ae470ae1\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-k8flp" May 26 15:23:56.426639 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.425701 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/08d8a00f-3b89-4724-bce7-d3a7f395346b-node-exporter-wtmp\") pod \"node-exporter-j87gv\" (UID: \"08d8a00f-3b89-4724-bce7-d3a7f395346b\") " pod="openshift-monitoring/node-exporter-j87gv" May 26 15:23:56.426639 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.425884 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a16892d4-26db-4f38-a60b-7a92ae470ae1-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-54fd6bc5c4-k8flp\" (UID: \"a16892d4-26db-4f38-a60b-7a92ae470ae1\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-k8flp" May 26 15:23:56.426639 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.426372 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/08d8a00f-3b89-4724-bce7-d3a7f395346b-root\") pod \"node-exporter-j87gv\" (UID: \"08d8a00f-3b89-4724-bce7-d3a7f395346b\") " pod="openshift-monitoring/node-exporter-j87gv" May 26 15:23:56.426639 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.426506 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/08d8a00f-3b89-4724-bce7-d3a7f395346b-node-exporter-wtmp\") pod \"node-exporter-j87gv\" (UID: \"08d8a00f-3b89-4724-bce7-d3a7f395346b\") " pod="openshift-monitoring/node-exporter-j87gv" May 26 15:23:56.427269 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.427045 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08d8a00f-3b89-4724-bce7-d3a7f395346b-metrics-client-ca\") pod \"node-exporter-j87gv\" (UID: \"08d8a00f-3b89-4724-bce7-d3a7f395346b\") " pod="openshift-monitoring/node-exporter-j87gv" May 26 15:23:56.427446 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.427426 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08d8a00f-3b89-4724-bce7-d3a7f395346b-sys\") pod \"node-exporter-j87gv\" (UID: \"08d8a00f-3b89-4724-bce7-d3a7f395346b\") " pod="openshift-monitoring/node-exporter-j87gv" May 26 15:23:56.427901 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.427833 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a16892d4-26db-4f38-a60b-7a92ae470ae1-volume-directive-shadow\") pod \"kube-state-metrics-54fd6bc5c4-k8flp\" (UID: \"a16892d4-26db-4f38-a60b-7a92ae470ae1\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-k8flp" May 26 15:23:56.428646 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.428628 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a16892d4-26db-4f38-a60b-7a92ae470ae1-metrics-client-ca\") pod \"kube-state-metrics-54fd6bc5c4-k8flp\" (UID: \"a16892d4-26db-4f38-a60b-7a92ae470ae1\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-k8flp" May 26 15:23:56.432253 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.432231 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a16892d4-26db-4f38-a60b-7a92ae470ae1-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-54fd6bc5c4-k8flp\" (UID: \"a16892d4-26db-4f38-a60b-7a92ae470ae1\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-k8flp" May 26 15:23:56.433917 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.433891 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a16892d4-26db-4f38-a60b-7a92ae470ae1-kube-state-metrics-tls\") pod \"kube-state-metrics-54fd6bc5c4-k8flp\" (UID: \"a16892d4-26db-4f38-a60b-7a92ae470ae1\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-k8flp" May 26 15:23:56.434374 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.434354 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/08d8a00f-3b89-4724-bce7-d3a7f395346b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-j87gv\" (UID: \"08d8a00f-3b89-4724-bce7-d3a7f395346b\") " pod="openshift-monitoring/node-exporter-j87gv" May 26 15:23:56.434598 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.434551 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/08d8a00f-3b89-4724-bce7-d3a7f395346b-node-exporter-tls\") pod \"node-exporter-j87gv\" (UID: \"08d8a00f-3b89-4724-bce7-d3a7f395346b\") " pod="openshift-monitoring/node-exporter-j87gv" May 26 15:23:56.437377 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.437351 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2cd9\" (UniqueName: \"kubernetes.io/projected/08d8a00f-3b89-4724-bce7-d3a7f395346b-kube-api-access-l2cd9\") pod \"node-exporter-j87gv\" (UID: \"08d8a00f-3b89-4724-bce7-d3a7f395346b\") " pod="openshift-monitoring/node-exporter-j87gv" May 26 15:23:56.438499 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.438482 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcp4l\" (UniqueName: \"kubernetes.io/projected/a16892d4-26db-4f38-a60b-7a92ae470ae1-kube-api-access-zcp4l\") pod \"kube-state-metrics-54fd6bc5c4-k8flp\" (UID: \"a16892d4-26db-4f38-a60b-7a92ae470ae1\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-k8flp" May 26 15:23:56.563147 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.563044 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-k8flp" May 26 15:23:56.578027 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.578006 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-j87gv" May 26 15:23:56.585960 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:23:56.585923 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08d8a00f_3b89_4724_bce7_d3a7f395346b.slice/crio-df24613f399b613054781f6e69c30df17ca889c31908bf05faa22568e5c6df30 WatchSource:0}: Error finding container df24613f399b613054781f6e69c30df17ca889c31908bf05faa22568e5c6df30: Status 404 returned error can't find the container with id df24613f399b613054781f6e69c30df17ca889c31908bf05faa22568e5c6df30 May 26 15:23:56.692357 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:56.692322 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-54fd6bc5c4-k8flp"] May 26 15:23:56.696517 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:23:56.696489 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda16892d4_26db_4f38_a60b_7a92ae470ae1.slice/crio-b7f328ce977743500b37480869311a2b20407491e71f1a3d27587c6bca3b049a WatchSource:0}: Error finding container b7f328ce977743500b37480869311a2b20407491e71f1a3d27587c6bca3b049a: Status 404 returned error can't find the container with id b7f328ce977743500b37480869311a2b20407491e71f1a3d27587c6bca3b049a May 26 15:23:57.473187 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:57.473113 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-j87gv" event={"ID":"08d8a00f-3b89-4724-bce7-d3a7f395346b","Type":"ContainerStarted","Data":"df24613f399b613054781f6e69c30df17ca889c31908bf05faa22568e5c6df30"} May 26 15:23:57.474477 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:57.474447 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-k8flp" event={"ID":"a16892d4-26db-4f38-a60b-7a92ae470ae1","Type":"ContainerStarted","Data":"b7f328ce977743500b37480869311a2b20407491e71f1a3d27587c6bca3b049a"} May 26 15:23:58.478268 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:58.478236 2547 generic.go:358] "Generic (PLEG): container finished" podID="08d8a00f-3b89-4724-bce7-d3a7f395346b" containerID="1a0970636a98638336b3e86d66e48b0d64a7ea18d8d5f0bbe9e253623e8ace79" exitCode=0 May 26 15:23:58.478665 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:58.478308 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-j87gv" event={"ID":"08d8a00f-3b89-4724-bce7-d3a7f395346b","Type":"ContainerDied","Data":"1a0970636a98638336b3e86d66e48b0d64a7ea18d8d5f0bbe9e253623e8ace79"} May 26 15:23:58.480198 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:58.480174 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-k8flp" event={"ID":"a16892d4-26db-4f38-a60b-7a92ae470ae1","Type":"ContainerStarted","Data":"03c6c3dd9dd27d12d2d2aa7ee7e74f5e3622ae42430c42758418e52ae0d5189b"} May 26 15:23:58.480288 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:58.480201 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-k8flp" event={"ID":"a16892d4-26db-4f38-a60b-7a92ae470ae1","Type":"ContainerStarted","Data":"142bcd3f9518366d643666c4865fcbd9762ac1f33b36bcbe5e4f4d1f54cf7a60"} May 26 15:23:58.480288 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:58.480210 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-k8flp" event={"ID":"a16892d4-26db-4f38-a60b-7a92ae470ae1","Type":"ContainerStarted","Data":"688c68705d5dbd4fef9023faf1b35750763a5cd51eaaf5e47c6646eaea7add80"} May 26 15:23:58.510817 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:58.510775 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-k8flp" podStartSLOduration=1.30981862 podStartE2EDuration="2.510761813s" podCreationTimestamp="2026-05-26 15:23:56 +0000 UTC" firstStartedPulling="2026-05-26 15:23:56.69840063 +0000 UTC m=+185.425075529" lastFinishedPulling="2026-05-26 15:23:57.899343821 +0000 UTC m=+186.626018722" observedRunningTime="2026-05-26 15:23:58.510185509 +0000 UTC m=+187.236860429" watchObservedRunningTime="2026-05-26 15:23:58.510761813 +0000 UTC m=+187.237436735" May 26 15:23:58.909893 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:58.909822 2547 scope.go:117] "RemoveContainer" containerID="ba1f1c3faf37af59cd908b38d94e6701bd2e4738cecead8b89a1317caed08299" May 26 15:23:58.910029 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:23:58.910009 2547 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-76dff48d59-qd2tx_openshift-console-operator(dd56d3c2-38ff-4bd8-9893-6b643455a976)\"" pod="openshift-console-operator/console-operator-76dff48d59-qd2tx" podUID="dd56d3c2-38ff-4bd8-9893-6b643455a976" May 26 15:23:59.484824 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:59.484782 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-j87gv" event={"ID":"08d8a00f-3b89-4724-bce7-d3a7f395346b","Type":"ContainerStarted","Data":"c1fb29fadf951b32253bccd324d0bbf10f16af9f4f420f24b1d0f17eb1f60d70"} May 26 15:23:59.485211 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:59.484830 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-j87gv" event={"ID":"08d8a00f-3b89-4724-bce7-d3a7f395346b","Type":"ContainerStarted","Data":"dba77d16bcbbffe7e62e9c412110a70fbd5604bca97da145338fb47167e15a9c"} May 26 15:23:59.506689 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:23:59.506643 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-j87gv" podStartSLOduration=2.227933136 podStartE2EDuration="3.506628959s" podCreationTimestamp="2026-05-26 15:23:56 +0000 UTC" firstStartedPulling="2026-05-26 15:23:56.587451595 +0000 UTC m=+185.314126494" lastFinishedPulling="2026-05-26 15:23:57.866147419 +0000 UTC m=+186.592822317" observedRunningTime="2026-05-26 15:23:59.505501733 +0000 UTC m=+188.232176654" watchObservedRunningTime="2026-05-26 15:23:59.506628959 +0000 UTC m=+188.233303893" May 26 15:24:00.793329 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:00.793298 2547 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-84b5bf67c7-925fq"] May 26 15:24:00.795541 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:00.795522 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-84b5bf67c7-925fq" May 26 15:24:00.798819 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:00.798680 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-ddgq8\"" May 26 15:24:00.798819 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:00.798696 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" May 26 15:24:00.798819 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:00.798695 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" May 26 15:24:00.798819 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:00.798725 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" May 26 15:24:00.798819 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:00.798696 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-ca2kgv6fgrmj6\"" May 26 15:24:00.798819 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:00.798694 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" May 26 15:24:00.808079 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:00.808034 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-84b5bf67c7-925fq"] May 26 15:24:00.865031 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:00.864996 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a4426673-24c2-480f-9a9d-6ec20a2f1ba0-secret-metrics-server-tls\") pod \"metrics-server-84b5bf67c7-925fq\" (UID: \"a4426673-24c2-480f-9a9d-6ec20a2f1ba0\") " pod="openshift-monitoring/metrics-server-84b5bf67c7-925fq" May 26 15:24:00.865202 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:00.865041 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c44ms\" (UniqueName: \"kubernetes.io/projected/a4426673-24c2-480f-9a9d-6ec20a2f1ba0-kube-api-access-c44ms\") pod \"metrics-server-84b5bf67c7-925fq\" (UID: \"a4426673-24c2-480f-9a9d-6ec20a2f1ba0\") " pod="openshift-monitoring/metrics-server-84b5bf67c7-925fq" May 26 15:24:00.865202 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:00.865111 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4426673-24c2-480f-9a9d-6ec20a2f1ba0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-84b5bf67c7-925fq\" (UID: \"a4426673-24c2-480f-9a9d-6ec20a2f1ba0\") " pod="openshift-monitoring/metrics-server-84b5bf67c7-925fq" May 26 15:24:00.865202 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:00.865170 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a4426673-24c2-480f-9a9d-6ec20a2f1ba0-client-ca-bundle\") pod \"metrics-server-84b5bf67c7-925fq\" (UID: \"a4426673-24c2-480f-9a9d-6ec20a2f1ba0\") " pod="openshift-monitoring/metrics-server-84b5bf67c7-925fq" May 26 15:24:00.865311 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:00.865227 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a4426673-24c2-480f-9a9d-6ec20a2f1ba0-secret-metrics-server-client-certs\") pod \"metrics-server-84b5bf67c7-925fq\" (UID: \"a4426673-24c2-480f-9a9d-6ec20a2f1ba0\") " pod="openshift-monitoring/metrics-server-84b5bf67c7-925fq" May 26 15:24:00.865311 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:00.865258 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a4426673-24c2-480f-9a9d-6ec20a2f1ba0-audit-log\") pod \"metrics-server-84b5bf67c7-925fq\" (UID: \"a4426673-24c2-480f-9a9d-6ec20a2f1ba0\") " pod="openshift-monitoring/metrics-server-84b5bf67c7-925fq" May 26 15:24:00.865311 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:00.865276 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a4426673-24c2-480f-9a9d-6ec20a2f1ba0-metrics-server-audit-profiles\") pod \"metrics-server-84b5bf67c7-925fq\" (UID: \"a4426673-24c2-480f-9a9d-6ec20a2f1ba0\") " pod="openshift-monitoring/metrics-server-84b5bf67c7-925fq" May 26 15:24:00.966436 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:00.966404 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a4426673-24c2-480f-9a9d-6ec20a2f1ba0-secret-metrics-server-client-certs\") pod \"metrics-server-84b5bf67c7-925fq\" (UID: \"a4426673-24c2-480f-9a9d-6ec20a2f1ba0\") " pod="openshift-monitoring/metrics-server-84b5bf67c7-925fq" May 26 15:24:00.966617 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:00.966446 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a4426673-24c2-480f-9a9d-6ec20a2f1ba0-audit-log\") pod \"metrics-server-84b5bf67c7-925fq\" (UID: \"a4426673-24c2-480f-9a9d-6ec20a2f1ba0\") " pod="openshift-monitoring/metrics-server-84b5bf67c7-925fq" May 26 15:24:00.966617 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:00.966465 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a4426673-24c2-480f-9a9d-6ec20a2f1ba0-metrics-server-audit-profiles\") pod \"metrics-server-84b5bf67c7-925fq\" (UID: \"a4426673-24c2-480f-9a9d-6ec20a2f1ba0\") " pod="openshift-monitoring/metrics-server-84b5bf67c7-925fq" May 26 15:24:00.966617 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:00.966496 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a4426673-24c2-480f-9a9d-6ec20a2f1ba0-secret-metrics-server-tls\") pod \"metrics-server-84b5bf67c7-925fq\" (UID: \"a4426673-24c2-480f-9a9d-6ec20a2f1ba0\") " pod="openshift-monitoring/metrics-server-84b5bf67c7-925fq" May 26 15:24:00.966617 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:00.966538 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-c44ms\" (UniqueName: \"kubernetes.io/projected/a4426673-24c2-480f-9a9d-6ec20a2f1ba0-kube-api-access-c44ms\") pod \"metrics-server-84b5bf67c7-925fq\" (UID: \"a4426673-24c2-480f-9a9d-6ec20a2f1ba0\") " pod="openshift-monitoring/metrics-server-84b5bf67c7-925fq" May 26 15:24:00.966617 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:00.966578 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4426673-24c2-480f-9a9d-6ec20a2f1ba0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-84b5bf67c7-925fq\" (UID: \"a4426673-24c2-480f-9a9d-6ec20a2f1ba0\") " pod="openshift-monitoring/metrics-server-84b5bf67c7-925fq" May 26 15:24:00.966847 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:00.966623 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a4426673-24c2-480f-9a9d-6ec20a2f1ba0-client-ca-bundle\") pod \"metrics-server-84b5bf67c7-925fq\" (UID: \"a4426673-24c2-480f-9a9d-6ec20a2f1ba0\") " pod="openshift-monitoring/metrics-server-84b5bf67c7-925fq" May 26 15:24:00.966901 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:00.966884 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a4426673-24c2-480f-9a9d-6ec20a2f1ba0-audit-log\") pod \"metrics-server-84b5bf67c7-925fq\" (UID: \"a4426673-24c2-480f-9a9d-6ec20a2f1ba0\") " pod="openshift-monitoring/metrics-server-84b5bf67c7-925fq" May 26 15:24:00.967351 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:00.967325 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4426673-24c2-480f-9a9d-6ec20a2f1ba0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-84b5bf67c7-925fq\" (UID: \"a4426673-24c2-480f-9a9d-6ec20a2f1ba0\") " pod="openshift-monitoring/metrics-server-84b5bf67c7-925fq" May 26 15:24:00.967944 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:00.967924 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a4426673-24c2-480f-9a9d-6ec20a2f1ba0-metrics-server-audit-profiles\") pod \"metrics-server-84b5bf67c7-925fq\" (UID: \"a4426673-24c2-480f-9a9d-6ec20a2f1ba0\") " pod="openshift-monitoring/metrics-server-84b5bf67c7-925fq" May 26 15:24:00.969479 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:00.969450 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a4426673-24c2-480f-9a9d-6ec20a2f1ba0-secret-metrics-server-tls\") pod \"metrics-server-84b5bf67c7-925fq\" (UID: \"a4426673-24c2-480f-9a9d-6ec20a2f1ba0\") " pod="openshift-monitoring/metrics-server-84b5bf67c7-925fq" May 26 15:24:00.969592 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:00.969505 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a4426673-24c2-480f-9a9d-6ec20a2f1ba0-secret-metrics-server-client-certs\") pod \"metrics-server-84b5bf67c7-925fq\" (UID: \"a4426673-24c2-480f-9a9d-6ec20a2f1ba0\") " pod="openshift-monitoring/metrics-server-84b5bf67c7-925fq" May 26 15:24:00.969592 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:00.969515 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a4426673-24c2-480f-9a9d-6ec20a2f1ba0-client-ca-bundle\") pod \"metrics-server-84b5bf67c7-925fq\" (UID: \"a4426673-24c2-480f-9a9d-6ec20a2f1ba0\") " pod="openshift-monitoring/metrics-server-84b5bf67c7-925fq" May 26 15:24:00.981769 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:00.981731 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c44ms\" (UniqueName: \"kubernetes.io/projected/a4426673-24c2-480f-9a9d-6ec20a2f1ba0-kube-api-access-c44ms\") pod \"metrics-server-84b5bf67c7-925fq\" (UID: \"a4426673-24c2-480f-9a9d-6ec20a2f1ba0\") " pod="openshift-monitoring/metrics-server-84b5bf67c7-925fq" May 26 15:24:01.031549 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:01.031514 2547 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-665c9c8584-5gppl"] May 26 15:24:01.037646 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:01.037619 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-665c9c8584-5gppl" May 26 15:24:01.040946 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:01.040917 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"default-dockercfg-spgb2\"" May 26 15:24:01.041172 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:01.041153 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" May 26 15:24:01.050536 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:01.050513 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-665c9c8584-5gppl"] May 26 15:24:01.067658 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:01.067631 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/812f69f2-f125-432a-8244-920534eba136-monitoring-plugin-cert\") pod \"monitoring-plugin-665c9c8584-5gppl\" (UID: \"812f69f2-f125-432a-8244-920534eba136\") " pod="openshift-monitoring/monitoring-plugin-665c9c8584-5gppl" May 26 15:24:01.104332 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:01.104311 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-84b5bf67c7-925fq" May 26 15:24:01.168428 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:01.168329 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/812f69f2-f125-432a-8244-920534eba136-monitoring-plugin-cert\") pod \"monitoring-plugin-665c9c8584-5gppl\" (UID: \"812f69f2-f125-432a-8244-920534eba136\") " pod="openshift-monitoring/monitoring-plugin-665c9c8584-5gppl" May 26 15:24:01.171125 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:01.171102 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/812f69f2-f125-432a-8244-920534eba136-monitoring-plugin-cert\") pod \"monitoring-plugin-665c9c8584-5gppl\" (UID: \"812f69f2-f125-432a-8244-920534eba136\") " pod="openshift-monitoring/monitoring-plugin-665c9c8584-5gppl" May 26 15:24:01.239852 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:01.239787 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-84b5bf67c7-925fq"] May 26 15:24:01.242333 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:24:01.242297 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4426673_24c2_480f_9a9d_6ec20a2f1ba0.slice/crio-ec9c72b962ac8d416695ee0f7b112a5dc2fd55c72dbe5cf282c6e9e1fbd7d1b2 WatchSource:0}: Error finding container ec9c72b962ac8d416695ee0f7b112a5dc2fd55c72dbe5cf282c6e9e1fbd7d1b2: Status 404 returned error can't find the container with id ec9c72b962ac8d416695ee0f7b112a5dc2fd55c72dbe5cf282c6e9e1fbd7d1b2 May 26 15:24:01.347679 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:01.347609 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-665c9c8584-5gppl" May 26 15:24:01.470174 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:01.470142 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-665c9c8584-5gppl"] May 26 15:24:01.472894 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:24:01.472872 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod812f69f2_f125_432a_8244_920534eba136.slice/crio-4b9b05c3ff53f8f993ee03714ecc32f2340dd818b875bd01218725cffcaa6ec7 WatchSource:0}: Error finding container 4b9b05c3ff53f8f993ee03714ecc32f2340dd818b875bd01218725cffcaa6ec7: Status 404 returned error can't find the container with id 4b9b05c3ff53f8f993ee03714ecc32f2340dd818b875bd01218725cffcaa6ec7 May 26 15:24:01.493277 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:01.493244 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-84b5bf67c7-925fq" event={"ID":"a4426673-24c2-480f-9a9d-6ec20a2f1ba0","Type":"ContainerStarted","Data":"ec9c72b962ac8d416695ee0f7b112a5dc2fd55c72dbe5cf282c6e9e1fbd7d1b2"} May 26 15:24:01.494277 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:01.494250 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-665c9c8584-5gppl" event={"ID":"812f69f2-f125-432a-8244-920534eba136","Type":"ContainerStarted","Data":"4b9b05c3ff53f8f993ee03714ecc32f2340dd818b875bd01218725cffcaa6ec7"} May 26 15:24:03.431198 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:03.431167 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6b645d86d4-g9z92" May 26 15:24:03.502448 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:03.502410 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-665c9c8584-5gppl" event={"ID":"812f69f2-f125-432a-8244-920534eba136","Type":"ContainerStarted","Data":"4d0d88d49a030400ce5698e4592947d7e9e823b3ac71f89f8bad61acddcaaaf3"} May 26 15:24:03.502633 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:03.502579 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-665c9c8584-5gppl" May 26 15:24:03.503902 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:03.503878 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-84b5bf67c7-925fq" event={"ID":"a4426673-24c2-480f-9a9d-6ec20a2f1ba0","Type":"ContainerStarted","Data":"d913d3b199b62513dbbc2a5efae3df1c51f9f02d2235d27945beb55f4ecc9420"} May 26 15:24:03.508479 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:03.508460 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-665c9c8584-5gppl" May 26 15:24:03.524046 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:03.524009 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-665c9c8584-5gppl" podStartSLOduration=0.905569784 podStartE2EDuration="2.523997693s" podCreationTimestamp="2026-05-26 15:24:01 +0000 UTC" firstStartedPulling="2026-05-26 15:24:01.474769743 +0000 UTC m=+190.201444643" lastFinishedPulling="2026-05-26 15:24:03.093197648 +0000 UTC m=+191.819872552" observedRunningTime="2026-05-26 15:24:03.522752585 +0000 UTC m=+192.249427505" watchObservedRunningTime="2026-05-26 15:24:03.523997693 +0000 UTC m=+192.250672613" May 26 15:24:03.566513 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:03.566465 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-84b5bf67c7-925fq" podStartSLOduration=1.71981663 podStartE2EDuration="3.566452154s" podCreationTimestamp="2026-05-26 15:24:00 +0000 UTC" firstStartedPulling="2026-05-26 15:24:01.243757462 +0000 UTC m=+189.970432362" lastFinishedPulling="2026-05-26 15:24:03.090392983 +0000 UTC m=+191.817067886" observedRunningTime="2026-05-26 15:24:03.564560637 +0000 UTC m=+192.291235559" watchObservedRunningTime="2026-05-26 15:24:03.566452154 +0000 UTC m=+192.293127087" May 26 15:24:12.909644 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:12.909616 2547 scope.go:117] "RemoveContainer" containerID="ba1f1c3faf37af59cd908b38d94e6701bd2e4738cecead8b89a1317caed08299" May 26 15:24:13.533954 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:13.533928 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-qd2tx_dd56d3c2-38ff-4bd8-9893-6b643455a976/console-operator/2.log" May 26 15:24:13.534158 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:13.534074 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-qd2tx" event={"ID":"dd56d3c2-38ff-4bd8-9893-6b643455a976","Type":"ContainerStarted","Data":"6261ca2273d20194734b7d67d3af82460dc4b36438ece1674a33b11b4831a1c9"} May 26 15:24:13.534373 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:13.534350 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-76dff48d59-qd2tx" May 26 15:24:13.551892 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:13.551850 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console-operator/console-operator-76dff48d59-qd2tx" podStartSLOduration=57.174814081 podStartE2EDuration="1m0.551837437s" podCreationTimestamp="2026-05-26 15:23:13 +0000 UTC" firstStartedPulling="2026-05-26 15:23:14.277860061 +0000 UTC m=+143.004534960" lastFinishedPulling="2026-05-26 15:23:17.654883403 +0000 UTC m=+146.381558316" observedRunningTime="2026-05-26 15:24:13.5510929 +0000 UTC m=+202.277767821" watchObservedRunningTime="2026-05-26 15:24:13.551837437 +0000 UTC m=+202.278512357" May 26 15:24:13.839734 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:13.839656 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-76dff48d59-qd2tx" May 26 15:24:14.020502 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:14.020474 2547 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-57bd96b888-krxkd"] May 26 15:24:14.023429 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:14.023411 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-57bd96b888-krxkd" May 26 15:24:14.026024 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:14.026002 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" May 26 15:24:14.026402 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:14.026383 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-4x2xz\"" May 26 15:24:14.026837 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:14.026812 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" May 26 15:24:14.031040 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:14.031020 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-57bd96b888-krxkd"] May 26 15:24:14.065804 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:14.065777 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88m9z\" (UniqueName: \"kubernetes.io/projected/1e3a0692-61b5-4201-beaf-ac3df33be93b-kube-api-access-88m9z\") pod \"downloads-57bd96b888-krxkd\" (UID: \"1e3a0692-61b5-4201-beaf-ac3df33be93b\") " pod="openshift-console/downloads-57bd96b888-krxkd" May 26 15:24:14.167030 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:14.166964 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-88m9z\" (UniqueName: \"kubernetes.io/projected/1e3a0692-61b5-4201-beaf-ac3df33be93b-kube-api-access-88m9z\") pod \"downloads-57bd96b888-krxkd\" (UID: \"1e3a0692-61b5-4201-beaf-ac3df33be93b\") " pod="openshift-console/downloads-57bd96b888-krxkd" May 26 15:24:14.174955 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:14.174931 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-88m9z\" (UniqueName: \"kubernetes.io/projected/1e3a0692-61b5-4201-beaf-ac3df33be93b-kube-api-access-88m9z\") pod \"downloads-57bd96b888-krxkd\" (UID: \"1e3a0692-61b5-4201-beaf-ac3df33be93b\") " pod="openshift-console/downloads-57bd96b888-krxkd" May 26 15:24:14.333440 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:14.333404 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-57bd96b888-krxkd" May 26 15:24:14.455458 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:14.455432 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-57bd96b888-krxkd"] May 26 15:24:14.458680 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:24:14.458652 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e3a0692_61b5_4201_beaf_ac3df33be93b.slice/crio-4857ec5b84f16f108ed75407515e5aa003411d736e95ae3f224fcee626c90d7b WatchSource:0}: Error finding container 4857ec5b84f16f108ed75407515e5aa003411d736e95ae3f224fcee626c90d7b: Status 404 returned error can't find the container with id 4857ec5b84f16f108ed75407515e5aa003411d736e95ae3f224fcee626c90d7b May 26 15:24:14.537205 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:14.537176 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-57bd96b888-krxkd" event={"ID":"1e3a0692-61b5-4201-beaf-ac3df33be93b","Type":"ContainerStarted","Data":"4857ec5b84f16f108ed75407515e5aa003411d736e95ae3f224fcee626c90d7b"} May 26 15:24:21.105487 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:21.105436 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-84b5bf67c7-925fq" May 26 15:24:21.105929 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:21.105504 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-84b5bf67c7-925fq" May 26 15:24:23.460144 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:23.460107 2547 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-8469668989-6tjfg"] May 26 15:24:23.467925 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:23.467900 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8469668989-6tjfg" May 26 15:24:23.471075 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:23.470880 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" May 26 15:24:23.471075 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:23.470910 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" May 26 15:24:23.471075 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:23.470965 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" May 26 15:24:23.471312 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:23.471118 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5dh48\"" May 26 15:24:23.471312 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:23.471131 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" May 26 15:24:23.471312 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:23.470889 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" May 26 15:24:23.473369 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:23.473348 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8469668989-6tjfg"] May 26 15:24:23.550366 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:23.550334 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxpkn\" (UniqueName: \"kubernetes.io/projected/87ec28cf-1c76-45e1-9af4-07ddf4bdcca8-kube-api-access-fxpkn\") pod \"console-8469668989-6tjfg\" (UID: \"87ec28cf-1c76-45e1-9af4-07ddf4bdcca8\") " pod="openshift-console/console-8469668989-6tjfg" May 26 15:24:23.550533 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:23.550405 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/87ec28cf-1c76-45e1-9af4-07ddf4bdcca8-console-oauth-config\") pod \"console-8469668989-6tjfg\" (UID: \"87ec28cf-1c76-45e1-9af4-07ddf4bdcca8\") " pod="openshift-console/console-8469668989-6tjfg" May 26 15:24:23.550533 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:23.550429 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/87ec28cf-1c76-45e1-9af4-07ddf4bdcca8-service-ca\") pod \"console-8469668989-6tjfg\" (UID: \"87ec28cf-1c76-45e1-9af4-07ddf4bdcca8\") " pod="openshift-console/console-8469668989-6tjfg" May 26 15:24:23.550533 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:23.550469 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/87ec28cf-1c76-45e1-9af4-07ddf4bdcca8-console-config\") pod \"console-8469668989-6tjfg\" (UID: \"87ec28cf-1c76-45e1-9af4-07ddf4bdcca8\") " pod="openshift-console/console-8469668989-6tjfg" May 26 15:24:23.550533 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:23.550502 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/87ec28cf-1c76-45e1-9af4-07ddf4bdcca8-oauth-serving-cert\") pod \"console-8469668989-6tjfg\" (UID: \"87ec28cf-1c76-45e1-9af4-07ddf4bdcca8\") " pod="openshift-console/console-8469668989-6tjfg" May 26 15:24:23.550533 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:23.550520 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/87ec28cf-1c76-45e1-9af4-07ddf4bdcca8-console-serving-cert\") pod \"console-8469668989-6tjfg\" (UID: \"87ec28cf-1c76-45e1-9af4-07ddf4bdcca8\") " pod="openshift-console/console-8469668989-6tjfg" May 26 15:24:23.651897 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:23.651852 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/87ec28cf-1c76-45e1-9af4-07ddf4bdcca8-console-oauth-config\") pod \"console-8469668989-6tjfg\" (UID: \"87ec28cf-1c76-45e1-9af4-07ddf4bdcca8\") " pod="openshift-console/console-8469668989-6tjfg" May 26 15:24:23.651897 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:23.651897 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/87ec28cf-1c76-45e1-9af4-07ddf4bdcca8-service-ca\") pod \"console-8469668989-6tjfg\" (UID: \"87ec28cf-1c76-45e1-9af4-07ddf4bdcca8\") " pod="openshift-console/console-8469668989-6tjfg" May 26 15:24:23.652177 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:23.652027 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/87ec28cf-1c76-45e1-9af4-07ddf4bdcca8-console-config\") pod \"console-8469668989-6tjfg\" (UID: \"87ec28cf-1c76-45e1-9af4-07ddf4bdcca8\") " pod="openshift-console/console-8469668989-6tjfg" May 26 15:24:23.652177 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:23.652111 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/87ec28cf-1c76-45e1-9af4-07ddf4bdcca8-oauth-serving-cert\") pod \"console-8469668989-6tjfg\" (UID: \"87ec28cf-1c76-45e1-9af4-07ddf4bdcca8\") " pod="openshift-console/console-8469668989-6tjfg" May 26 15:24:23.652177 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:23.652133 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/87ec28cf-1c76-45e1-9af4-07ddf4bdcca8-console-serving-cert\") pod \"console-8469668989-6tjfg\" (UID: \"87ec28cf-1c76-45e1-9af4-07ddf4bdcca8\") " pod="openshift-console/console-8469668989-6tjfg" May 26 15:24:23.652338 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:23.652180 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fxpkn\" (UniqueName: \"kubernetes.io/projected/87ec28cf-1c76-45e1-9af4-07ddf4bdcca8-kube-api-access-fxpkn\") pod \"console-8469668989-6tjfg\" (UID: \"87ec28cf-1c76-45e1-9af4-07ddf4bdcca8\") " pod="openshift-console/console-8469668989-6tjfg" May 26 15:24:23.652726 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:23.652696 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/87ec28cf-1c76-45e1-9af4-07ddf4bdcca8-service-ca\") pod \"console-8469668989-6tjfg\" (UID: \"87ec28cf-1c76-45e1-9af4-07ddf4bdcca8\") " pod="openshift-console/console-8469668989-6tjfg" May 26 15:24:23.652836 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:23.652774 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/87ec28cf-1c76-45e1-9af4-07ddf4bdcca8-console-config\") pod \"console-8469668989-6tjfg\" (UID: \"87ec28cf-1c76-45e1-9af4-07ddf4bdcca8\") " pod="openshift-console/console-8469668989-6tjfg" May 26 15:24:23.652836 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:23.652813 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/87ec28cf-1c76-45e1-9af4-07ddf4bdcca8-oauth-serving-cert\") pod \"console-8469668989-6tjfg\" (UID: \"87ec28cf-1c76-45e1-9af4-07ddf4bdcca8\") " pod="openshift-console/console-8469668989-6tjfg" May 26 15:24:23.654443 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:23.654423 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/87ec28cf-1c76-45e1-9af4-07ddf4bdcca8-console-oauth-config\") pod \"console-8469668989-6tjfg\" (UID: \"87ec28cf-1c76-45e1-9af4-07ddf4bdcca8\") " pod="openshift-console/console-8469668989-6tjfg" May 26 15:24:23.654534 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:23.654423 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/87ec28cf-1c76-45e1-9af4-07ddf4bdcca8-console-serving-cert\") pod \"console-8469668989-6tjfg\" (UID: \"87ec28cf-1c76-45e1-9af4-07ddf4bdcca8\") " pod="openshift-console/console-8469668989-6tjfg" May 26 15:24:23.659498 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:23.659470 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxpkn\" (UniqueName: \"kubernetes.io/projected/87ec28cf-1c76-45e1-9af4-07ddf4bdcca8-kube-api-access-fxpkn\") pod \"console-8469668989-6tjfg\" (UID: \"87ec28cf-1c76-45e1-9af4-07ddf4bdcca8\") " pod="openshift-console/console-8469668989-6tjfg" May 26 15:24:23.781043 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:23.780957 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8469668989-6tjfg" May 26 15:24:23.914890 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:23.914834 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8469668989-6tjfg"] May 26 15:24:23.917508 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:24:23.917478 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87ec28cf_1c76_45e1_9af4_07ddf4bdcca8.slice/crio-4c405e1a781121c58c8df9c4ac6e8d65129cf40cd33c2edaad4459dfe87054c6 WatchSource:0}: Error finding container 4c405e1a781121c58c8df9c4ac6e8d65129cf40cd33c2edaad4459dfe87054c6: Status 404 returned error can't find the container with id 4c405e1a781121c58c8df9c4ac6e8d65129cf40cd33c2edaad4459dfe87054c6 May 26 15:24:24.567550 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:24.567506 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8469668989-6tjfg" event={"ID":"87ec28cf-1c76-45e1-9af4-07ddf4bdcca8","Type":"ContainerStarted","Data":"4c405e1a781121c58c8df9c4ac6e8d65129cf40cd33c2edaad4459dfe87054c6"} May 26 15:24:31.124461 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:31.124421 2547 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-cd5fd7d5b-5rhrj"] May 26 15:24:31.126912 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:31.126887 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cd5fd7d5b-5rhrj" May 26 15:24:31.135003 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:31.134971 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" May 26 15:24:31.135381 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:31.135355 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cd5fd7d5b-5rhrj"] May 26 15:24:31.226186 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:31.226150 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d764c454-9207-40d7-97b8-febbae6f60c4-console-serving-cert\") pod \"console-cd5fd7d5b-5rhrj\" (UID: \"d764c454-9207-40d7-97b8-febbae6f60c4\") " pod="openshift-console/console-cd5fd7d5b-5rhrj" May 26 15:24:31.226367 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:31.226224 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d764c454-9207-40d7-97b8-febbae6f60c4-console-config\") pod \"console-cd5fd7d5b-5rhrj\" (UID: \"d764c454-9207-40d7-97b8-febbae6f60c4\") " pod="openshift-console/console-cd5fd7d5b-5rhrj" May 26 15:24:31.226367 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:31.226267 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d764c454-9207-40d7-97b8-febbae6f60c4-oauth-serving-cert\") pod \"console-cd5fd7d5b-5rhrj\" (UID: \"d764c454-9207-40d7-97b8-febbae6f60c4\") " pod="openshift-console/console-cd5fd7d5b-5rhrj" May 26 15:24:31.226367 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:31.226308 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7xk7\" (UniqueName: \"kubernetes.io/projected/d764c454-9207-40d7-97b8-febbae6f60c4-kube-api-access-p7xk7\") pod \"console-cd5fd7d5b-5rhrj\" (UID: \"d764c454-9207-40d7-97b8-febbae6f60c4\") " pod="openshift-console/console-cd5fd7d5b-5rhrj" May 26 15:24:31.226367 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:31.226353 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d764c454-9207-40d7-97b8-febbae6f60c4-service-ca\") pod \"console-cd5fd7d5b-5rhrj\" (UID: \"d764c454-9207-40d7-97b8-febbae6f60c4\") " pod="openshift-console/console-cd5fd7d5b-5rhrj" May 26 15:24:31.226541 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:31.226376 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d764c454-9207-40d7-97b8-febbae6f60c4-trusted-ca-bundle\") pod \"console-cd5fd7d5b-5rhrj\" (UID: \"d764c454-9207-40d7-97b8-febbae6f60c4\") " pod="openshift-console/console-cd5fd7d5b-5rhrj" May 26 15:24:31.226541 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:31.226457 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d764c454-9207-40d7-97b8-febbae6f60c4-console-oauth-config\") pod \"console-cd5fd7d5b-5rhrj\" (UID: \"d764c454-9207-40d7-97b8-febbae6f60c4\") " pod="openshift-console/console-cd5fd7d5b-5rhrj" May 26 15:24:31.327498 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:31.327454 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d764c454-9207-40d7-97b8-febbae6f60c4-console-serving-cert\") pod \"console-cd5fd7d5b-5rhrj\" (UID: \"d764c454-9207-40d7-97b8-febbae6f60c4\") " pod="openshift-console/console-cd5fd7d5b-5rhrj" May 26 15:24:31.327695 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:31.327525 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d764c454-9207-40d7-97b8-febbae6f60c4-console-config\") pod \"console-cd5fd7d5b-5rhrj\" (UID: \"d764c454-9207-40d7-97b8-febbae6f60c4\") " pod="openshift-console/console-cd5fd7d5b-5rhrj" May 26 15:24:31.327695 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:31.327565 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d764c454-9207-40d7-97b8-febbae6f60c4-oauth-serving-cert\") pod \"console-cd5fd7d5b-5rhrj\" (UID: \"d764c454-9207-40d7-97b8-febbae6f60c4\") " pod="openshift-console/console-cd5fd7d5b-5rhrj" May 26 15:24:31.327695 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:31.327607 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p7xk7\" (UniqueName: \"kubernetes.io/projected/d764c454-9207-40d7-97b8-febbae6f60c4-kube-api-access-p7xk7\") pod \"console-cd5fd7d5b-5rhrj\" (UID: \"d764c454-9207-40d7-97b8-febbae6f60c4\") " pod="openshift-console/console-cd5fd7d5b-5rhrj" May 26 15:24:31.327695 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:31.327633 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d764c454-9207-40d7-97b8-febbae6f60c4-service-ca\") pod \"console-cd5fd7d5b-5rhrj\" (UID: \"d764c454-9207-40d7-97b8-febbae6f60c4\") " pod="openshift-console/console-cd5fd7d5b-5rhrj" May 26 15:24:31.327695 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:31.327655 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d764c454-9207-40d7-97b8-febbae6f60c4-trusted-ca-bundle\") pod \"console-cd5fd7d5b-5rhrj\" (UID: \"d764c454-9207-40d7-97b8-febbae6f60c4\") " pod="openshift-console/console-cd5fd7d5b-5rhrj" May 26 15:24:31.327953 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:31.327713 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d764c454-9207-40d7-97b8-febbae6f60c4-console-oauth-config\") pod \"console-cd5fd7d5b-5rhrj\" (UID: \"d764c454-9207-40d7-97b8-febbae6f60c4\") " pod="openshift-console/console-cd5fd7d5b-5rhrj" May 26 15:24:31.328364 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:31.328335 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d764c454-9207-40d7-97b8-febbae6f60c4-console-config\") pod \"console-cd5fd7d5b-5rhrj\" (UID: \"d764c454-9207-40d7-97b8-febbae6f60c4\") " pod="openshift-console/console-cd5fd7d5b-5rhrj" May 26 15:24:31.328480 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:31.328424 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d764c454-9207-40d7-97b8-febbae6f60c4-service-ca\") pod \"console-cd5fd7d5b-5rhrj\" (UID: \"d764c454-9207-40d7-97b8-febbae6f60c4\") " pod="openshift-console/console-cd5fd7d5b-5rhrj" May 26 15:24:31.328934 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:31.328911 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d764c454-9207-40d7-97b8-febbae6f60c4-oauth-serving-cert\") pod \"console-cd5fd7d5b-5rhrj\" (UID: \"d764c454-9207-40d7-97b8-febbae6f60c4\") " pod="openshift-console/console-cd5fd7d5b-5rhrj" May 26 15:24:31.329135 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:31.329092 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d764c454-9207-40d7-97b8-febbae6f60c4-trusted-ca-bundle\") pod \"console-cd5fd7d5b-5rhrj\" (UID: \"d764c454-9207-40d7-97b8-febbae6f60c4\") " pod="openshift-console/console-cd5fd7d5b-5rhrj" May 26 15:24:31.330791 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:31.330770 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d764c454-9207-40d7-97b8-febbae6f60c4-console-serving-cert\") pod \"console-cd5fd7d5b-5rhrj\" (UID: \"d764c454-9207-40d7-97b8-febbae6f60c4\") " pod="openshift-console/console-cd5fd7d5b-5rhrj" May 26 15:24:31.330892 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:31.330851 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d764c454-9207-40d7-97b8-febbae6f60c4-console-oauth-config\") pod \"console-cd5fd7d5b-5rhrj\" (UID: \"d764c454-9207-40d7-97b8-febbae6f60c4\") " pod="openshift-console/console-cd5fd7d5b-5rhrj" May 26 15:24:31.336991 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:31.336954 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7xk7\" (UniqueName: \"kubernetes.io/projected/d764c454-9207-40d7-97b8-febbae6f60c4-kube-api-access-p7xk7\") pod \"console-cd5fd7d5b-5rhrj\" (UID: \"d764c454-9207-40d7-97b8-febbae6f60c4\") " pod="openshift-console/console-cd5fd7d5b-5rhrj" May 26 15:24:31.438181 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:31.437946 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cd5fd7d5b-5rhrj" May 26 15:24:31.591344 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:31.590169 2547 generic.go:358] "Generic (PLEG): container finished" podID="c380575f-335e-4023-9b3a-277b8b5fcbbf" containerID="6cbf6430d0f50255a4ec3e4e1095bf5272a5fd9d5e7ffbc97405514bdd4f6809" exitCode=0 May 26 15:24:31.591344 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:31.590320 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-mrfrn" event={"ID":"c380575f-335e-4023-9b3a-277b8b5fcbbf","Type":"ContainerDied","Data":"6cbf6430d0f50255a4ec3e4e1095bf5272a5fd9d5e7ffbc97405514bdd4f6809"} May 26 15:24:31.591344 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:31.590770 2547 scope.go:117] "RemoveContainer" containerID="6cbf6430d0f50255a4ec3e4e1095bf5272a5fd9d5e7ffbc97405514bdd4f6809" May 26 15:24:31.594174 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:31.593414 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-57bd96b888-krxkd" event={"ID":"1e3a0692-61b5-4201-beaf-ac3df33be93b","Type":"ContainerStarted","Data":"944ad42787cd7f954fa84c83cc154e3abd681296dc83b00fae82fb529c74cd12"} May 26 15:24:31.594174 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:31.594128 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-57bd96b888-krxkd" May 26 15:24:31.604641 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:31.604618 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cd5fd7d5b-5rhrj"] May 26 15:24:31.605370 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:31.605348 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-57bd96b888-krxkd" May 26 15:24:31.608803 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:24:31.608771 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd764c454_9207_40d7_97b8_febbae6f60c4.slice/crio-82a57c0bb3f846e31b30c44a5105f8675ea6ee992b33730ea2c49e2d30b8c4cb WatchSource:0}: Error finding container 82a57c0bb3f846e31b30c44a5105f8675ea6ee992b33730ea2c49e2d30b8c4cb: Status 404 returned error can't find the container with id 82a57c0bb3f846e31b30c44a5105f8675ea6ee992b33730ea2c49e2d30b8c4cb May 26 15:24:31.621144 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:31.621098 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/downloads-57bd96b888-krxkd" podStartSLOduration=1.11003609 podStartE2EDuration="17.62108029s" podCreationTimestamp="2026-05-26 15:24:14 +0000 UTC" firstStartedPulling="2026-05-26 15:24:14.460489421 +0000 UTC m=+203.187164324" lastFinishedPulling="2026-05-26 15:24:30.971533624 +0000 UTC m=+219.698208524" observedRunningTime="2026-05-26 15:24:31.619711647 +0000 UTC m=+220.346386582" watchObservedRunningTime="2026-05-26 15:24:31.62108029 +0000 UTC m=+220.347755209" May 26 15:24:32.602326 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:32.602254 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cd5fd7d5b-5rhrj" event={"ID":"d764c454-9207-40d7-97b8-febbae6f60c4","Type":"ContainerStarted","Data":"82a57c0bb3f846e31b30c44a5105f8675ea6ee992b33730ea2c49e2d30b8c4cb"} May 26 15:24:32.606204 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:32.606090 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-mrfrn" event={"ID":"c380575f-335e-4023-9b3a-277b8b5fcbbf","Type":"ContainerStarted","Data":"c4be576bc2e7ebe94db0e933e30004274d124e40ad22e751d5bd7ae7fd072a58"} May 26 15:24:35.620475 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:35.620432 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8469668989-6tjfg" event={"ID":"87ec28cf-1c76-45e1-9af4-07ddf4bdcca8","Type":"ContainerStarted","Data":"fd8c80e7312884a0d9d4e0b9b48f222ec4f0bb545e211d16e18a94d3a3c152df"} May 26 15:24:35.622357 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:35.622325 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cd5fd7d5b-5rhrj" event={"ID":"d764c454-9207-40d7-97b8-febbae6f60c4","Type":"ContainerStarted","Data":"e85c4c28f5ee20388378c9a873eeff2268ccb40fecbfc150e2c7d18bae9160a6"} May 26 15:24:35.636745 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:35.636698 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-8469668989-6tjfg" podStartSLOduration=1.267582708 podStartE2EDuration="12.636683808s" podCreationTimestamp="2026-05-26 15:24:23 +0000 UTC" firstStartedPulling="2026-05-26 15:24:23.919707407 +0000 UTC m=+212.646382306" lastFinishedPulling="2026-05-26 15:24:35.28880849 +0000 UTC m=+224.015483406" observedRunningTime="2026-05-26 15:24:35.635232552 +0000 UTC m=+224.361907473" watchObservedRunningTime="2026-05-26 15:24:35.636683808 +0000 UTC m=+224.363358729" May 26 15:24:35.651750 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:35.651701 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-cd5fd7d5b-5rhrj" podStartSLOduration=0.967369325 podStartE2EDuration="4.651688647s" podCreationTimestamp="2026-05-26 15:24:31 +0000 UTC" firstStartedPulling="2026-05-26 15:24:31.611992694 +0000 UTC m=+220.338667610" lastFinishedPulling="2026-05-26 15:24:35.296312028 +0000 UTC m=+224.022986932" observedRunningTime="2026-05-26 15:24:35.650157552 +0000 UTC m=+224.376832474" watchObservedRunningTime="2026-05-26 15:24:35.651688647 +0000 UTC m=+224.378363569" May 26 15:24:41.110509 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:41.110478 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-84b5bf67c7-925fq" May 26 15:24:41.114601 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:41.114573 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-84b5bf67c7-925fq" May 26 15:24:41.438646 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:41.438617 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-cd5fd7d5b-5rhrj" May 26 15:24:41.438881 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:41.438657 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-cd5fd7d5b-5rhrj" May 26 15:24:41.443181 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:41.443160 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-cd5fd7d5b-5rhrj" May 26 15:24:41.641289 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:41.641263 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-cd5fd7d5b-5rhrj" May 26 15:24:41.715463 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:41.715388 2547 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8469668989-6tjfg"] May 26 15:24:43.781587 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:43.781558 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8469668989-6tjfg" May 26 15:24:44.646502 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:44.646468 2547 generic.go:358] "Generic (PLEG): container finished" podID="90be60c1-e9f6-476d-9a27-f050b27cff15" containerID="9732e6b113a75c3305ea05d3fbac1b4d9cb33e17007e130709e966adf64e0d48" exitCode=0 May 26 15:24:44.646661 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:44.646541 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-78bdfb888c-ht62j" event={"ID":"90be60c1-e9f6-476d-9a27-f050b27cff15","Type":"ContainerDied","Data":"9732e6b113a75c3305ea05d3fbac1b4d9cb33e17007e130709e966adf64e0d48"} May 26 15:24:44.646879 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:44.646865 2547 scope.go:117] "RemoveContainer" containerID="9732e6b113a75c3305ea05d3fbac1b4d9cb33e17007e130709e966adf64e0d48" May 26 15:24:45.650974 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:24:45.650944 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-78bdfb888c-ht62j" event={"ID":"90be60c1-e9f6-476d-9a27-f050b27cff15","Type":"ContainerStarted","Data":"4fe49d9065f5199b8f75d1a31c4da3330d613ce61e769096c209e2e419888004"} May 26 15:25:02.596166 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:02.596132 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fac6c416-4fef-4b1b-846f-ea63a14c9efe-metrics-certs\") pod \"network-metrics-daemon-5xh82\" (UID: \"fac6c416-4fef-4b1b-846f-ea63a14c9efe\") " pod="openshift-multus/network-metrics-daemon-5xh82" May 26 15:25:02.598466 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:02.598442 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fac6c416-4fef-4b1b-846f-ea63a14c9efe-metrics-certs\") pod \"network-metrics-daemon-5xh82\" (UID: \"fac6c416-4fef-4b1b-846f-ea63a14c9efe\") " pod="openshift-multus/network-metrics-daemon-5xh82" May 26 15:25:02.611663 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:02.611641 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pgdkb\"" May 26 15:25:02.620276 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:02.620260 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5xh82" May 26 15:25:02.744101 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:02.743999 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5xh82"] May 26 15:25:02.758151 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:25:02.758121 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfac6c416_4fef_4b1b_846f_ea63a14c9efe.slice/crio-cf6b1bb8890f7df6cb90c2137d0714f84eff4421daec06c29e6ddbeb480c9c05 WatchSource:0}: Error finding container cf6b1bb8890f7df6cb90c2137d0714f84eff4421daec06c29e6ddbeb480c9c05: Status 404 returned error can't find the container with id cf6b1bb8890f7df6cb90c2137d0714f84eff4421daec06c29e6ddbeb480c9c05 May 26 15:25:03.705767 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:03.705733 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5xh82" event={"ID":"fac6c416-4fef-4b1b-846f-ea63a14c9efe","Type":"ContainerStarted","Data":"cf6b1bb8890f7df6cb90c2137d0714f84eff4421daec06c29e6ddbeb480c9c05"} May 26 15:25:04.710674 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:04.710631 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5xh82" event={"ID":"fac6c416-4fef-4b1b-846f-ea63a14c9efe","Type":"ContainerStarted","Data":"c483ed249f76d98c9e9e0a972c204d614d04f29e2ff48a2c282244a73fa2db77"} May 26 15:25:04.710674 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:04.710678 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5xh82" event={"ID":"fac6c416-4fef-4b1b-846f-ea63a14c9efe","Type":"ContainerStarted","Data":"4b86f16757bd4968bb2098feca4200f95f61f23ccf196e04b13a44f079285b37"} May 26 15:25:04.727413 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:04.727367 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-5xh82" podStartSLOduration=252.213149187 podStartE2EDuration="4m13.727352919s" podCreationTimestamp="2026-05-26 15:20:51 +0000 UTC" firstStartedPulling="2026-05-26 15:25:02.759923634 +0000 UTC m=+251.486598533" lastFinishedPulling="2026-05-26 15:25:04.274127364 +0000 UTC m=+253.000802265" observedRunningTime="2026-05-26 15:25:04.725805446 +0000 UTC m=+253.452480370" watchObservedRunningTime="2026-05-26 15:25:04.727352919 +0000 UTC m=+253.454027840" May 26 15:25:06.735736 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:06.735641 2547 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-8469668989-6tjfg" podUID="87ec28cf-1c76-45e1-9af4-07ddf4bdcca8" containerName="console" containerID="cri-o://fd8c80e7312884a0d9d4e0b9b48f222ec4f0bb545e211d16e18a94d3a3c152df" gracePeriod=15 May 26 15:25:06.978591 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:06.978569 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-8469668989-6tjfg_87ec28cf-1c76-45e1-9af4-07ddf4bdcca8/console/0.log" May 26 15:25:06.978718 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:06.978655 2547 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8469668989-6tjfg" May 26 15:25:07.029019 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:07.028939 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxpkn\" (UniqueName: \"kubernetes.io/projected/87ec28cf-1c76-45e1-9af4-07ddf4bdcca8-kube-api-access-fxpkn\") pod \"87ec28cf-1c76-45e1-9af4-07ddf4bdcca8\" (UID: \"87ec28cf-1c76-45e1-9af4-07ddf4bdcca8\") " May 26 15:25:07.029019 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:07.028979 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/87ec28cf-1c76-45e1-9af4-07ddf4bdcca8-service-ca\") pod \"87ec28cf-1c76-45e1-9af4-07ddf4bdcca8\" (UID: \"87ec28cf-1c76-45e1-9af4-07ddf4bdcca8\") " May 26 15:25:07.029019 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:07.029000 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/87ec28cf-1c76-45e1-9af4-07ddf4bdcca8-oauth-serving-cert\") pod \"87ec28cf-1c76-45e1-9af4-07ddf4bdcca8\" (UID: \"87ec28cf-1c76-45e1-9af4-07ddf4bdcca8\") " May 26 15:25:07.029296 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:07.029051 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/87ec28cf-1c76-45e1-9af4-07ddf4bdcca8-console-config\") pod \"87ec28cf-1c76-45e1-9af4-07ddf4bdcca8\" (UID: \"87ec28cf-1c76-45e1-9af4-07ddf4bdcca8\") " May 26 15:25:07.029296 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:07.029102 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/87ec28cf-1c76-45e1-9af4-07ddf4bdcca8-console-oauth-config\") pod \"87ec28cf-1c76-45e1-9af4-07ddf4bdcca8\" (UID: \"87ec28cf-1c76-45e1-9af4-07ddf4bdcca8\") " May 26 15:25:07.029296 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:07.029149 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/87ec28cf-1c76-45e1-9af4-07ddf4bdcca8-console-serving-cert\") pod \"87ec28cf-1c76-45e1-9af4-07ddf4bdcca8\" (UID: \"87ec28cf-1c76-45e1-9af4-07ddf4bdcca8\") " May 26 15:25:07.029442 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:07.029407 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87ec28cf-1c76-45e1-9af4-07ddf4bdcca8-service-ca" (OuterVolumeSpecName: "service-ca") pod "87ec28cf-1c76-45e1-9af4-07ddf4bdcca8" (UID: "87ec28cf-1c76-45e1-9af4-07ddf4bdcca8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 26 15:25:07.029496 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:07.029461 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87ec28cf-1c76-45e1-9af4-07ddf4bdcca8-console-config" (OuterVolumeSpecName: "console-config") pod "87ec28cf-1c76-45e1-9af4-07ddf4bdcca8" (UID: "87ec28cf-1c76-45e1-9af4-07ddf4bdcca8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 26 15:25:07.029496 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:07.029479 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87ec28cf-1c76-45e1-9af4-07ddf4bdcca8-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "87ec28cf-1c76-45e1-9af4-07ddf4bdcca8" (UID: "87ec28cf-1c76-45e1-9af4-07ddf4bdcca8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 26 15:25:07.031272 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:07.031242 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87ec28cf-1c76-45e1-9af4-07ddf4bdcca8-kube-api-access-fxpkn" (OuterVolumeSpecName: "kube-api-access-fxpkn") pod "87ec28cf-1c76-45e1-9af4-07ddf4bdcca8" (UID: "87ec28cf-1c76-45e1-9af4-07ddf4bdcca8"). InnerVolumeSpecName "kube-api-access-fxpkn". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 26 15:25:07.031370 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:07.031303 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87ec28cf-1c76-45e1-9af4-07ddf4bdcca8-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "87ec28cf-1c76-45e1-9af4-07ddf4bdcca8" (UID: "87ec28cf-1c76-45e1-9af4-07ddf4bdcca8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 26 15:25:07.031370 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:07.031333 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87ec28cf-1c76-45e1-9af4-07ddf4bdcca8-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "87ec28cf-1c76-45e1-9af4-07ddf4bdcca8" (UID: "87ec28cf-1c76-45e1-9af4-07ddf4bdcca8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 26 15:25:07.129883 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:07.129858 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxpkn\" (UniqueName: \"kubernetes.io/projected/87ec28cf-1c76-45e1-9af4-07ddf4bdcca8-kube-api-access-fxpkn\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:25:07.129883 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:07.129881 2547 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/87ec28cf-1c76-45e1-9af4-07ddf4bdcca8-service-ca\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:25:07.130033 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:07.129892 2547 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/87ec28cf-1c76-45e1-9af4-07ddf4bdcca8-oauth-serving-cert\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:25:07.130033 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:07.129901 2547 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/87ec28cf-1c76-45e1-9af4-07ddf4bdcca8-console-config\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:25:07.130033 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:07.129911 2547 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/87ec28cf-1c76-45e1-9af4-07ddf4bdcca8-console-oauth-config\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:25:07.130033 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:07.129919 2547 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/87ec28cf-1c76-45e1-9af4-07ddf4bdcca8-console-serving-cert\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:25:07.719366 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:07.719340 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-8469668989-6tjfg_87ec28cf-1c76-45e1-9af4-07ddf4bdcca8/console/0.log" May 26 15:25:07.719546 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:07.719378 2547 generic.go:358] "Generic (PLEG): container finished" podID="87ec28cf-1c76-45e1-9af4-07ddf4bdcca8" containerID="fd8c80e7312884a0d9d4e0b9b48f222ec4f0bb545e211d16e18a94d3a3c152df" exitCode=2 May 26 15:25:07.719546 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:07.719440 2547 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8469668989-6tjfg" May 26 15:25:07.719546 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:07.719479 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8469668989-6tjfg" event={"ID":"87ec28cf-1c76-45e1-9af4-07ddf4bdcca8","Type":"ContainerDied","Data":"fd8c80e7312884a0d9d4e0b9b48f222ec4f0bb545e211d16e18a94d3a3c152df"} May 26 15:25:07.719546 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:07.719526 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8469668989-6tjfg" event={"ID":"87ec28cf-1c76-45e1-9af4-07ddf4bdcca8","Type":"ContainerDied","Data":"4c405e1a781121c58c8df9c4ac6e8d65129cf40cd33c2edaad4459dfe87054c6"} May 26 15:25:07.719546 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:07.719546 2547 scope.go:117] "RemoveContainer" containerID="fd8c80e7312884a0d9d4e0b9b48f222ec4f0bb545e211d16e18a94d3a3c152df" May 26 15:25:07.727751 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:07.727737 2547 scope.go:117] "RemoveContainer" containerID="fd8c80e7312884a0d9d4e0b9b48f222ec4f0bb545e211d16e18a94d3a3c152df" May 26 15:25:07.727997 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:25:07.727975 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd8c80e7312884a0d9d4e0b9b48f222ec4f0bb545e211d16e18a94d3a3c152df\": container with ID starting with fd8c80e7312884a0d9d4e0b9b48f222ec4f0bb545e211d16e18a94d3a3c152df not found: ID does not exist" containerID="fd8c80e7312884a0d9d4e0b9b48f222ec4f0bb545e211d16e18a94d3a3c152df" May 26 15:25:07.728042 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:07.728005 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd8c80e7312884a0d9d4e0b9b48f222ec4f0bb545e211d16e18a94d3a3c152df"} err="failed to get container status \"fd8c80e7312884a0d9d4e0b9b48f222ec4f0bb545e211d16e18a94d3a3c152df\": rpc error: code = NotFound desc = could not find container \"fd8c80e7312884a0d9d4e0b9b48f222ec4f0bb545e211d16e18a94d3a3c152df\": container with ID starting with fd8c80e7312884a0d9d4e0b9b48f222ec4f0bb545e211d16e18a94d3a3c152df not found: ID does not exist" May 26 15:25:07.739502 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:07.739481 2547 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8469668989-6tjfg"] May 26 15:25:07.744623 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:07.744600 2547 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8469668989-6tjfg"] May 26 15:25:07.913275 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:07.913247 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87ec28cf-1c76-45e1-9af4-07ddf4bdcca8" path="/var/lib/kubelet/pods/87ec28cf-1c76-45e1-9af4-07ddf4bdcca8/volumes" May 26 15:25:14.221914 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:14.221879 2547 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-65f77d89dd-qlvlt"] May 26 15:25:14.222557 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:14.222510 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87ec28cf-1c76-45e1-9af4-07ddf4bdcca8" containerName="console" May 26 15:25:14.222557 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:14.222554 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="87ec28cf-1c76-45e1-9af4-07ddf4bdcca8" containerName="console" May 26 15:25:14.222694 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:14.222608 2547 memory_manager.go:356] "RemoveStaleState removing state" podUID="87ec28cf-1c76-45e1-9af4-07ddf4bdcca8" containerName="console" May 26 15:25:14.256431 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:14.256403 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65f77d89dd-qlvlt"] May 26 15:25:14.256560 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:14.256477 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65f77d89dd-qlvlt" May 26 15:25:14.385995 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:14.385964 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98947d39-4497-4bcf-bb34-9c88b251a8bf-trusted-ca-bundle\") pod \"console-65f77d89dd-qlvlt\" (UID: \"98947d39-4497-4bcf-bb34-9c88b251a8bf\") " pod="openshift-console/console-65f77d89dd-qlvlt" May 26 15:25:14.386144 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:14.385999 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/98947d39-4497-4bcf-bb34-9c88b251a8bf-console-serving-cert\") pod \"console-65f77d89dd-qlvlt\" (UID: \"98947d39-4497-4bcf-bb34-9c88b251a8bf\") " pod="openshift-console/console-65f77d89dd-qlvlt" May 26 15:25:14.386144 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:14.386023 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/98947d39-4497-4bcf-bb34-9c88b251a8bf-console-config\") pod \"console-65f77d89dd-qlvlt\" (UID: \"98947d39-4497-4bcf-bb34-9c88b251a8bf\") " pod="openshift-console/console-65f77d89dd-qlvlt" May 26 15:25:14.386144 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:14.386096 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/98947d39-4497-4bcf-bb34-9c88b251a8bf-console-oauth-config\") pod \"console-65f77d89dd-qlvlt\" (UID: \"98947d39-4497-4bcf-bb34-9c88b251a8bf\") " pod="openshift-console/console-65f77d89dd-qlvlt" May 26 15:25:14.386261 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:14.386151 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/98947d39-4497-4bcf-bb34-9c88b251a8bf-service-ca\") pod \"console-65f77d89dd-qlvlt\" (UID: \"98947d39-4497-4bcf-bb34-9c88b251a8bf\") " pod="openshift-console/console-65f77d89dd-qlvlt" May 26 15:25:14.386261 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:14.386168 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/98947d39-4497-4bcf-bb34-9c88b251a8bf-oauth-serving-cert\") pod \"console-65f77d89dd-qlvlt\" (UID: \"98947d39-4497-4bcf-bb34-9c88b251a8bf\") " pod="openshift-console/console-65f77d89dd-qlvlt" May 26 15:25:14.386261 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:14.386201 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbbt6\" (UniqueName: \"kubernetes.io/projected/98947d39-4497-4bcf-bb34-9c88b251a8bf-kube-api-access-gbbt6\") pod \"console-65f77d89dd-qlvlt\" (UID: \"98947d39-4497-4bcf-bb34-9c88b251a8bf\") " pod="openshift-console/console-65f77d89dd-qlvlt" May 26 15:25:14.487106 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:14.487016 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/98947d39-4497-4bcf-bb34-9c88b251a8bf-console-serving-cert\") pod \"console-65f77d89dd-qlvlt\" (UID: \"98947d39-4497-4bcf-bb34-9c88b251a8bf\") " pod="openshift-console/console-65f77d89dd-qlvlt" May 26 15:25:14.487106 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:14.487071 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/98947d39-4497-4bcf-bb34-9c88b251a8bf-console-config\") pod \"console-65f77d89dd-qlvlt\" (UID: \"98947d39-4497-4bcf-bb34-9c88b251a8bf\") " pod="openshift-console/console-65f77d89dd-qlvlt" May 26 15:25:14.487281 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:14.487158 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/98947d39-4497-4bcf-bb34-9c88b251a8bf-console-oauth-config\") pod \"console-65f77d89dd-qlvlt\" (UID: \"98947d39-4497-4bcf-bb34-9c88b251a8bf\") " pod="openshift-console/console-65f77d89dd-qlvlt" May 26 15:25:14.487281 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:14.487186 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/98947d39-4497-4bcf-bb34-9c88b251a8bf-service-ca\") pod \"console-65f77d89dd-qlvlt\" (UID: \"98947d39-4497-4bcf-bb34-9c88b251a8bf\") " pod="openshift-console/console-65f77d89dd-qlvlt" May 26 15:25:14.487281 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:14.487201 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/98947d39-4497-4bcf-bb34-9c88b251a8bf-oauth-serving-cert\") pod \"console-65f77d89dd-qlvlt\" (UID: \"98947d39-4497-4bcf-bb34-9c88b251a8bf\") " pod="openshift-console/console-65f77d89dd-qlvlt" May 26 15:25:14.487281 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:14.487216 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gbbt6\" (UniqueName: \"kubernetes.io/projected/98947d39-4497-4bcf-bb34-9c88b251a8bf-kube-api-access-gbbt6\") pod \"console-65f77d89dd-qlvlt\" (UID: \"98947d39-4497-4bcf-bb34-9c88b251a8bf\") " pod="openshift-console/console-65f77d89dd-qlvlt" May 26 15:25:14.487281 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:14.487266 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98947d39-4497-4bcf-bb34-9c88b251a8bf-trusted-ca-bundle\") pod \"console-65f77d89dd-qlvlt\" (UID: \"98947d39-4497-4bcf-bb34-9c88b251a8bf\") " pod="openshift-console/console-65f77d89dd-qlvlt" May 26 15:25:14.487935 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:14.487903 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/98947d39-4497-4bcf-bb34-9c88b251a8bf-service-ca\") pod \"console-65f77d89dd-qlvlt\" (UID: \"98947d39-4497-4bcf-bb34-9c88b251a8bf\") " pod="openshift-console/console-65f77d89dd-qlvlt" May 26 15:25:14.488079 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:14.487952 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/98947d39-4497-4bcf-bb34-9c88b251a8bf-console-config\") pod \"console-65f77d89dd-qlvlt\" (UID: \"98947d39-4497-4bcf-bb34-9c88b251a8bf\") " pod="openshift-console/console-65f77d89dd-qlvlt" May 26 15:25:14.488079 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:14.488024 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98947d39-4497-4bcf-bb34-9c88b251a8bf-trusted-ca-bundle\") pod \"console-65f77d89dd-qlvlt\" (UID: \"98947d39-4497-4bcf-bb34-9c88b251a8bf\") " pod="openshift-console/console-65f77d89dd-qlvlt" May 26 15:25:14.488079 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:14.488040 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/98947d39-4497-4bcf-bb34-9c88b251a8bf-oauth-serving-cert\") pod \"console-65f77d89dd-qlvlt\" (UID: \"98947d39-4497-4bcf-bb34-9c88b251a8bf\") " pod="openshift-console/console-65f77d89dd-qlvlt" May 26 15:25:14.489474 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:14.489453 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/98947d39-4497-4bcf-bb34-9c88b251a8bf-console-serving-cert\") pod \"console-65f77d89dd-qlvlt\" (UID: \"98947d39-4497-4bcf-bb34-9c88b251a8bf\") " pod="openshift-console/console-65f77d89dd-qlvlt" May 26 15:25:14.489795 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:14.489775 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/98947d39-4497-4bcf-bb34-9c88b251a8bf-console-oauth-config\") pod \"console-65f77d89dd-qlvlt\" (UID: \"98947d39-4497-4bcf-bb34-9c88b251a8bf\") " pod="openshift-console/console-65f77d89dd-qlvlt" May 26 15:25:14.494619 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:14.494602 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbbt6\" (UniqueName: \"kubernetes.io/projected/98947d39-4497-4bcf-bb34-9c88b251a8bf-kube-api-access-gbbt6\") pod \"console-65f77d89dd-qlvlt\" (UID: \"98947d39-4497-4bcf-bb34-9c88b251a8bf\") " pod="openshift-console/console-65f77d89dd-qlvlt" May 26 15:25:14.571636 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:14.571613 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65f77d89dd-qlvlt" May 26 15:25:14.696222 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:14.696197 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65f77d89dd-qlvlt"] May 26 15:25:14.698685 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:25:14.698656 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98947d39_4497_4bcf_bb34_9c88b251a8bf.slice/crio-670d023fe641f1a17fc93480d695f1878068ede0ed55d64bfee26ba580e2d8f2 WatchSource:0}: Error finding container 670d023fe641f1a17fc93480d695f1878068ede0ed55d64bfee26ba580e2d8f2: Status 404 returned error can't find the container with id 670d023fe641f1a17fc93480d695f1878068ede0ed55d64bfee26ba580e2d8f2 May 26 15:25:14.742051 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:14.741991 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65f77d89dd-qlvlt" event={"ID":"98947d39-4497-4bcf-bb34-9c88b251a8bf","Type":"ContainerStarted","Data":"670d023fe641f1a17fc93480d695f1878068ede0ed55d64bfee26ba580e2d8f2"} May 26 15:25:15.746367 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:15.746327 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65f77d89dd-qlvlt" event={"ID":"98947d39-4497-4bcf-bb34-9c88b251a8bf","Type":"ContainerStarted","Data":"62918e4ecf4108d5017c099fb39d29d24f3e21e85d11552f3d3cd035540962a6"} May 26 15:25:15.787994 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:15.787946 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-65f77d89dd-qlvlt" podStartSLOduration=1.7879317270000001 podStartE2EDuration="1.787931727s" podCreationTimestamp="2026-05-26 15:25:14 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 15:25:15.785900128 +0000 UTC m=+264.512575062" watchObservedRunningTime="2026-05-26 15:25:15.787931727 +0000 UTC m=+264.514606649" May 26 15:25:24.572488 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:24.572450 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-65f77d89dd-qlvlt" May 26 15:25:24.572847 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:24.572529 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-65f77d89dd-qlvlt" May 26 15:25:24.577073 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:24.577034 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-65f77d89dd-qlvlt" May 26 15:25:24.774641 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:24.774610 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-65f77d89dd-qlvlt" May 26 15:25:24.818731 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:24.818701 2547 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cd5fd7d5b-5rhrj"] May 26 15:25:33.025321 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:33.025285 2547 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-7d47p"] May 26 15:25:33.027663 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:33.027640 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7d47p" May 26 15:25:33.029641 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:33.029621 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" May 26 15:25:33.035185 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:33.035164 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7d47p"] May 26 15:25:33.137201 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:33.137175 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ba53dd75-3f69-4a05-aad2-72d6859eeb38-kubelet-config\") pod \"global-pull-secret-syncer-7d47p\" (UID: \"ba53dd75-3f69-4a05-aad2-72d6859eeb38\") " pod="kube-system/global-pull-secret-syncer-7d47p" May 26 15:25:33.137317 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:33.137272 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ba53dd75-3f69-4a05-aad2-72d6859eeb38-dbus\") pod \"global-pull-secret-syncer-7d47p\" (UID: \"ba53dd75-3f69-4a05-aad2-72d6859eeb38\") " pod="kube-system/global-pull-secret-syncer-7d47p" May 26 15:25:33.137317 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:33.137306 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba53dd75-3f69-4a05-aad2-72d6859eeb38-original-pull-secret\") pod \"global-pull-secret-syncer-7d47p\" (UID: \"ba53dd75-3f69-4a05-aad2-72d6859eeb38\") " pod="kube-system/global-pull-secret-syncer-7d47p" May 26 15:25:33.238325 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:33.238298 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ba53dd75-3f69-4a05-aad2-72d6859eeb38-dbus\") pod \"global-pull-secret-syncer-7d47p\" (UID: \"ba53dd75-3f69-4a05-aad2-72d6859eeb38\") " pod="kube-system/global-pull-secret-syncer-7d47p" May 26 15:25:33.238466 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:33.238336 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba53dd75-3f69-4a05-aad2-72d6859eeb38-original-pull-secret\") pod \"global-pull-secret-syncer-7d47p\" (UID: \"ba53dd75-3f69-4a05-aad2-72d6859eeb38\") " pod="kube-system/global-pull-secret-syncer-7d47p" May 26 15:25:33.238466 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:33.238363 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ba53dd75-3f69-4a05-aad2-72d6859eeb38-kubelet-config\") pod \"global-pull-secret-syncer-7d47p\" (UID: \"ba53dd75-3f69-4a05-aad2-72d6859eeb38\") " pod="kube-system/global-pull-secret-syncer-7d47p" May 26 15:25:33.238466 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:33.238430 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ba53dd75-3f69-4a05-aad2-72d6859eeb38-kubelet-config\") pod \"global-pull-secret-syncer-7d47p\" (UID: \"ba53dd75-3f69-4a05-aad2-72d6859eeb38\") " pod="kube-system/global-pull-secret-syncer-7d47p" May 26 15:25:33.238567 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:33.238473 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ba53dd75-3f69-4a05-aad2-72d6859eeb38-dbus\") pod \"global-pull-secret-syncer-7d47p\" (UID: \"ba53dd75-3f69-4a05-aad2-72d6859eeb38\") " pod="kube-system/global-pull-secret-syncer-7d47p" May 26 15:25:33.240523 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:33.240497 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba53dd75-3f69-4a05-aad2-72d6859eeb38-original-pull-secret\") pod \"global-pull-secret-syncer-7d47p\" (UID: \"ba53dd75-3f69-4a05-aad2-72d6859eeb38\") " pod="kube-system/global-pull-secret-syncer-7d47p" May 26 15:25:33.337702 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:33.337645 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7d47p" May 26 15:25:33.486293 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:33.486270 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7d47p"] May 26 15:25:33.488614 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:25:33.488584 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba53dd75_3f69_4a05_aad2_72d6859eeb38.slice/crio-791c41749a5afa46b4115cd059b475a475b84d00b546c238be99ab9691ede44d WatchSource:0}: Error finding container 791c41749a5afa46b4115cd059b475a475b84d00b546c238be99ab9691ede44d: Status 404 returned error can't find the container with id 791c41749a5afa46b4115cd059b475a475b84d00b546c238be99ab9691ede44d May 26 15:25:33.796559 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:33.796523 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7d47p" event={"ID":"ba53dd75-3f69-4a05-aad2-72d6859eeb38","Type":"ContainerStarted","Data":"791c41749a5afa46b4115cd059b475a475b84d00b546c238be99ab9691ede44d"} May 26 15:25:38.811130 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:38.811091 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7d47p" event={"ID":"ba53dd75-3f69-4a05-aad2-72d6859eeb38","Type":"ContainerStarted","Data":"ba4397ffbb03594eafa341bd583f7c09e1d2de2eceea5aac104bba52726ded8c"} May 26 15:25:38.825736 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:38.825696 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-7d47p" podStartSLOduration=0.923281812 podStartE2EDuration="5.825682758s" podCreationTimestamp="2026-05-26 15:25:33 +0000 UTC" firstStartedPulling="2026-05-26 15:25:33.490181127 +0000 UTC m=+282.216856039" lastFinishedPulling="2026-05-26 15:25:38.392582069 +0000 UTC m=+287.119256985" observedRunningTime="2026-05-26 15:25:38.823671447 +0000 UTC m=+287.550346369" watchObservedRunningTime="2026-05-26 15:25:38.825682758 +0000 UTC m=+287.552357680" May 26 15:25:49.838917 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:49.838882 2547 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-cd5fd7d5b-5rhrj" podUID="d764c454-9207-40d7-97b8-febbae6f60c4" containerName="console" containerID="cri-o://e85c4c28f5ee20388378c9a873eeff2268ccb40fecbfc150e2c7d18bae9160a6" gracePeriod=15 May 26 15:25:50.081776 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:50.081754 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-cd5fd7d5b-5rhrj_d764c454-9207-40d7-97b8-febbae6f60c4/console/0.log" May 26 15:25:50.081890 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:50.081811 2547 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cd5fd7d5b-5rhrj" May 26 15:25:50.169631 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:50.169534 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d764c454-9207-40d7-97b8-febbae6f60c4-service-ca\") pod \"d764c454-9207-40d7-97b8-febbae6f60c4\" (UID: \"d764c454-9207-40d7-97b8-febbae6f60c4\") " May 26 15:25:50.169631 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:50.169589 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7xk7\" (UniqueName: \"kubernetes.io/projected/d764c454-9207-40d7-97b8-febbae6f60c4-kube-api-access-p7xk7\") pod \"d764c454-9207-40d7-97b8-febbae6f60c4\" (UID: \"d764c454-9207-40d7-97b8-febbae6f60c4\") " May 26 15:25:50.169631 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:50.169615 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d764c454-9207-40d7-97b8-febbae6f60c4-console-oauth-config\") pod \"d764c454-9207-40d7-97b8-febbae6f60c4\" (UID: \"d764c454-9207-40d7-97b8-febbae6f60c4\") " May 26 15:25:50.169957 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:50.169640 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d764c454-9207-40d7-97b8-febbae6f60c4-console-config\") pod \"d764c454-9207-40d7-97b8-febbae6f60c4\" (UID: \"d764c454-9207-40d7-97b8-febbae6f60c4\") " May 26 15:25:50.169957 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:50.169690 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d764c454-9207-40d7-97b8-febbae6f60c4-oauth-serving-cert\") pod \"d764c454-9207-40d7-97b8-febbae6f60c4\" (UID: \"d764c454-9207-40d7-97b8-febbae6f60c4\") " May 26 15:25:50.169957 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:50.169719 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d764c454-9207-40d7-97b8-febbae6f60c4-console-serving-cert\") pod \"d764c454-9207-40d7-97b8-febbae6f60c4\" (UID: \"d764c454-9207-40d7-97b8-febbae6f60c4\") " May 26 15:25:50.169957 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:50.169797 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d764c454-9207-40d7-97b8-febbae6f60c4-trusted-ca-bundle\") pod \"d764c454-9207-40d7-97b8-febbae6f60c4\" (UID: \"d764c454-9207-40d7-97b8-febbae6f60c4\") " May 26 15:25:50.170197 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:50.170049 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d764c454-9207-40d7-97b8-febbae6f60c4-service-ca" (OuterVolumeSpecName: "service-ca") pod "d764c454-9207-40d7-97b8-febbae6f60c4" (UID: "d764c454-9207-40d7-97b8-febbae6f60c4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 26 15:25:50.170197 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:50.170104 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d764c454-9207-40d7-97b8-febbae6f60c4-console-config" (OuterVolumeSpecName: "console-config") pod "d764c454-9207-40d7-97b8-febbae6f60c4" (UID: "d764c454-9207-40d7-97b8-febbae6f60c4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 26 15:25:50.170425 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:50.170375 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d764c454-9207-40d7-97b8-febbae6f60c4-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d764c454-9207-40d7-97b8-febbae6f60c4" (UID: "d764c454-9207-40d7-97b8-febbae6f60c4"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 26 15:25:50.170506 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:50.170418 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d764c454-9207-40d7-97b8-febbae6f60c4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d764c454-9207-40d7-97b8-febbae6f60c4" (UID: "d764c454-9207-40d7-97b8-febbae6f60c4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 26 15:25:50.171895 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:50.171878 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d764c454-9207-40d7-97b8-febbae6f60c4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d764c454-9207-40d7-97b8-febbae6f60c4" (UID: "d764c454-9207-40d7-97b8-febbae6f60c4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 26 15:25:50.172302 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:50.172283 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d764c454-9207-40d7-97b8-febbae6f60c4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d764c454-9207-40d7-97b8-febbae6f60c4" (UID: "d764c454-9207-40d7-97b8-febbae6f60c4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 26 15:25:50.172372 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:50.172323 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d764c454-9207-40d7-97b8-febbae6f60c4-kube-api-access-p7xk7" (OuterVolumeSpecName: "kube-api-access-p7xk7") pod "d764c454-9207-40d7-97b8-febbae6f60c4" (UID: "d764c454-9207-40d7-97b8-febbae6f60c4"). InnerVolumeSpecName "kube-api-access-p7xk7". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 26 15:25:50.270595 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:50.270556 2547 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d764c454-9207-40d7-97b8-febbae6f60c4-oauth-serving-cert\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:25:50.270595 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:50.270587 2547 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d764c454-9207-40d7-97b8-febbae6f60c4-console-serving-cert\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:25:50.270595 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:50.270596 2547 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d764c454-9207-40d7-97b8-febbae6f60c4-trusted-ca-bundle\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:25:50.270816 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:50.270606 2547 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d764c454-9207-40d7-97b8-febbae6f60c4-service-ca\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:25:50.270816 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:50.270617 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7xk7\" (UniqueName: \"kubernetes.io/projected/d764c454-9207-40d7-97b8-febbae6f60c4-kube-api-access-p7xk7\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:25:50.270816 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:50.270626 2547 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d764c454-9207-40d7-97b8-febbae6f60c4-console-oauth-config\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:25:50.270816 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:50.270636 2547 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d764c454-9207-40d7-97b8-febbae6f60c4-console-config\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:25:50.852277 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:50.852251 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-cd5fd7d5b-5rhrj_d764c454-9207-40d7-97b8-febbae6f60c4/console/0.log" May 26 15:25:50.852739 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:50.852309 2547 generic.go:358] "Generic (PLEG): container finished" podID="d764c454-9207-40d7-97b8-febbae6f60c4" containerID="e85c4c28f5ee20388378c9a873eeff2268ccb40fecbfc150e2c7d18bae9160a6" exitCode=2 May 26 15:25:50.852739 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:50.852391 2547 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cd5fd7d5b-5rhrj" May 26 15:25:50.852739 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:50.852398 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cd5fd7d5b-5rhrj" event={"ID":"d764c454-9207-40d7-97b8-febbae6f60c4","Type":"ContainerDied","Data":"e85c4c28f5ee20388378c9a873eeff2268ccb40fecbfc150e2c7d18bae9160a6"} May 26 15:25:50.852739 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:50.852436 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cd5fd7d5b-5rhrj" event={"ID":"d764c454-9207-40d7-97b8-febbae6f60c4","Type":"ContainerDied","Data":"82a57c0bb3f846e31b30c44a5105f8675ea6ee992b33730ea2c49e2d30b8c4cb"} May 26 15:25:50.852739 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:50.852452 2547 scope.go:117] "RemoveContainer" containerID="e85c4c28f5ee20388378c9a873eeff2268ccb40fecbfc150e2c7d18bae9160a6" May 26 15:25:50.861644 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:50.861630 2547 scope.go:117] "RemoveContainer" containerID="e85c4c28f5ee20388378c9a873eeff2268ccb40fecbfc150e2c7d18bae9160a6" May 26 15:25:50.861895 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:25:50.861877 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e85c4c28f5ee20388378c9a873eeff2268ccb40fecbfc150e2c7d18bae9160a6\": container with ID starting with e85c4c28f5ee20388378c9a873eeff2268ccb40fecbfc150e2c7d18bae9160a6 not found: ID does not exist" containerID="e85c4c28f5ee20388378c9a873eeff2268ccb40fecbfc150e2c7d18bae9160a6" May 26 15:25:50.861947 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:50.861901 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e85c4c28f5ee20388378c9a873eeff2268ccb40fecbfc150e2c7d18bae9160a6"} err="failed to get container status \"e85c4c28f5ee20388378c9a873eeff2268ccb40fecbfc150e2c7d18bae9160a6\": rpc error: code = NotFound desc = could not find container \"e85c4c28f5ee20388378c9a873eeff2268ccb40fecbfc150e2c7d18bae9160a6\": container with ID starting with e85c4c28f5ee20388378c9a873eeff2268ccb40fecbfc150e2c7d18bae9160a6 not found: ID does not exist" May 26 15:25:50.872879 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:50.872857 2547 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cd5fd7d5b-5rhrj"] May 26 15:25:50.878613 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:50.878592 2547 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-cd5fd7d5b-5rhrj"] May 26 15:25:51.813945 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:51.813920 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-qd2tx_dd56d3c2-38ff-4bd8-9893-6b643455a976/console-operator/2.log" May 26 15:25:51.814145 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:51.814116 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-qd2tx_dd56d3c2-38ff-4bd8-9893-6b643455a976/console-operator/2.log" May 26 15:25:51.823679 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:51.823665 2547 kubelet.go:1628] "Image garbage collection succeeded" May 26 15:25:51.914326 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:25:51.914222 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d764c454-9207-40d7-97b8-febbae6f60c4" path="/var/lib/kubelet/pods/d764c454-9207-40d7-97b8-febbae6f60c4/volumes" May 26 15:27:04.027164 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:27:04.027133 2547 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-7864478f55-8ktb5"] May 26 15:27:04.027702 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:27:04.027448 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d764c454-9207-40d7-97b8-febbae6f60c4" containerName="console" May 26 15:27:04.027702 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:27:04.027460 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="d764c454-9207-40d7-97b8-febbae6f60c4" containerName="console" May 26 15:27:04.027702 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:27:04.027530 2547 memory_manager.go:356] "RemoveStaleState removing state" podUID="d764c454-9207-40d7-97b8-febbae6f60c4" containerName="console" May 26 15:27:04.029357 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:27:04.029341 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-7864478f55-8ktb5" May 26 15:27:04.031487 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:27:04.031460 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" May 26 15:27:04.031601 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:27:04.031486 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" May 26 15:27:04.031946 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:27:04.031928 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-krrl9\"" May 26 15:27:04.036198 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:27:04.036174 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-7864478f55-8ktb5"] May 26 15:27:04.119741 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:27:04.119716 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kklq\" (UniqueName: \"kubernetes.io/projected/2581a7a4-2b53-45c9-8baa-229be7205d4e-kube-api-access-8kklq\") pod \"feast-operator-controller-manager-7864478f55-8ktb5\" (UID: \"2581a7a4-2b53-45c9-8baa-229be7205d4e\") " pod="feast-operator-system/feast-operator-controller-manager-7864478f55-8ktb5" May 26 15:27:04.220982 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:27:04.220952 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8kklq\" (UniqueName: \"kubernetes.io/projected/2581a7a4-2b53-45c9-8baa-229be7205d4e-kube-api-access-8kklq\") pod \"feast-operator-controller-manager-7864478f55-8ktb5\" (UID: \"2581a7a4-2b53-45c9-8baa-229be7205d4e\") " pod="feast-operator-system/feast-operator-controller-manager-7864478f55-8ktb5" May 26 15:27:04.229274 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:27:04.229248 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kklq\" (UniqueName: \"kubernetes.io/projected/2581a7a4-2b53-45c9-8baa-229be7205d4e-kube-api-access-8kklq\") pod \"feast-operator-controller-manager-7864478f55-8ktb5\" (UID: \"2581a7a4-2b53-45c9-8baa-229be7205d4e\") " pod="feast-operator-system/feast-operator-controller-manager-7864478f55-8ktb5" May 26 15:27:04.340345 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:27:04.340271 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-7864478f55-8ktb5" May 26 15:27:04.459218 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:27:04.459186 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-7864478f55-8ktb5"] May 26 15:27:04.462464 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:27:04.462439 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2581a7a4_2b53_45c9_8baa_229be7205d4e.slice/crio-7581d6e7f9ed2fff7051ea3264cc66551ec50260c6c63c16e20965a9c14ac6cc WatchSource:0}: Error finding container 7581d6e7f9ed2fff7051ea3264cc66551ec50260c6c63c16e20965a9c14ac6cc: Status 404 returned error can't find the container with id 7581d6e7f9ed2fff7051ea3264cc66551ec50260c6c63c16e20965a9c14ac6cc May 26 15:27:04.464594 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:27:04.464578 2547 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 26 15:27:05.048449 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:27:05.048412 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-7864478f55-8ktb5" event={"ID":"2581a7a4-2b53-45c9-8baa-229be7205d4e","Type":"ContainerStarted","Data":"7581d6e7f9ed2fff7051ea3264cc66551ec50260c6c63c16e20965a9c14ac6cc"} May 26 15:27:07.057042 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:27:07.056949 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-7864478f55-8ktb5" event={"ID":"2581a7a4-2b53-45c9-8baa-229be7205d4e","Type":"ContainerStarted","Data":"db08243eb60c306c7650a9373f51de2d507320c3e0fc71b84f8d6d5e6d48a506"} May 26 15:27:07.057413 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:27:07.057213 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-7864478f55-8ktb5" May 26 15:27:07.071879 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:27:07.071826 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-7864478f55-8ktb5" podStartSLOduration=0.748872121 podStartE2EDuration="3.071791996s" podCreationTimestamp="2026-05-26 15:27:04 +0000 UTC" firstStartedPulling="2026-05-26 15:27:04.464699998 +0000 UTC m=+373.191374896" lastFinishedPulling="2026-05-26 15:27:06.787619859 +0000 UTC m=+375.514294771" observedRunningTime="2026-05-26 15:27:07.071656757 +0000 UTC m=+375.798331678" watchObservedRunningTime="2026-05-26 15:27:07.071791996 +0000 UTC m=+375.798466917" May 26 15:27:18.062533 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:27:18.062500 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-7864478f55-8ktb5" May 26 15:30:14.610564 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:14.610524 2547 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-784b5459d6-54mst"] May 26 15:30:14.613853 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:14.613831 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-784b5459d6-54mst" May 26 15:30:14.625423 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:14.625397 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-784b5459d6-54mst"] May 26 15:30:14.739608 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:14.739577 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4b31c58-7803-4692-9aa5-786b3b974e31-trusted-ca-bundle\") pod \"console-784b5459d6-54mst\" (UID: \"d4b31c58-7803-4692-9aa5-786b3b974e31\") " pod="openshift-console/console-784b5459d6-54mst" May 26 15:30:14.739761 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:14.739623 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4b31c58-7803-4692-9aa5-786b3b974e31-console-serving-cert\") pod \"console-784b5459d6-54mst\" (UID: \"d4b31c58-7803-4692-9aa5-786b3b974e31\") " pod="openshift-console/console-784b5459d6-54mst" May 26 15:30:14.739761 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:14.739654 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4b31c58-7803-4692-9aa5-786b3b974e31-service-ca\") pod \"console-784b5459d6-54mst\" (UID: \"d4b31c58-7803-4692-9aa5-786b3b974e31\") " pod="openshift-console/console-784b5459d6-54mst" May 26 15:30:14.739761 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:14.739672 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4b31c58-7803-4692-9aa5-786b3b974e31-console-oauth-config\") pod \"console-784b5459d6-54mst\" (UID: \"d4b31c58-7803-4692-9aa5-786b3b974e31\") " pod="openshift-console/console-784b5459d6-54mst" May 26 15:30:14.739761 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:14.739693 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4b31c58-7803-4692-9aa5-786b3b974e31-console-config\") pod \"console-784b5459d6-54mst\" (UID: \"d4b31c58-7803-4692-9aa5-786b3b974e31\") " pod="openshift-console/console-784b5459d6-54mst" May 26 15:30:14.739761 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:14.739727 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gx5pb\" (UniqueName: \"kubernetes.io/projected/d4b31c58-7803-4692-9aa5-786b3b974e31-kube-api-access-gx5pb\") pod \"console-784b5459d6-54mst\" (UID: \"d4b31c58-7803-4692-9aa5-786b3b974e31\") " pod="openshift-console/console-784b5459d6-54mst" May 26 15:30:14.739761 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:14.739753 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4b31c58-7803-4692-9aa5-786b3b974e31-oauth-serving-cert\") pod \"console-784b5459d6-54mst\" (UID: \"d4b31c58-7803-4692-9aa5-786b3b974e31\") " pod="openshift-console/console-784b5459d6-54mst" May 26 15:30:14.841087 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:14.841038 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4b31c58-7803-4692-9aa5-786b3b974e31-service-ca\") pod \"console-784b5459d6-54mst\" (UID: \"d4b31c58-7803-4692-9aa5-786b3b974e31\") " pod="openshift-console/console-784b5459d6-54mst" May 26 15:30:14.841087 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:14.841090 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4b31c58-7803-4692-9aa5-786b3b974e31-console-oauth-config\") pod \"console-784b5459d6-54mst\" (UID: \"d4b31c58-7803-4692-9aa5-786b3b974e31\") " pod="openshift-console/console-784b5459d6-54mst" May 26 15:30:14.841313 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:14.841122 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4b31c58-7803-4692-9aa5-786b3b974e31-console-config\") pod \"console-784b5459d6-54mst\" (UID: \"d4b31c58-7803-4692-9aa5-786b3b974e31\") " pod="openshift-console/console-784b5459d6-54mst" May 26 15:30:14.841313 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:14.841149 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gx5pb\" (UniqueName: \"kubernetes.io/projected/d4b31c58-7803-4692-9aa5-786b3b974e31-kube-api-access-gx5pb\") pod \"console-784b5459d6-54mst\" (UID: \"d4b31c58-7803-4692-9aa5-786b3b974e31\") " pod="openshift-console/console-784b5459d6-54mst" May 26 15:30:14.841313 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:14.841181 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4b31c58-7803-4692-9aa5-786b3b974e31-oauth-serving-cert\") pod \"console-784b5459d6-54mst\" (UID: \"d4b31c58-7803-4692-9aa5-786b3b974e31\") " pod="openshift-console/console-784b5459d6-54mst" May 26 15:30:14.841313 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:14.841230 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4b31c58-7803-4692-9aa5-786b3b974e31-trusted-ca-bundle\") pod \"console-784b5459d6-54mst\" (UID: \"d4b31c58-7803-4692-9aa5-786b3b974e31\") " pod="openshift-console/console-784b5459d6-54mst" May 26 15:30:14.841313 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:14.841275 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4b31c58-7803-4692-9aa5-786b3b974e31-console-serving-cert\") pod \"console-784b5459d6-54mst\" (UID: \"d4b31c58-7803-4692-9aa5-786b3b974e31\") " pod="openshift-console/console-784b5459d6-54mst" May 26 15:30:14.841833 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:14.841810 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4b31c58-7803-4692-9aa5-786b3b974e31-service-ca\") pod \"console-784b5459d6-54mst\" (UID: \"d4b31c58-7803-4692-9aa5-786b3b974e31\") " pod="openshift-console/console-784b5459d6-54mst" May 26 15:30:14.841959 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:14.841860 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4b31c58-7803-4692-9aa5-786b3b974e31-oauth-serving-cert\") pod \"console-784b5459d6-54mst\" (UID: \"d4b31c58-7803-4692-9aa5-786b3b974e31\") " pod="openshift-console/console-784b5459d6-54mst" May 26 15:30:14.841959 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:14.841910 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4b31c58-7803-4692-9aa5-786b3b974e31-console-config\") pod \"console-784b5459d6-54mst\" (UID: \"d4b31c58-7803-4692-9aa5-786b3b974e31\") " pod="openshift-console/console-784b5459d6-54mst" May 26 15:30:14.842575 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:14.842552 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4b31c58-7803-4692-9aa5-786b3b974e31-trusted-ca-bundle\") pod \"console-784b5459d6-54mst\" (UID: \"d4b31c58-7803-4692-9aa5-786b3b974e31\") " pod="openshift-console/console-784b5459d6-54mst" May 26 15:30:14.843758 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:14.843742 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4b31c58-7803-4692-9aa5-786b3b974e31-console-oauth-config\") pod \"console-784b5459d6-54mst\" (UID: \"d4b31c58-7803-4692-9aa5-786b3b974e31\") " pod="openshift-console/console-784b5459d6-54mst" May 26 15:30:14.843844 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:14.843827 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4b31c58-7803-4692-9aa5-786b3b974e31-console-serving-cert\") pod \"console-784b5459d6-54mst\" (UID: \"d4b31c58-7803-4692-9aa5-786b3b974e31\") " pod="openshift-console/console-784b5459d6-54mst" May 26 15:30:14.848737 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:14.848717 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gx5pb\" (UniqueName: \"kubernetes.io/projected/d4b31c58-7803-4692-9aa5-786b3b974e31-kube-api-access-gx5pb\") pod \"console-784b5459d6-54mst\" (UID: \"d4b31c58-7803-4692-9aa5-786b3b974e31\") " pod="openshift-console/console-784b5459d6-54mst" May 26 15:30:14.925533 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:14.925515 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-784b5459d6-54mst" May 26 15:30:15.051333 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:15.051209 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-784b5459d6-54mst"] May 26 15:30:15.054141 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:30:15.054113 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4b31c58_7803_4692_9aa5_786b3b974e31.slice/crio-ba325f8c4d2276eefa31d267d00ddf8ca46c1d8e2b113601763417317edf9fa8 WatchSource:0}: Error finding container ba325f8c4d2276eefa31d267d00ddf8ca46c1d8e2b113601763417317edf9fa8: Status 404 returned error can't find the container with id ba325f8c4d2276eefa31d267d00ddf8ca46c1d8e2b113601763417317edf9fa8 May 26 15:30:15.574464 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:15.574426 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-784b5459d6-54mst" event={"ID":"d4b31c58-7803-4692-9aa5-786b3b974e31","Type":"ContainerStarted","Data":"e0383b8bea9ba45752a4eaa7bf7df0b0621928c51e676cd4f8a8f85af5b538eb"} May 26 15:30:15.574464 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:15.574464 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-784b5459d6-54mst" event={"ID":"d4b31c58-7803-4692-9aa5-786b3b974e31","Type":"ContainerStarted","Data":"ba325f8c4d2276eefa31d267d00ddf8ca46c1d8e2b113601763417317edf9fa8"} May 26 15:30:15.591520 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:15.591467 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-784b5459d6-54mst" podStartSLOduration=1.591454377 podStartE2EDuration="1.591454377s" podCreationTimestamp="2026-05-26 15:30:14 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 15:30:15.589810682 +0000 UTC m=+564.316485603" watchObservedRunningTime="2026-05-26 15:30:15.591454377 +0000 UTC m=+564.318129300" May 26 15:30:24.926283 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:24.926248 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-784b5459d6-54mst" May 26 15:30:24.926283 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:24.926288 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-784b5459d6-54mst" May 26 15:30:24.930742 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:24.930722 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-784b5459d6-54mst" May 26 15:30:25.605575 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:25.605546 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-784b5459d6-54mst" May 26 15:30:25.656393 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:25.656359 2547 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65f77d89dd-qlvlt"] May 26 15:30:50.679747 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:50.679690 2547 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-65f77d89dd-qlvlt" podUID="98947d39-4497-4bcf-bb34-9c88b251a8bf" containerName="console" containerID="cri-o://62918e4ecf4108d5017c099fb39d29d24f3e21e85d11552f3d3cd035540962a6" gracePeriod=15 May 26 15:30:50.915360 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:50.915335 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-65f77d89dd-qlvlt_98947d39-4497-4bcf-bb34-9c88b251a8bf/console/0.log" May 26 15:30:50.915480 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:50.915396 2547 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65f77d89dd-qlvlt" May 26 15:30:51.014507 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:51.014448 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/98947d39-4497-4bcf-bb34-9c88b251a8bf-console-config\") pod \"98947d39-4497-4bcf-bb34-9c88b251a8bf\" (UID: \"98947d39-4497-4bcf-bb34-9c88b251a8bf\") " May 26 15:30:51.014607 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:51.014537 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/98947d39-4497-4bcf-bb34-9c88b251a8bf-console-oauth-config\") pod \"98947d39-4497-4bcf-bb34-9c88b251a8bf\" (UID: \"98947d39-4497-4bcf-bb34-9c88b251a8bf\") " May 26 15:30:51.014607 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:51.014558 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/98947d39-4497-4bcf-bb34-9c88b251a8bf-console-serving-cert\") pod \"98947d39-4497-4bcf-bb34-9c88b251a8bf\" (UID: \"98947d39-4497-4bcf-bb34-9c88b251a8bf\") " May 26 15:30:51.014607 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:51.014573 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/98947d39-4497-4bcf-bb34-9c88b251a8bf-service-ca\") pod \"98947d39-4497-4bcf-bb34-9c88b251a8bf\" (UID: \"98947d39-4497-4bcf-bb34-9c88b251a8bf\") " May 26 15:30:51.014607 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:51.014601 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98947d39-4497-4bcf-bb34-9c88b251a8bf-trusted-ca-bundle\") pod \"98947d39-4497-4bcf-bb34-9c88b251a8bf\" (UID: \"98947d39-4497-4bcf-bb34-9c88b251a8bf\") " May 26 15:30:51.014788 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:51.014624 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gbbt6\" (UniqueName: \"kubernetes.io/projected/98947d39-4497-4bcf-bb34-9c88b251a8bf-kube-api-access-gbbt6\") pod \"98947d39-4497-4bcf-bb34-9c88b251a8bf\" (UID: \"98947d39-4497-4bcf-bb34-9c88b251a8bf\") " May 26 15:30:51.014788 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:51.014659 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/98947d39-4497-4bcf-bb34-9c88b251a8bf-oauth-serving-cert\") pod \"98947d39-4497-4bcf-bb34-9c88b251a8bf\" (UID: \"98947d39-4497-4bcf-bb34-9c88b251a8bf\") " May 26 15:30:51.014879 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:51.014841 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98947d39-4497-4bcf-bb34-9c88b251a8bf-console-config" (OuterVolumeSpecName: "console-config") pod "98947d39-4497-4bcf-bb34-9c88b251a8bf" (UID: "98947d39-4497-4bcf-bb34-9c88b251a8bf"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 26 15:30:51.014972 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:51.014957 2547 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/98947d39-4497-4bcf-bb34-9c88b251a8bf-console-config\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:30:51.015025 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:51.014974 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98947d39-4497-4bcf-bb34-9c88b251a8bf-service-ca" (OuterVolumeSpecName: "service-ca") pod "98947d39-4497-4bcf-bb34-9c88b251a8bf" (UID: "98947d39-4497-4bcf-bb34-9c88b251a8bf"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 26 15:30:51.015379 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:51.015358 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98947d39-4497-4bcf-bb34-9c88b251a8bf-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "98947d39-4497-4bcf-bb34-9c88b251a8bf" (UID: "98947d39-4497-4bcf-bb34-9c88b251a8bf"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 26 15:30:51.015498 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:51.015475 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98947d39-4497-4bcf-bb34-9c88b251a8bf-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "98947d39-4497-4bcf-bb34-9c88b251a8bf" (UID: "98947d39-4497-4bcf-bb34-9c88b251a8bf"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 26 15:30:51.016789 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:51.016762 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98947d39-4497-4bcf-bb34-9c88b251a8bf-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "98947d39-4497-4bcf-bb34-9c88b251a8bf" (UID: "98947d39-4497-4bcf-bb34-9c88b251a8bf"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 26 15:30:51.016872 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:51.016800 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98947d39-4497-4bcf-bb34-9c88b251a8bf-kube-api-access-gbbt6" (OuterVolumeSpecName: "kube-api-access-gbbt6") pod "98947d39-4497-4bcf-bb34-9c88b251a8bf" (UID: "98947d39-4497-4bcf-bb34-9c88b251a8bf"). InnerVolumeSpecName "kube-api-access-gbbt6". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 26 15:30:51.016872 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:51.016809 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98947d39-4497-4bcf-bb34-9c88b251a8bf-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "98947d39-4497-4bcf-bb34-9c88b251a8bf" (UID: "98947d39-4497-4bcf-bb34-9c88b251a8bf"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 26 15:30:51.115554 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:51.115528 2547 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/98947d39-4497-4bcf-bb34-9c88b251a8bf-console-oauth-config\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:30:51.115554 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:51.115552 2547 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/98947d39-4497-4bcf-bb34-9c88b251a8bf-console-serving-cert\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:30:51.115677 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:51.115562 2547 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/98947d39-4497-4bcf-bb34-9c88b251a8bf-service-ca\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:30:51.115677 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:51.115572 2547 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98947d39-4497-4bcf-bb34-9c88b251a8bf-trusted-ca-bundle\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:30:51.115677 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:51.115581 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbbt6\" (UniqueName: \"kubernetes.io/projected/98947d39-4497-4bcf-bb34-9c88b251a8bf-kube-api-access-gbbt6\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:30:51.115677 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:51.115597 2547 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/98947d39-4497-4bcf-bb34-9c88b251a8bf-oauth-serving-cert\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:30:51.674649 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:51.674625 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-65f77d89dd-qlvlt_98947d39-4497-4bcf-bb34-9c88b251a8bf/console/0.log" May 26 15:30:51.674811 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:51.674664 2547 generic.go:358] "Generic (PLEG): container finished" podID="98947d39-4497-4bcf-bb34-9c88b251a8bf" containerID="62918e4ecf4108d5017c099fb39d29d24f3e21e85d11552f3d3cd035540962a6" exitCode=2 May 26 15:30:51.674811 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:51.674694 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65f77d89dd-qlvlt" event={"ID":"98947d39-4497-4bcf-bb34-9c88b251a8bf","Type":"ContainerDied","Data":"62918e4ecf4108d5017c099fb39d29d24f3e21e85d11552f3d3cd035540962a6"} May 26 15:30:51.674811 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:51.674738 2547 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65f77d89dd-qlvlt" May 26 15:30:51.674811 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:51.674752 2547 scope.go:117] "RemoveContainer" containerID="62918e4ecf4108d5017c099fb39d29d24f3e21e85d11552f3d3cd035540962a6" May 26 15:30:51.674989 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:51.674739 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65f77d89dd-qlvlt" event={"ID":"98947d39-4497-4bcf-bb34-9c88b251a8bf","Type":"ContainerDied","Data":"670d023fe641f1a17fc93480d695f1878068ede0ed55d64bfee26ba580e2d8f2"} May 26 15:30:51.684511 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:51.684191 2547 scope.go:117] "RemoveContainer" containerID="62918e4ecf4108d5017c099fb39d29d24f3e21e85d11552f3d3cd035540962a6" May 26 15:30:51.685030 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:30:51.685001 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62918e4ecf4108d5017c099fb39d29d24f3e21e85d11552f3d3cd035540962a6\": container with ID starting with 62918e4ecf4108d5017c099fb39d29d24f3e21e85d11552f3d3cd035540962a6 not found: ID does not exist" containerID="62918e4ecf4108d5017c099fb39d29d24f3e21e85d11552f3d3cd035540962a6" May 26 15:30:51.685198 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:51.685038 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62918e4ecf4108d5017c099fb39d29d24f3e21e85d11552f3d3cd035540962a6"} err="failed to get container status \"62918e4ecf4108d5017c099fb39d29d24f3e21e85d11552f3d3cd035540962a6\": rpc error: code = NotFound desc = could not find container \"62918e4ecf4108d5017c099fb39d29d24f3e21e85d11552f3d3cd035540962a6\": container with ID starting with 62918e4ecf4108d5017c099fb39d29d24f3e21e85d11552f3d3cd035540962a6 not found: ID does not exist" May 26 15:30:51.696154 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:51.696131 2547 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65f77d89dd-qlvlt"] May 26 15:30:51.701724 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:51.701706 2547 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-65f77d89dd-qlvlt"] May 26 15:30:51.833856 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:51.833829 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-qd2tx_dd56d3c2-38ff-4bd8-9893-6b643455a976/console-operator/2.log" May 26 15:30:51.835507 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:51.835486 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-qd2tx_dd56d3c2-38ff-4bd8-9893-6b643455a976/console-operator/2.log" May 26 15:30:51.913514 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:30:51.913487 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98947d39-4497-4bcf-bb34-9c88b251a8bf" path="/var/lib/kubelet/pods/98947d39-4497-4bcf-bb34-9c88b251a8bf/volumes" May 26 15:33:11.657984 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:33:11.657944 2547 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-w4np5"] May 26 15:33:11.658574 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:33:11.658409 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98947d39-4497-4bcf-bb34-9c88b251a8bf" containerName="console" May 26 15:33:11.658574 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:33:11.658430 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="98947d39-4497-4bcf-bb34-9c88b251a8bf" containerName="console" May 26 15:33:11.658574 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:33:11.658526 2547 memory_manager.go:356] "RemoveStaleState removing state" podUID="98947d39-4497-4bcf-bb34-9c88b251a8bf" containerName="console" May 26 15:33:11.661601 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:33:11.661578 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-w4np5" May 26 15:33:11.663746 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:33:11.663721 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" May 26 15:33:11.663879 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:33:11.663721 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"postgres-secret\"" May 26 15:33:11.663879 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:33:11.663762 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" May 26 15:33:11.664187 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:33:11.664171 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-4n88m\"" May 26 15:33:11.668421 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:33:11.668394 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-w4np5"] May 26 15:33:11.748728 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:33:11.748691 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frsr7\" (UniqueName: \"kubernetes.io/projected/456ca27c-e131-41ca-8d1a-37007a5c1b3d-kube-api-access-frsr7\") pod \"postgres-7c7656cc6d-w4np5\" (UID: \"456ca27c-e131-41ca-8d1a-37007a5c1b3d\") " pod="test-ns-feast/postgres-7c7656cc6d-w4np5" May 26 15:33:11.748879 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:33:11.748795 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/456ca27c-e131-41ca-8d1a-37007a5c1b3d-postgresdata\") pod \"postgres-7c7656cc6d-w4np5\" (UID: \"456ca27c-e131-41ca-8d1a-37007a5c1b3d\") " pod="test-ns-feast/postgres-7c7656cc6d-w4np5" May 26 15:33:11.849890 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:33:11.849862 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-frsr7\" (UniqueName: \"kubernetes.io/projected/456ca27c-e131-41ca-8d1a-37007a5c1b3d-kube-api-access-frsr7\") pod \"postgres-7c7656cc6d-w4np5\" (UID: \"456ca27c-e131-41ca-8d1a-37007a5c1b3d\") " pod="test-ns-feast/postgres-7c7656cc6d-w4np5" May 26 15:33:11.850037 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:33:11.849912 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/456ca27c-e131-41ca-8d1a-37007a5c1b3d-postgresdata\") pod \"postgres-7c7656cc6d-w4np5\" (UID: \"456ca27c-e131-41ca-8d1a-37007a5c1b3d\") " pod="test-ns-feast/postgres-7c7656cc6d-w4np5" May 26 15:33:11.850302 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:33:11.850285 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/456ca27c-e131-41ca-8d1a-37007a5c1b3d-postgresdata\") pod \"postgres-7c7656cc6d-w4np5\" (UID: \"456ca27c-e131-41ca-8d1a-37007a5c1b3d\") " pod="test-ns-feast/postgres-7c7656cc6d-w4np5" May 26 15:33:11.857804 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:33:11.857779 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frsr7\" (UniqueName: \"kubernetes.io/projected/456ca27c-e131-41ca-8d1a-37007a5c1b3d-kube-api-access-frsr7\") pod \"postgres-7c7656cc6d-w4np5\" (UID: \"456ca27c-e131-41ca-8d1a-37007a5c1b3d\") " pod="test-ns-feast/postgres-7c7656cc6d-w4np5" May 26 15:33:11.983220 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:33:11.983197 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-w4np5" May 26 15:33:12.103893 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:33:12.103867 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-w4np5"] May 26 15:33:12.106574 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:33:12.106549 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod456ca27c_e131_41ca_8d1a_37007a5c1b3d.slice/crio-0248151a3a12c1c2af6bb89290a443f7d137f7c9bcebb662b8317201a9fecffb WatchSource:0}: Error finding container 0248151a3a12c1c2af6bb89290a443f7d137f7c9bcebb662b8317201a9fecffb: Status 404 returned error can't find the container with id 0248151a3a12c1c2af6bb89290a443f7d137f7c9bcebb662b8317201a9fecffb May 26 15:33:12.108238 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:33:12.108222 2547 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 26 15:33:13.066488 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:33:13.066457 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-w4np5" event={"ID":"456ca27c-e131-41ca-8d1a-37007a5c1b3d","Type":"ContainerStarted","Data":"0248151a3a12c1c2af6bb89290a443f7d137f7c9bcebb662b8317201a9fecffb"} May 26 15:33:19.087321 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:33:19.087286 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-w4np5" event={"ID":"456ca27c-e131-41ca-8d1a-37007a5c1b3d","Type":"ContainerStarted","Data":"2dbb652f8d70447e60d6227f9c8be6b9c70990d4085a614c5de710c23b313e3d"} May 26 15:33:19.103651 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:33:19.103608 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/postgres-7c7656cc6d-w4np5" podStartSLOduration=2.123465964 podStartE2EDuration="8.103595713s" podCreationTimestamp="2026-05-26 15:33:11 +0000 UTC" firstStartedPulling="2026-05-26 15:33:12.108350588 +0000 UTC m=+740.835025486" lastFinishedPulling="2026-05-26 15:33:18.088480335 +0000 UTC m=+746.815155235" observedRunningTime="2026-05-26 15:33:19.10191294 +0000 UTC m=+747.828587861" watchObservedRunningTime="2026-05-26 15:33:19.103595713 +0000 UTC m=+747.830270644" May 26 15:34:48.007039 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:34:48.007003 2547 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-w4np5"] May 26 15:34:48.007508 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:34:48.007198 2547 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/postgres-7c7656cc6d-w4np5" podUID="456ca27c-e131-41ca-8d1a-37007a5c1b3d" containerName="postgres" containerID="cri-o://2dbb652f8d70447e60d6227f9c8be6b9c70990d4085a614c5de710c23b313e3d" gracePeriod=30 May 26 15:34:48.240660 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:34:48.240638 2547 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-w4np5" May 26 15:34:48.340714 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:34:48.340642 2547 generic.go:358] "Generic (PLEG): container finished" podID="456ca27c-e131-41ca-8d1a-37007a5c1b3d" containerID="2dbb652f8d70447e60d6227f9c8be6b9c70990d4085a614c5de710c23b313e3d" exitCode=0 May 26 15:34:48.340714 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:34:48.340695 2547 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-w4np5" May 26 15:34:48.340714 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:34:48.340698 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-w4np5" event={"ID":"456ca27c-e131-41ca-8d1a-37007a5c1b3d","Type":"ContainerDied","Data":"2dbb652f8d70447e60d6227f9c8be6b9c70990d4085a614c5de710c23b313e3d"} May 26 15:34:48.340909 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:34:48.340731 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-w4np5" event={"ID":"456ca27c-e131-41ca-8d1a-37007a5c1b3d","Type":"ContainerDied","Data":"0248151a3a12c1c2af6bb89290a443f7d137f7c9bcebb662b8317201a9fecffb"} May 26 15:34:48.340909 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:34:48.340751 2547 scope.go:117] "RemoveContainer" containerID="2dbb652f8d70447e60d6227f9c8be6b9c70990d4085a614c5de710c23b313e3d" May 26 15:34:48.348153 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:34:48.348133 2547 scope.go:117] "RemoveContainer" containerID="2dbb652f8d70447e60d6227f9c8be6b9c70990d4085a614c5de710c23b313e3d" May 26 15:34:48.348397 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:34:48.348377 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2dbb652f8d70447e60d6227f9c8be6b9c70990d4085a614c5de710c23b313e3d\": container with ID starting with 2dbb652f8d70447e60d6227f9c8be6b9c70990d4085a614c5de710c23b313e3d not found: ID does not exist" containerID="2dbb652f8d70447e60d6227f9c8be6b9c70990d4085a614c5de710c23b313e3d" May 26 15:34:48.348446 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:34:48.348407 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2dbb652f8d70447e60d6227f9c8be6b9c70990d4085a614c5de710c23b313e3d"} err="failed to get container status \"2dbb652f8d70447e60d6227f9c8be6b9c70990d4085a614c5de710c23b313e3d\": rpc error: code = NotFound desc = could not find container \"2dbb652f8d70447e60d6227f9c8be6b9c70990d4085a614c5de710c23b313e3d\": container with ID starting with 2dbb652f8d70447e60d6227f9c8be6b9c70990d4085a614c5de710c23b313e3d not found: ID does not exist" May 26 15:34:48.374705 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:34:48.374682 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frsr7\" (UniqueName: \"kubernetes.io/projected/456ca27c-e131-41ca-8d1a-37007a5c1b3d-kube-api-access-frsr7\") pod \"456ca27c-e131-41ca-8d1a-37007a5c1b3d\" (UID: \"456ca27c-e131-41ca-8d1a-37007a5c1b3d\") " May 26 15:34:48.374792 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:34:48.374723 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/456ca27c-e131-41ca-8d1a-37007a5c1b3d-postgresdata\") pod \"456ca27c-e131-41ca-8d1a-37007a5c1b3d\" (UID: \"456ca27c-e131-41ca-8d1a-37007a5c1b3d\") " May 26 15:34:48.376690 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:34:48.376663 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/456ca27c-e131-41ca-8d1a-37007a5c1b3d-kube-api-access-frsr7" (OuterVolumeSpecName: "kube-api-access-frsr7") pod "456ca27c-e131-41ca-8d1a-37007a5c1b3d" (UID: "456ca27c-e131-41ca-8d1a-37007a5c1b3d"). InnerVolumeSpecName "kube-api-access-frsr7". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 26 15:34:48.403993 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:34:48.403968 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/456ca27c-e131-41ca-8d1a-37007a5c1b3d-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "456ca27c-e131-41ca-8d1a-37007a5c1b3d" (UID: "456ca27c-e131-41ca-8d1a-37007a5c1b3d"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 26 15:34:48.475857 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:34:48.475832 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frsr7\" (UniqueName: \"kubernetes.io/projected/456ca27c-e131-41ca-8d1a-37007a5c1b3d-kube-api-access-frsr7\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:34:48.475857 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:34:48.475857 2547 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/456ca27c-e131-41ca-8d1a-37007a5c1b3d-postgresdata\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:34:48.660001 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:34:48.659977 2547 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-w4np5"] May 26 15:34:48.663725 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:34:48.663704 2547 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-w4np5"] May 26 15:34:49.912970 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:34:49.912938 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="456ca27c-e131-41ca-8d1a-37007a5c1b3d" path="/var/lib/kubelet/pods/456ca27c-e131-41ca-8d1a-37007a5c1b3d/volumes" May 26 15:35:51.852657 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:35:51.852631 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-qd2tx_dd56d3c2-38ff-4bd8-9893-6b643455a976/console-operator/2.log" May 26 15:35:51.854995 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:35:51.854973 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-qd2tx_dd56d3c2-38ff-4bd8-9893-6b643455a976/console-operator/2.log" May 26 15:36:48.731348 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:36:48.731310 2547 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/redis-854544c579-hs7kn"] May 26 15:36:48.731758 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:36:48.731740 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="456ca27c-e131-41ca-8d1a-37007a5c1b3d" containerName="postgres" May 26 15:36:48.731801 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:36:48.731762 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="456ca27c-e131-41ca-8d1a-37007a5c1b3d" containerName="postgres" May 26 15:36:48.731861 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:36:48.731849 2547 memory_manager.go:356] "RemoveStaleState removing state" podUID="456ca27c-e131-41ca-8d1a-37007a5c1b3d" containerName="postgres" May 26 15:36:48.734797 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:36:48.734779 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-hs7kn" May 26 15:36:48.736775 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:36:48.736757 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"openshift-service-ca.crt\"" May 26 15:36:48.737354 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:36:48.737332 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"default-dockercfg-psdjh\"" May 26 15:36:48.737354 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:36:48.737342 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"kube-root-ca.crt\"" May 26 15:36:48.750799 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:36:48.750775 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-854544c579-hs7kn"] May 26 15:36:48.859385 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:36:48.859355 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2dxn\" (UniqueName: \"kubernetes.io/projected/d30a51b6-d0b8-49bc-870c-c4b5fc7be485-kube-api-access-q2dxn\") pod \"redis-854544c579-hs7kn\" (UID: \"d30a51b6-d0b8-49bc-870c-c4b5fc7be485\") " pod="test-ns-feast-rest/redis-854544c579-hs7kn" May 26 15:36:48.960834 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:36:48.960785 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-q2dxn\" (UniqueName: \"kubernetes.io/projected/d30a51b6-d0b8-49bc-870c-c4b5fc7be485-kube-api-access-q2dxn\") pod \"redis-854544c579-hs7kn\" (UID: \"d30a51b6-d0b8-49bc-870c-c4b5fc7be485\") " pod="test-ns-feast-rest/redis-854544c579-hs7kn" May 26 15:36:48.968484 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:36:48.968449 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2dxn\" (UniqueName: \"kubernetes.io/projected/d30a51b6-d0b8-49bc-870c-c4b5fc7be485-kube-api-access-q2dxn\") pod \"redis-854544c579-hs7kn\" (UID: \"d30a51b6-d0b8-49bc-870c-c4b5fc7be485\") " pod="test-ns-feast-rest/redis-854544c579-hs7kn" May 26 15:36:49.059612 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:36:49.059533 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-hs7kn" May 26 15:36:49.187018 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:36:49.186993 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-854544c579-hs7kn"] May 26 15:36:49.189395 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:36:49.189357 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd30a51b6_d0b8_49bc_870c_c4b5fc7be485.slice/crio-f0379abc97740d660790e0119616f918ef8c2f599be2de2bba4e010aee2d098a WatchSource:0}: Error finding container f0379abc97740d660790e0119616f918ef8c2f599be2de2bba4e010aee2d098a: Status 404 returned error can't find the container with id f0379abc97740d660790e0119616f918ef8c2f599be2de2bba4e010aee2d098a May 26 15:36:49.677786 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:36:49.677749 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-hs7kn" event={"ID":"d30a51b6-d0b8-49bc-870c-c4b5fc7be485","Type":"ContainerStarted","Data":"f0379abc97740d660790e0119616f918ef8c2f599be2de2bba4e010aee2d098a"} May 26 15:36:51.683590 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:36:51.683553 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-hs7kn" event={"ID":"d30a51b6-d0b8-49bc-870c-c4b5fc7be485","Type":"ContainerStarted","Data":"6326d5b9e2a54a9802e0c082414d35709957a53b603510693230e35f93586d73"} May 26 15:36:51.698506 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:36:51.698461 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/redis-854544c579-hs7kn" podStartSLOduration=2.16168823 podStartE2EDuration="3.698448157s" podCreationTimestamp="2026-05-26 15:36:48 +0000 UTC" firstStartedPulling="2026-05-26 15:36:49.191041667 +0000 UTC m=+957.917716566" lastFinishedPulling="2026-05-26 15:36:50.727801591 +0000 UTC m=+959.454476493" observedRunningTime="2026-05-26 15:36:51.696776347 +0000 UTC m=+960.423451279" watchObservedRunningTime="2026-05-26 15:36:51.698448157 +0000 UTC m=+960.425123089" May 26 15:36:54.222008 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:36:54.221977 2547 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-pvl96"] May 26 15:36:54.225263 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:36:54.225245 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-pvl96" May 26 15:36:54.227265 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:36:54.227242 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"postgres-secret\"" May 26 15:36:54.233427 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:36:54.233400 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-pvl96"] May 26 15:36:54.305374 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:36:54.305344 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rz5gw\" (UniqueName: \"kubernetes.io/projected/0fb78bd2-626a-470f-9f2a-1c55e466e11e-kube-api-access-rz5gw\") pod \"postgres-7c7656cc6d-pvl96\" (UID: \"0fb78bd2-626a-470f-9f2a-1c55e466e11e\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-pvl96" May 26 15:36:54.305495 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:36:54.305394 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/0fb78bd2-626a-470f-9f2a-1c55e466e11e-postgresdata\") pod \"postgres-7c7656cc6d-pvl96\" (UID: \"0fb78bd2-626a-470f-9f2a-1c55e466e11e\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-pvl96" May 26 15:36:54.406447 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:36:54.406412 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rz5gw\" (UniqueName: \"kubernetes.io/projected/0fb78bd2-626a-470f-9f2a-1c55e466e11e-kube-api-access-rz5gw\") pod \"postgres-7c7656cc6d-pvl96\" (UID: \"0fb78bd2-626a-470f-9f2a-1c55e466e11e\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-pvl96" May 26 15:36:54.406590 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:36:54.406464 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/0fb78bd2-626a-470f-9f2a-1c55e466e11e-postgresdata\") pod \"postgres-7c7656cc6d-pvl96\" (UID: \"0fb78bd2-626a-470f-9f2a-1c55e466e11e\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-pvl96" May 26 15:36:54.406804 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:36:54.406789 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/0fb78bd2-626a-470f-9f2a-1c55e466e11e-postgresdata\") pod \"postgres-7c7656cc6d-pvl96\" (UID: \"0fb78bd2-626a-470f-9f2a-1c55e466e11e\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-pvl96" May 26 15:36:54.414720 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:36:54.414694 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rz5gw\" (UniqueName: \"kubernetes.io/projected/0fb78bd2-626a-470f-9f2a-1c55e466e11e-kube-api-access-rz5gw\") pod \"postgres-7c7656cc6d-pvl96\" (UID: \"0fb78bd2-626a-470f-9f2a-1c55e466e11e\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-pvl96" May 26 15:36:54.535848 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:36:54.535761 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-pvl96" May 26 15:36:54.659131 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:36:54.659106 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-pvl96"] May 26 15:36:54.660624 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:36:54.660590 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0fb78bd2_626a_470f_9f2a_1c55e466e11e.slice/crio-4daccb40af47af851e0d09ade3e991b06800ca3160e029f406c44b5adc8252a0 WatchSource:0}: Error finding container 4daccb40af47af851e0d09ade3e991b06800ca3160e029f406c44b5adc8252a0: Status 404 returned error can't find the container with id 4daccb40af47af851e0d09ade3e991b06800ca3160e029f406c44b5adc8252a0 May 26 15:36:54.692821 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:36:54.692790 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-pvl96" event={"ID":"0fb78bd2-626a-470f-9f2a-1c55e466e11e","Type":"ContainerStarted","Data":"4daccb40af47af851e0d09ade3e991b06800ca3160e029f406c44b5adc8252a0"} May 26 15:36:55.696993 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:36:55.696952 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-pvl96" event={"ID":"0fb78bd2-626a-470f-9f2a-1c55e466e11e","Type":"ContainerStarted","Data":"d72d624dbb7c29cbf095c291f39ff038123b7a7fff630e269a2924585d12e3c6"} May 26 15:36:55.712597 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:36:55.712537 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/postgres-7c7656cc6d-pvl96" podStartSLOduration=1.712523749 podStartE2EDuration="1.712523749s" podCreationTimestamp="2026-05-26 15:36:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 15:36:55.711757122 +0000 UTC m=+964.438432045" watchObservedRunningTime="2026-05-26 15:36:55.712523749 +0000 UTC m=+964.439198669" May 26 15:39:04.969659 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:04.969620 2547 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-7864478f55-8ktb5"] May 26 15:39:04.970115 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:04.969851 2547 kuberuntime_container.go:864] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-7864478f55-8ktb5" podUID="2581a7a4-2b53-45c9-8baa-229be7205d4e" containerName="manager" containerID="cri-o://db08243eb60c306c7650a9373f51de2d507320c3e0fc71b84f8d6d5e6d48a506" gracePeriod=10 May 26 15:39:05.253825 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:05.253803 2547 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-7864478f55-8ktb5" May 26 15:39:05.389365 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:05.389334 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8kklq\" (UniqueName: \"kubernetes.io/projected/2581a7a4-2b53-45c9-8baa-229be7205d4e-kube-api-access-8kklq\") pod \"2581a7a4-2b53-45c9-8baa-229be7205d4e\" (UID: \"2581a7a4-2b53-45c9-8baa-229be7205d4e\") " May 26 15:39:05.391384 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:05.391362 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2581a7a4-2b53-45c9-8baa-229be7205d4e-kube-api-access-8kklq" (OuterVolumeSpecName: "kube-api-access-8kklq") pod "2581a7a4-2b53-45c9-8baa-229be7205d4e" (UID: "2581a7a4-2b53-45c9-8baa-229be7205d4e"). InnerVolumeSpecName "kube-api-access-8kklq". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 26 15:39:05.490485 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:05.490410 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8kklq\" (UniqueName: \"kubernetes.io/projected/2581a7a4-2b53-45c9-8baa-229be7205d4e-kube-api-access-8kklq\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:39:06.071035 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:06.071000 2547 generic.go:358] "Generic (PLEG): container finished" podID="2581a7a4-2b53-45c9-8baa-229be7205d4e" containerID="db08243eb60c306c7650a9373f51de2d507320c3e0fc71b84f8d6d5e6d48a506" exitCode=0 May 26 15:39:06.071483 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:06.071079 2547 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-7864478f55-8ktb5" May 26 15:39:06.071483 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:06.071085 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-7864478f55-8ktb5" event={"ID":"2581a7a4-2b53-45c9-8baa-229be7205d4e","Type":"ContainerDied","Data":"db08243eb60c306c7650a9373f51de2d507320c3e0fc71b84f8d6d5e6d48a506"} May 26 15:39:06.071483 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:06.071133 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-7864478f55-8ktb5" event={"ID":"2581a7a4-2b53-45c9-8baa-229be7205d4e","Type":"ContainerDied","Data":"7581d6e7f9ed2fff7051ea3264cc66551ec50260c6c63c16e20965a9c14ac6cc"} May 26 15:39:06.071483 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:06.071155 2547 scope.go:117] "RemoveContainer" containerID="db08243eb60c306c7650a9373f51de2d507320c3e0fc71b84f8d6d5e6d48a506" May 26 15:39:06.079178 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:06.079163 2547 scope.go:117] "RemoveContainer" containerID="db08243eb60c306c7650a9373f51de2d507320c3e0fc71b84f8d6d5e6d48a506" May 26 15:39:06.079422 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:39:06.079404 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db08243eb60c306c7650a9373f51de2d507320c3e0fc71b84f8d6d5e6d48a506\": container with ID starting with db08243eb60c306c7650a9373f51de2d507320c3e0fc71b84f8d6d5e6d48a506 not found: ID does not exist" containerID="db08243eb60c306c7650a9373f51de2d507320c3e0fc71b84f8d6d5e6d48a506" May 26 15:39:06.079493 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:06.079433 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db08243eb60c306c7650a9373f51de2d507320c3e0fc71b84f8d6d5e6d48a506"} err="failed to get container status \"db08243eb60c306c7650a9373f51de2d507320c3e0fc71b84f8d6d5e6d48a506\": rpc error: code = NotFound desc = could not find container \"db08243eb60c306c7650a9373f51de2d507320c3e0fc71b84f8d6d5e6d48a506\": container with ID starting with db08243eb60c306c7650a9373f51de2d507320c3e0fc71b84f8d6d5e6d48a506 not found: ID does not exist" May 26 15:39:06.085660 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:06.085634 2547 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-7864478f55-8ktb5"] May 26 15:39:06.087383 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:06.087364 2547 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-7864478f55-8ktb5"] May 26 15:39:07.648707 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:07.648670 2547 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-pvl96"] May 26 15:39:07.649102 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:07.648871 2547 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/postgres-7c7656cc6d-pvl96" podUID="0fb78bd2-626a-470f-9f2a-1c55e466e11e" containerName="postgres" containerID="cri-o://d72d624dbb7c29cbf095c291f39ff038123b7a7fff630e269a2924585d12e3c6" gracePeriod=30 May 26 15:39:07.654269 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:07.654244 2547 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-854544c579-hs7kn"] May 26 15:39:07.654430 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:07.654410 2547 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/redis-854544c579-hs7kn" podUID="d30a51b6-d0b8-49bc-870c-c4b5fc7be485" containerName="redis" containerID="cri-o://6326d5b9e2a54a9802e0c082414d35709957a53b603510693230e35f93586d73" gracePeriod=30 May 26 15:39:07.913779 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:07.913753 2547 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-hs7kn" May 26 15:39:07.913872 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:07.913845 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2581a7a4-2b53-45c9-8baa-229be7205d4e" path="/var/lib/kubelet/pods/2581a7a4-2b53-45c9-8baa-229be7205d4e/volumes" May 26 15:39:07.917070 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:07.917040 2547 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-pvl96" May 26 15:39:08.078666 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:08.078635 2547 generic.go:358] "Generic (PLEG): container finished" podID="d30a51b6-d0b8-49bc-870c-c4b5fc7be485" containerID="6326d5b9e2a54a9802e0c082414d35709957a53b603510693230e35f93586d73" exitCode=0 May 26 15:39:08.078853 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:08.078677 2547 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-hs7kn" May 26 15:39:08.078853 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:08.078734 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-hs7kn" event={"ID":"d30a51b6-d0b8-49bc-870c-c4b5fc7be485","Type":"ContainerDied","Data":"6326d5b9e2a54a9802e0c082414d35709957a53b603510693230e35f93586d73"} May 26 15:39:08.078853 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:08.078766 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-hs7kn" event={"ID":"d30a51b6-d0b8-49bc-870c-c4b5fc7be485","Type":"ContainerDied","Data":"f0379abc97740d660790e0119616f918ef8c2f599be2de2bba4e010aee2d098a"} May 26 15:39:08.078853 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:08.078789 2547 scope.go:117] "RemoveContainer" containerID="6326d5b9e2a54a9802e0c082414d35709957a53b603510693230e35f93586d73" May 26 15:39:08.079719 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:08.079699 2547 generic.go:358] "Generic (PLEG): container finished" podID="0fb78bd2-626a-470f-9f2a-1c55e466e11e" containerID="d72d624dbb7c29cbf095c291f39ff038123b7a7fff630e269a2924585d12e3c6" exitCode=0 May 26 15:39:08.079776 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:08.079763 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-pvl96" event={"ID":"0fb78bd2-626a-470f-9f2a-1c55e466e11e","Type":"ContainerDied","Data":"d72d624dbb7c29cbf095c291f39ff038123b7a7fff630e269a2924585d12e3c6"} May 26 15:39:08.079811 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:08.079784 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-pvl96" event={"ID":"0fb78bd2-626a-470f-9f2a-1c55e466e11e","Type":"ContainerDied","Data":"4daccb40af47af851e0d09ade3e991b06800ca3160e029f406c44b5adc8252a0"} May 26 15:39:08.079811 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:08.079766 2547 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-pvl96" May 26 15:39:08.086153 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:08.086123 2547 scope.go:117] "RemoveContainer" containerID="6326d5b9e2a54a9802e0c082414d35709957a53b603510693230e35f93586d73" May 26 15:39:08.086394 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:39:08.086375 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6326d5b9e2a54a9802e0c082414d35709957a53b603510693230e35f93586d73\": container with ID starting with 6326d5b9e2a54a9802e0c082414d35709957a53b603510693230e35f93586d73 not found: ID does not exist" containerID="6326d5b9e2a54a9802e0c082414d35709957a53b603510693230e35f93586d73" May 26 15:39:08.086454 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:08.086401 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6326d5b9e2a54a9802e0c082414d35709957a53b603510693230e35f93586d73"} err="failed to get container status \"6326d5b9e2a54a9802e0c082414d35709957a53b603510693230e35f93586d73\": rpc error: code = NotFound desc = could not find container \"6326d5b9e2a54a9802e0c082414d35709957a53b603510693230e35f93586d73\": container with ID starting with 6326d5b9e2a54a9802e0c082414d35709957a53b603510693230e35f93586d73 not found: ID does not exist" May 26 15:39:08.086454 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:08.086416 2547 scope.go:117] "RemoveContainer" containerID="d72d624dbb7c29cbf095c291f39ff038123b7a7fff630e269a2924585d12e3c6" May 26 15:39:08.092963 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:08.092947 2547 scope.go:117] "RemoveContainer" containerID="d72d624dbb7c29cbf095c291f39ff038123b7a7fff630e269a2924585d12e3c6" May 26 15:39:08.093277 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:39:08.093257 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d72d624dbb7c29cbf095c291f39ff038123b7a7fff630e269a2924585d12e3c6\": container with ID starting with d72d624dbb7c29cbf095c291f39ff038123b7a7fff630e269a2924585d12e3c6 not found: ID does not exist" containerID="d72d624dbb7c29cbf095c291f39ff038123b7a7fff630e269a2924585d12e3c6" May 26 15:39:08.093341 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:08.093284 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d72d624dbb7c29cbf095c291f39ff038123b7a7fff630e269a2924585d12e3c6"} err="failed to get container status \"d72d624dbb7c29cbf095c291f39ff038123b7a7fff630e269a2924585d12e3c6\": rpc error: code = NotFound desc = could not find container \"d72d624dbb7c29cbf095c291f39ff038123b7a7fff630e269a2924585d12e3c6\": container with ID starting with d72d624dbb7c29cbf095c291f39ff038123b7a7fff630e269a2924585d12e3c6 not found: ID does not exist" May 26 15:39:08.112836 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:08.112817 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rz5gw\" (UniqueName: \"kubernetes.io/projected/0fb78bd2-626a-470f-9f2a-1c55e466e11e-kube-api-access-rz5gw\") pod \"0fb78bd2-626a-470f-9f2a-1c55e466e11e\" (UID: \"0fb78bd2-626a-470f-9f2a-1c55e466e11e\") " May 26 15:39:08.112941 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:08.112873 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q2dxn\" (UniqueName: \"kubernetes.io/projected/d30a51b6-d0b8-49bc-870c-c4b5fc7be485-kube-api-access-q2dxn\") pod \"d30a51b6-d0b8-49bc-870c-c4b5fc7be485\" (UID: \"d30a51b6-d0b8-49bc-870c-c4b5fc7be485\") " May 26 15:39:08.112941 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:08.112901 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/0fb78bd2-626a-470f-9f2a-1c55e466e11e-postgresdata\") pod \"0fb78bd2-626a-470f-9f2a-1c55e466e11e\" (UID: \"0fb78bd2-626a-470f-9f2a-1c55e466e11e\") " May 26 15:39:08.114868 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:08.114845 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fb78bd2-626a-470f-9f2a-1c55e466e11e-kube-api-access-rz5gw" (OuterVolumeSpecName: "kube-api-access-rz5gw") pod "0fb78bd2-626a-470f-9f2a-1c55e466e11e" (UID: "0fb78bd2-626a-470f-9f2a-1c55e466e11e"). InnerVolumeSpecName "kube-api-access-rz5gw". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 26 15:39:08.114933 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:08.114887 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d30a51b6-d0b8-49bc-870c-c4b5fc7be485-kube-api-access-q2dxn" (OuterVolumeSpecName: "kube-api-access-q2dxn") pod "d30a51b6-d0b8-49bc-870c-c4b5fc7be485" (UID: "d30a51b6-d0b8-49bc-870c-c4b5fc7be485"). InnerVolumeSpecName "kube-api-access-q2dxn". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 26 15:39:08.142615 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:08.142580 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fb78bd2-626a-470f-9f2a-1c55e466e11e-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "0fb78bd2-626a-470f-9f2a-1c55e466e11e" (UID: "0fb78bd2-626a-470f-9f2a-1c55e466e11e"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 26 15:39:08.214142 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:08.214113 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rz5gw\" (UniqueName: \"kubernetes.io/projected/0fb78bd2-626a-470f-9f2a-1c55e466e11e-kube-api-access-rz5gw\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:39:08.214278 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:08.214143 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2dxn\" (UniqueName: \"kubernetes.io/projected/d30a51b6-d0b8-49bc-870c-c4b5fc7be485-kube-api-access-q2dxn\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:39:08.214278 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:08.214163 2547 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/0fb78bd2-626a-470f-9f2a-1c55e466e11e-postgresdata\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:39:08.402271 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:08.402243 2547 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-854544c579-hs7kn"] May 26 15:39:08.406319 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:08.406297 2547 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/redis-854544c579-hs7kn"] May 26 15:39:08.415727 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:08.415707 2547 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-pvl96"] May 26 15:39:08.419500 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:08.419481 2547 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-pvl96"] May 26 15:39:09.912973 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:09.912939 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0fb78bd2-626a-470f-9f2a-1c55e466e11e" path="/var/lib/kubelet/pods/0fb78bd2-626a-470f-9f2a-1c55e466e11e/volumes" May 26 15:39:09.913420 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:09.913266 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d30a51b6-d0b8-49bc-870c-c4b5fc7be485" path="/var/lib/kubelet/pods/d30a51b6-d0b8-49bc-870c-c4b5fc7be485/volumes" May 26 15:39:16.351221 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:16.351184 2547 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-cf99db75-9cw24"] May 26 15:39:16.351564 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:16.351472 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d30a51b6-d0b8-49bc-870c-c4b5fc7be485" containerName="redis" May 26 15:39:16.351564 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:16.351484 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="d30a51b6-d0b8-49bc-870c-c4b5fc7be485" containerName="redis" May 26 15:39:16.351564 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:16.351497 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2581a7a4-2b53-45c9-8baa-229be7205d4e" containerName="manager" May 26 15:39:16.351564 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:16.351503 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="2581a7a4-2b53-45c9-8baa-229be7205d4e" containerName="manager" May 26 15:39:16.351564 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:16.351527 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0fb78bd2-626a-470f-9f2a-1c55e466e11e" containerName="postgres" May 26 15:39:16.351564 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:16.351533 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="0fb78bd2-626a-470f-9f2a-1c55e466e11e" containerName="postgres" May 26 15:39:16.351744 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:16.351575 2547 memory_manager.go:356] "RemoveStaleState removing state" podUID="d30a51b6-d0b8-49bc-870c-c4b5fc7be485" containerName="redis" May 26 15:39:16.351744 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:16.351585 2547 memory_manager.go:356] "RemoveStaleState removing state" podUID="0fb78bd2-626a-470f-9f2a-1c55e466e11e" containerName="postgres" May 26 15:39:16.351744 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:16.351592 2547 memory_manager.go:356] "RemoveStaleState removing state" podUID="2581a7a4-2b53-45c9-8baa-229be7205d4e" containerName="manager" May 26 15:39:16.354467 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:16.354451 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-9cw24" May 26 15:39:16.360994 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:16.360658 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" May 26 15:39:16.361403 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:16.361386 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" May 26 15:39:16.361630 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:16.361616 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-vqksw\"" May 26 15:39:16.366706 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:16.366683 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48x29\" (UniqueName: \"kubernetes.io/projected/620ffd1c-3511-4e5f-8c5f-19ec9f3aaaa8-kube-api-access-48x29\") pod \"feast-operator-controller-manager-cf99db75-9cw24\" (UID: \"620ffd1c-3511-4e5f-8c5f-19ec9f3aaaa8\") " pod="feast-operator-system/feast-operator-controller-manager-cf99db75-9cw24" May 26 15:39:16.390925 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:16.390898 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-cf99db75-9cw24"] May 26 15:39:16.467880 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:16.467855 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-48x29\" (UniqueName: \"kubernetes.io/projected/620ffd1c-3511-4e5f-8c5f-19ec9f3aaaa8-kube-api-access-48x29\") pod \"feast-operator-controller-manager-cf99db75-9cw24\" (UID: \"620ffd1c-3511-4e5f-8c5f-19ec9f3aaaa8\") " pod="feast-operator-system/feast-operator-controller-manager-cf99db75-9cw24" May 26 15:39:16.477217 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:16.477191 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-48x29\" (UniqueName: \"kubernetes.io/projected/620ffd1c-3511-4e5f-8c5f-19ec9f3aaaa8-kube-api-access-48x29\") pod \"feast-operator-controller-manager-cf99db75-9cw24\" (UID: \"620ffd1c-3511-4e5f-8c5f-19ec9f3aaaa8\") " pod="feast-operator-system/feast-operator-controller-manager-cf99db75-9cw24" May 26 15:39:16.664525 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:16.664466 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-9cw24" May 26 15:39:16.791257 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:16.791228 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-cf99db75-9cw24"] May 26 15:39:16.794634 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:39:16.794603 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod620ffd1c_3511_4e5f_8c5f_19ec9f3aaaa8.slice/crio-239a6a844b72a53c165fd62c3c18b65498d2942e758d5bd43326e480f6161257 WatchSource:0}: Error finding container 239a6a844b72a53c165fd62c3c18b65498d2942e758d5bd43326e480f6161257: Status 404 returned error can't find the container with id 239a6a844b72a53c165fd62c3c18b65498d2942e758d5bd43326e480f6161257 May 26 15:39:16.796330 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:16.796314 2547 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 26 15:39:17.109131 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:17.109097 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-9cw24" event={"ID":"620ffd1c-3511-4e5f-8c5f-19ec9f3aaaa8","Type":"ContainerStarted","Data":"239a6a844b72a53c165fd62c3c18b65498d2942e758d5bd43326e480f6161257"} May 26 15:39:19.117987 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:19.117942 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-9cw24" event={"ID":"620ffd1c-3511-4e5f-8c5f-19ec9f3aaaa8","Type":"ContainerStarted","Data":"05e1019496d2e5d6e975393b39112e74e1723892acc2cf19a76e078c34c885bb"} May 26 15:39:19.118578 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:19.118555 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-9cw24" May 26 15:39:19.134133 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:19.134078 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-9cw24" podStartSLOduration=0.949480442 podStartE2EDuration="3.134039974s" podCreationTimestamp="2026-05-26 15:39:16 +0000 UTC" firstStartedPulling="2026-05-26 15:39:16.796446507 +0000 UTC m=+1105.523121406" lastFinishedPulling="2026-05-26 15:39:18.981006033 +0000 UTC m=+1107.707680938" observedRunningTime="2026-05-26 15:39:19.132671641 +0000 UTC m=+1107.859346564" watchObservedRunningTime="2026-05-26 15:39:19.134039974 +0000 UTC m=+1107.860714895" May 26 15:39:30.124087 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:30.124038 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-9cw24" May 26 15:39:31.021075 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:31.021019 2547 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j"] May 26 15:39:31.025196 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:31.025173 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:39:31.027313 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:31.027287 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" May 26 15:39:31.027432 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:31.027371 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" May 26 15:39:31.027513 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:31.027300 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" May 26 15:39:31.028226 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:31.028009 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" May 26 15:39:31.029480 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:31.029457 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-29g6h\"" May 26 15:39:31.029969 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:31.029945 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" May 26 15:39:31.030524 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:31.030504 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" May 26 15:39:31.040147 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:31.035603 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j"] May 26 15:39:31.074914 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:31.074881 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xq5fn\" (UniqueName: \"kubernetes.io/projected/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-kube-api-access-xq5fn\") pod \"feast-simple-feast-setup-664fcfcc58-jft6j\" (UID: \"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:39:31.075100 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:31.074967 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-jft6j\" (UID: \"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:39:31.075100 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:31.075004 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-jft6j\" (UID: \"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:39:31.075100 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:31.075037 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-jft6j\" (UID: \"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:39:31.075100 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:31.075081 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-jft6j\" (UID: \"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:39:31.075271 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:31.075117 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-jft6j\" (UID: \"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:39:31.175976 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:31.175937 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xq5fn\" (UniqueName: \"kubernetes.io/projected/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-kube-api-access-xq5fn\") pod \"feast-simple-feast-setup-664fcfcc58-jft6j\" (UID: \"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:39:31.176461 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:31.176000 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-jft6j\" (UID: \"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:39:31.176461 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:31.176036 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-jft6j\" (UID: \"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:39:31.176461 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:31.176069 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-jft6j\" (UID: \"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:39:31.176461 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:31.176089 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-jft6j\" (UID: \"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:39:31.176461 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:31.176119 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-jft6j\" (UID: \"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:39:31.176461 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:39:31.176233 2547 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found May 26 15:39:31.176461 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:39:31.176295 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-online-tls podName:8a3ad2b2-91f0-46f9-90a5-e962b8163ee8 nodeName:}" failed. No retries permitted until 2026-05-26 15:39:31.676280028 +0000 UTC m=+1120.402954928 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-online-tls") pod "feast-simple-feast-setup-664fcfcc58-jft6j" (UID: "8a3ad2b2-91f0-46f9-90a5-e962b8163ee8") : secret "feast-simple-feast-setup-online-tls" not found May 26 15:39:31.176461 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:39:31.176373 2547 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found May 26 15:39:31.176461 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:39:31.176433 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-registry-tls podName:8a3ad2b2-91f0-46f9-90a5-e962b8163ee8 nodeName:}" failed. No retries permitted until 2026-05-26 15:39:31.676418535 +0000 UTC m=+1120.403093443 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-registry-tls") pod "feast-simple-feast-setup-664fcfcc58-jft6j" (UID: "8a3ad2b2-91f0-46f9-90a5-e962b8163ee8") : secret "feast-simple-feast-setup-registry-tls" not found May 26 15:39:31.176994 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:31.176464 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-jft6j\" (UID: \"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:39:31.176994 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:39:31.176501 2547 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found May 26 15:39:31.176994 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:39:31.176547 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-ui-tls podName:8a3ad2b2-91f0-46f9-90a5-e962b8163ee8 nodeName:}" failed. No retries permitted until 2026-05-26 15:39:31.676533275 +0000 UTC m=+1120.403208193 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-ui-tls") pod "feast-simple-feast-setup-664fcfcc58-jft6j" (UID: "8a3ad2b2-91f0-46f9-90a5-e962b8163ee8") : secret "feast-simple-feast-setup-ui-tls" not found May 26 15:39:31.179007 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:31.178984 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-jft6j\" (UID: \"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:39:31.184329 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:31.184306 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xq5fn\" (UniqueName: \"kubernetes.io/projected/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-kube-api-access-xq5fn\") pod \"feast-simple-feast-setup-664fcfcc58-jft6j\" (UID: \"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:39:31.680271 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:31.680241 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-jft6j\" (UID: \"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:39:31.680428 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:31.680278 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-jft6j\" (UID: \"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:39:31.680428 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:31.680314 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-jft6j\" (UID: \"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:39:31.682708 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:31.682680 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-jft6j\" (UID: \"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:39:31.682822 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:31.682733 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-jft6j\" (UID: \"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:39:31.682822 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:31.682769 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-jft6j\" (UID: \"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:39:31.951667 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:31.951573 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:39:32.090520 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:32.090495 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j"] May 26 15:39:32.092417 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:39:32.092393 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8a3ad2b2_91f0_46f9_90a5_e962b8163ee8.slice/crio-350508946f3a6566868d8eacf3412c8797d8806d2ec384cec362f5032d6d4452 WatchSource:0}: Error finding container 350508946f3a6566868d8eacf3412c8797d8806d2ec384cec362f5032d6d4452: Status 404 returned error can't find the container with id 350508946f3a6566868d8eacf3412c8797d8806d2ec384cec362f5032d6d4452 May 26 15:39:32.152786 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:32.152759 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" event={"ID":"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8","Type":"ContainerStarted","Data":"350508946f3a6566868d8eacf3412c8797d8806d2ec384cec362f5032d6d4452"} May 26 15:39:47.205519 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:47.205474 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" event={"ID":"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8","Type":"ContainerStarted","Data":"b3b84df9b1a852e35fddde24b0466e3cf69b8a8a15145d8bc1405b9d273c8ba6"} May 26 15:39:51.219153 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:51.219119 2547 generic.go:358] "Generic (PLEG): container finished" podID="8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" containerID="b3b84df9b1a852e35fddde24b0466e3cf69b8a8a15145d8bc1405b9d273c8ba6" exitCode=0 May 26 15:39:51.219521 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:51.219200 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" event={"ID":"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8","Type":"ContainerDied","Data":"b3b84df9b1a852e35fddde24b0466e3cf69b8a8a15145d8bc1405b9d273c8ba6"} May 26 15:39:52.223761 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:52.223721 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" event={"ID":"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8","Type":"ContainerStarted","Data":"0d1af9fda0a71e0e1f99a457f4efbaa8e64aa6e98deb936210d919a986bfd400"} May 26 15:39:57.240803 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:57.240768 2547 generic.go:358] "Generic (PLEG): container finished" podID="8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" containerID="0d1af9fda0a71e0e1f99a457f4efbaa8e64aa6e98deb936210d919a986bfd400" exitCode=0 May 26 15:39:57.241175 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:57.240848 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" event={"ID":"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8","Type":"ContainerDied","Data":"0d1af9fda0a71e0e1f99a457f4efbaa8e64aa6e98deb936210d919a986bfd400"} May 26 15:39:58.249047 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:58.249004 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" event={"ID":"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8","Type":"ContainerStarted","Data":"c976676a8587c60258cbad9efcd447dc686b36b2fde691b89073f8b0697b7bc2"} May 26 15:39:58.249504 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:58.249079 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" event={"ID":"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8","Type":"ContainerStarted","Data":"37b10e293c4486b7c7b174d837f180ee997c58c01c08e9b93536576a99cb37ac"} May 26 15:39:58.249504 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:58.249096 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" event={"ID":"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8","Type":"ContainerStarted","Data":"d734f44c36b9fccfcb010c3230856d79149b5527dba72db9b87c311d036470cf"} May 26 15:39:58.249504 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:58.249108 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" event={"ID":"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8","Type":"ContainerStarted","Data":"cc0ce75b3b879abaad310cbd9fcbc76c4e130b1177b8b838984878a8c6bdc940"} May 26 15:39:58.274964 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:58.274900 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" podStartSLOduration=12.929153403 podStartE2EDuration="27.274879793s" podCreationTimestamp="2026-05-26 15:39:31 +0000 UTC" firstStartedPulling="2026-05-26 15:39:32.094082603 +0000 UTC m=+1120.820757502" lastFinishedPulling="2026-05-26 15:39:46.439808993 +0000 UTC m=+1135.166483892" observedRunningTime="2026-05-26 15:39:58.271644791 +0000 UTC m=+1146.998319712" watchObservedRunningTime="2026-05-26 15:39:58.274879793 +0000 UTC m=+1147.001554715" May 26 15:39:58.952227 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:58.952187 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:39:58.952472 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:58.952458 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:39:58.952573 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:58.952563 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:39:58.952654 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:58.952645 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:39:58.954718 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:58.954669 2547 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" podUID="8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.31:8443: connect: connection refused" May 26 15:39:58.954718 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:58.954696 2547 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" podUID="8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.31:8816: connect: connection refused" May 26 15:39:58.954902 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:58.954713 2547 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" podUID="8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.31:6571: connect: connection refused" May 26 15:39:58.954902 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:39:58.954786 2547 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" podUID="8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" containerName="online" probeResult="failure" output="Get \"https://10.134.0.31:6567/health\": dial tcp 10.134.0.31:6567: connect: connection refused" May 26 15:40:01.952380 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:40:01.952335 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:40:01.952936 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:40:01.952908 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:40:01.952936 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:40:01.952939 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:40:01.953148 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:40:01.952616 2547 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" podUID="8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.31:6571: connect: connection refused" May 26 15:40:01.953148 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:40:01.952789 2547 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" podUID="8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.31:8816: connect: connection refused" May 26 15:40:01.953148 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:40:01.952953 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:40:01.953148 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:40:01.952671 2547 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" podUID="8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" containerName="online" probeResult="failure" output="Get \"https://10.134.0.31:6567/health\": dial tcp 10.134.0.31:6567: connect: connection refused" May 26 15:40:01.953148 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:40:01.952624 2547 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" podUID="8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.31:8443: connect: connection refused" May 26 15:40:04.952367 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:40:04.952327 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:40:04.952746 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:40:04.952509 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:40:04.952746 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:40:04.952741 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:40:04.953329 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:40:04.953307 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:40:04.953416 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:40:04.953360 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:40:04.953623 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:40:04.953603 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:40:04.957691 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:40:04.957669 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:40:05.275666 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:40:05.275586 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:40:51.872691 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:40:51.872658 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-qd2tx_dd56d3c2-38ff-4bd8-9893-6b643455a976/console-operator/2.log" May 26 15:40:51.875550 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:40:51.875527 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-qd2tx_dd56d3c2-38ff-4bd8-9893-6b643455a976/console-operator/2.log" May 26 15:41:08.239480 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:08.239446 2547 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j"] May 26 15:41:08.239946 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:08.239813 2547 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" podUID="8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" containerName="registry" containerID="cri-o://cc0ce75b3b879abaad310cbd9fcbc76c4e130b1177b8b838984878a8c6bdc940" gracePeriod=30 May 26 15:41:08.240472 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:08.240028 2547 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" podUID="8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" containerName="ui" containerID="cri-o://c976676a8587c60258cbad9efcd447dc686b36b2fde691b89073f8b0697b7bc2" gracePeriod=30 May 26 15:41:08.240472 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:08.240097 2547 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" podUID="8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" containerName="online" containerID="cri-o://d734f44c36b9fccfcb010c3230856d79149b5527dba72db9b87c311d036470cf" gracePeriod=30 May 26 15:41:08.240472 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:08.240205 2547 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" podUID="8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" containerName="offline" containerID="cri-o://37b10e293c4486b7c7b174d837f180ee997c58c01c08e9b93536576a99cb37ac" gracePeriod=30 May 26 15:41:08.624734 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:08.624647 2547 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5"] May 26 15:41:08.629484 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:08.629459 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" May 26 15:41:08.631391 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:08.631361 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-n22x4\"" May 26 15:41:08.638927 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:08.638908 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5"] May 26 15:41:08.752789 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:08.752740 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7548eb6f-39bf-4632-9bd9-8e0780a40550-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-r2gm5\" (UID: \"7548eb6f-39bf-4632-9bd9-8e0780a40550\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" May 26 15:41:08.752955 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:08.752846 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7548eb6f-39bf-4632-9bd9-8e0780a40550-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-r2gm5\" (UID: \"7548eb6f-39bf-4632-9bd9-8e0780a40550\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" May 26 15:41:08.753015 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:08.752946 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7548eb6f-39bf-4632-9bd9-8e0780a40550-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-r2gm5\" (UID: \"7548eb6f-39bf-4632-9bd9-8e0780a40550\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" May 26 15:41:08.753015 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:08.752997 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7548eb6f-39bf-4632-9bd9-8e0780a40550-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-r2gm5\" (UID: \"7548eb6f-39bf-4632-9bd9-8e0780a40550\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" May 26 15:41:08.753132 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:08.753025 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tc9qp\" (UniqueName: \"kubernetes.io/projected/7548eb6f-39bf-4632-9bd9-8e0780a40550-kube-api-access-tc9qp\") pod \"feast-simple-feast-setup-664fcfcc58-r2gm5\" (UID: \"7548eb6f-39bf-4632-9bd9-8e0780a40550\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" May 26 15:41:08.753132 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:08.753073 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7548eb6f-39bf-4632-9bd9-8e0780a40550-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-r2gm5\" (UID: \"7548eb6f-39bf-4632-9bd9-8e0780a40550\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" May 26 15:41:08.853537 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:08.853499 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7548eb6f-39bf-4632-9bd9-8e0780a40550-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-r2gm5\" (UID: \"7548eb6f-39bf-4632-9bd9-8e0780a40550\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" May 26 15:41:08.853718 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:08.853548 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7548eb6f-39bf-4632-9bd9-8e0780a40550-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-r2gm5\" (UID: \"7548eb6f-39bf-4632-9bd9-8e0780a40550\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" May 26 15:41:08.853718 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:41:08.853661 2547 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found May 26 15:41:08.853718 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:08.853677 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7548eb6f-39bf-4632-9bd9-8e0780a40550-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-r2gm5\" (UID: \"7548eb6f-39bf-4632-9bd9-8e0780a40550\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" May 26 15:41:08.853718 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:41:08.853684 2547 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found May 26 15:41:08.854168 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:41:08.853725 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7548eb6f-39bf-4632-9bd9-8e0780a40550-offline-tls podName:7548eb6f-39bf-4632-9bd9-8e0780a40550 nodeName:}" failed. No retries permitted until 2026-05-26 15:41:09.353703612 +0000 UTC m=+1218.080378535 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/7548eb6f-39bf-4632-9bd9-8e0780a40550-offline-tls") pod "feast-simple-feast-setup-664fcfcc58-r2gm5" (UID: "7548eb6f-39bf-4632-9bd9-8e0780a40550") : secret "feast-simple-feast-setup-offline-tls" not found May 26 15:41:08.854168 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:08.853744 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7548eb6f-39bf-4632-9bd9-8e0780a40550-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-r2gm5\" (UID: \"7548eb6f-39bf-4632-9bd9-8e0780a40550\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" May 26 15:41:08.854168 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:41:08.853772 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7548eb6f-39bf-4632-9bd9-8e0780a40550-ui-tls podName:7548eb6f-39bf-4632-9bd9-8e0780a40550 nodeName:}" failed. No retries permitted until 2026-05-26 15:41:09.353752307 +0000 UTC m=+1218.080427232 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/7548eb6f-39bf-4632-9bd9-8e0780a40550-ui-tls") pod "feast-simple-feast-setup-664fcfcc58-r2gm5" (UID: "7548eb6f-39bf-4632-9bd9-8e0780a40550") : secret "feast-simple-feast-setup-ui-tls" not found May 26 15:41:08.854168 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:08.853817 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tc9qp\" (UniqueName: \"kubernetes.io/projected/7548eb6f-39bf-4632-9bd9-8e0780a40550-kube-api-access-tc9qp\") pod \"feast-simple-feast-setup-664fcfcc58-r2gm5\" (UID: \"7548eb6f-39bf-4632-9bd9-8e0780a40550\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" May 26 15:41:08.854168 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:08.853856 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7548eb6f-39bf-4632-9bd9-8e0780a40550-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-r2gm5\" (UID: \"7548eb6f-39bf-4632-9bd9-8e0780a40550\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" May 26 15:41:08.854453 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:08.854206 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7548eb6f-39bf-4632-9bd9-8e0780a40550-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-r2gm5\" (UID: \"7548eb6f-39bf-4632-9bd9-8e0780a40550\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" May 26 15:41:08.856896 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:08.856872 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7548eb6f-39bf-4632-9bd9-8e0780a40550-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-r2gm5\" (UID: \"7548eb6f-39bf-4632-9bd9-8e0780a40550\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" May 26 15:41:08.857001 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:08.856879 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7548eb6f-39bf-4632-9bd9-8e0780a40550-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-r2gm5\" (UID: \"7548eb6f-39bf-4632-9bd9-8e0780a40550\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" May 26 15:41:08.861801 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:08.861777 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tc9qp\" (UniqueName: \"kubernetes.io/projected/7548eb6f-39bf-4632-9bd9-8e0780a40550-kube-api-access-tc9qp\") pod \"feast-simple-feast-setup-664fcfcc58-r2gm5\" (UID: \"7548eb6f-39bf-4632-9bd9-8e0780a40550\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" May 26 15:41:09.357525 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:09.357451 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7548eb6f-39bf-4632-9bd9-8e0780a40550-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-r2gm5\" (UID: \"7548eb6f-39bf-4632-9bd9-8e0780a40550\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" May 26 15:41:09.357856 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:09.357537 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7548eb6f-39bf-4632-9bd9-8e0780a40550-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-r2gm5\" (UID: \"7548eb6f-39bf-4632-9bd9-8e0780a40550\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" May 26 15:41:09.359872 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:09.359849 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7548eb6f-39bf-4632-9bd9-8e0780a40550-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-r2gm5\" (UID: \"7548eb6f-39bf-4632-9bd9-8e0780a40550\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" May 26 15:41:09.359981 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:09.359936 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7548eb6f-39bf-4632-9bd9-8e0780a40550-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-r2gm5\" (UID: \"7548eb6f-39bf-4632-9bd9-8e0780a40550\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" May 26 15:41:09.460133 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:09.460103 2547 generic.go:358] "Generic (PLEG): container finished" podID="8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" containerID="c976676a8587c60258cbad9efcd447dc686b36b2fde691b89073f8b0697b7bc2" exitCode=0 May 26 15:41:09.460133 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:09.460128 2547 generic.go:358] "Generic (PLEG): container finished" podID="8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" containerID="d734f44c36b9fccfcb010c3230856d79149b5527dba72db9b87c311d036470cf" exitCode=0 May 26 15:41:09.460289 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:09.460175 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" event={"ID":"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8","Type":"ContainerDied","Data":"c976676a8587c60258cbad9efcd447dc686b36b2fde691b89073f8b0697b7bc2"} May 26 15:41:09.460289 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:09.460209 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" event={"ID":"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8","Type":"ContainerDied","Data":"d734f44c36b9fccfcb010c3230856d79149b5527dba72db9b87c311d036470cf"} May 26 15:41:09.541000 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:09.540974 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" May 26 15:41:09.666774 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:09.666742 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5"] May 26 15:41:09.669795 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:41:09.669771 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7548eb6f_39bf_4632_9bd9_8e0780a40550.slice/crio-4bbc834aa2cbaa29e7f4c2e94a0b5a7b4735d32d4cbcce51fb8c49bf0cede95b WatchSource:0}: Error finding container 4bbc834aa2cbaa29e7f4c2e94a0b5a7b4735d32d4cbcce51fb8c49bf0cede95b: Status 404 returned error can't find the container with id 4bbc834aa2cbaa29e7f4c2e94a0b5a7b4735d32d4cbcce51fb8c49bf0cede95b May 26 15:41:10.465040 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:10.464998 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" event={"ID":"7548eb6f-39bf-4632-9bd9-8e0780a40550","Type":"ContainerStarted","Data":"0e4daa7f6ce8f09a66d07af7503d0a8e5409b09fe80cf1d137c82879404e9740"} May 26 15:41:10.465040 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:10.465045 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" event={"ID":"7548eb6f-39bf-4632-9bd9-8e0780a40550","Type":"ContainerStarted","Data":"4bbc834aa2cbaa29e7f4c2e94a0b5a7b4735d32d4cbcce51fb8c49bf0cede95b"} May 26 15:41:14.477390 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:14.477307 2547 generic.go:358] "Generic (PLEG): container finished" podID="7548eb6f-39bf-4632-9bd9-8e0780a40550" containerID="0e4daa7f6ce8f09a66d07af7503d0a8e5409b09fe80cf1d137c82879404e9740" exitCode=0 May 26 15:41:14.477802 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:14.477385 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" event={"ID":"7548eb6f-39bf-4632-9bd9-8e0780a40550","Type":"ContainerDied","Data":"0e4daa7f6ce8f09a66d07af7503d0a8e5409b09fe80cf1d137c82879404e9740"} May 26 15:41:14.953504 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:14.953441 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" podUID="8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.31:8443: connect: connection refused" May 26 15:41:15.273412 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:15.273325 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" podUID="8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" containerName="online" probeResult="failure" output="Get \"https://10.134.0.31:6567/health\": dial tcp 10.134.0.31:6567: connect: connection refused" May 26 15:41:15.483290 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:15.483257 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" event={"ID":"7548eb6f-39bf-4632-9bd9-8e0780a40550","Type":"ContainerStarted","Data":"a9861e6366b29d47d55036c555b673856746cfc1291a2c5dacca8b22f98c6b37"} May 26 15:41:20.500347 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:20.500314 2547 generic.go:358] "Generic (PLEG): container finished" podID="7548eb6f-39bf-4632-9bd9-8e0780a40550" containerID="a9861e6366b29d47d55036c555b673856746cfc1291a2c5dacca8b22f98c6b37" exitCode=0 May 26 15:41:20.500719 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:20.500399 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" event={"ID":"7548eb6f-39bf-4632-9bd9-8e0780a40550","Type":"ContainerDied","Data":"a9861e6366b29d47d55036c555b673856746cfc1291a2c5dacca8b22f98c6b37"} May 26 15:41:21.508840 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:21.508797 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" event={"ID":"7548eb6f-39bf-4632-9bd9-8e0780a40550","Type":"ContainerStarted","Data":"48de061a279017064ec4474ba7ebcade035496a2d19fde74ff969612f3a9332f"} May 26 15:41:21.509332 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:21.508855 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" event={"ID":"7548eb6f-39bf-4632-9bd9-8e0780a40550","Type":"ContainerStarted","Data":"985d9dc0497ab53ca1c88f93280cf218790ffe688f02d2f6b0148401e6a8ec03"} May 26 15:41:21.509332 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:21.508874 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" event={"ID":"7548eb6f-39bf-4632-9bd9-8e0780a40550","Type":"ContainerStarted","Data":"a45507e726e6f853a163d180b304f73a03d566351b72dfa2a7e43a9fd57aa35e"} May 26 15:41:21.509332 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:21.508887 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" event={"ID":"7548eb6f-39bf-4632-9bd9-8e0780a40550","Type":"ContainerStarted","Data":"f138312b178c4064fcd07588b119dc4daa960f2ab45dbbb8433d32804fe6e406"} May 26 15:41:21.532544 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:21.532486 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" podStartSLOduration=13.532467487 podStartE2EDuration="13.532467487s" podCreationTimestamp="2026-05-26 15:41:08 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 15:41:21.530700837 +0000 UTC m=+1230.257375759" watchObservedRunningTime="2026-05-26 15:41:21.532467487 +0000 UTC m=+1230.259142409" May 26 15:41:21.541575 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:21.541531 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" May 26 15:41:21.541575 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:21.541576 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" May 26 15:41:21.541795 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:21.541591 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" May 26 15:41:21.541795 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:21.541604 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" May 26 15:41:21.543958 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:21.543919 2547 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" podUID="7548eb6f-39bf-4632-9bd9-8e0780a40550" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.32:6571: connect: connection refused" May 26 15:41:21.544196 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:21.543941 2547 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" podUID="7548eb6f-39bf-4632-9bd9-8e0780a40550" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.32:8443: connect: connection refused" May 26 15:41:21.544274 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:21.543978 2547 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" podUID="7548eb6f-39bf-4632-9bd9-8e0780a40550" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.32:8816: connect: connection refused" May 26 15:41:21.544274 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:21.543979 2547 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" podUID="7548eb6f-39bf-4632-9bd9-8e0780a40550" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused" May 26 15:41:24.541505 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:24.541460 2547 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" podUID="7548eb6f-39bf-4632-9bd9-8e0780a40550" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.32:8816: connect: connection refused" May 26 15:41:24.542184 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:24.541460 2547 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" podUID="7548eb6f-39bf-4632-9bd9-8e0780a40550" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.32:6571: connect: connection refused" May 26 15:41:24.542295 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:24.541520 2547 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" podUID="7548eb6f-39bf-4632-9bd9-8e0780a40550" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.32:8443: connect: connection refused" May 26 15:41:24.542295 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:24.541540 2547 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" podUID="7548eb6f-39bf-4632-9bd9-8e0780a40550" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused" May 26 15:41:24.953876 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:24.953820 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" podUID="8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.31:8443: connect: connection refused" May 26 15:41:25.273419 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:25.273308 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" podUID="8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" containerName="online" probeResult="failure" output="Get \"https://10.134.0.31:6567/health\": dial tcp 10.134.0.31:6567: connect: connection refused" May 26 15:41:27.542822 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:27.542787 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" May 26 15:41:27.543215 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:27.543003 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" May 26 15:41:27.543286 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:27.543269 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" May 26 15:41:27.543871 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:27.543847 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" May 26 15:41:27.543975 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:27.543876 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" May 26 15:41:27.543975 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:27.543885 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" May 26 15:41:27.543975 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:27.543928 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" May 26 15:41:27.544121 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:27.544047 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" May 26 15:41:27.544121 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:27.544106 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" May 26 15:41:27.547585 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:27.547569 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" May 26 15:41:28.532974 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:28.532940 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" May 26 15:41:28.535939 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:28.535915 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" May 26 15:41:34.953413 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:34.953363 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" podUID="8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.31:8443: connect: connection refused" May 26 15:41:34.953921 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:34.953524 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:41:35.273515 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:35.273421 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" podUID="8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" containerName="online" probeResult="failure" output="Get \"https://10.134.0.31:6567/health\": dial tcp 10.134.0.31:6567: connect: connection refused" May 26 15:41:35.273660 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:35.273549 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:41:38.562481 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:38.562453 2547 generic.go:358] "Generic (PLEG): container finished" podID="8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" containerID="37b10e293c4486b7c7b174d837f180ee997c58c01c08e9b93536576a99cb37ac" exitCode=137 May 26 15:41:38.562481 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:38.562479 2547 generic.go:358] "Generic (PLEG): container finished" podID="8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" containerID="cc0ce75b3b879abaad310cbd9fcbc76c4e130b1177b8b838984878a8c6bdc940" exitCode=137 May 26 15:41:38.562881 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:38.562529 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" event={"ID":"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8","Type":"ContainerDied","Data":"37b10e293c4486b7c7b174d837f180ee997c58c01c08e9b93536576a99cb37ac"} May 26 15:41:38.562881 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:38.562567 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" event={"ID":"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8","Type":"ContainerDied","Data":"cc0ce75b3b879abaad310cbd9fcbc76c4e130b1177b8b838984878a8c6bdc940"} May 26 15:41:38.884222 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:38.884197 2547 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:41:38.918981 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:38.918951 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-offline-tls\") pod \"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8\" (UID: \"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8\") " May 26 15:41:38.919193 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:38.919000 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-registry-tls\") pod \"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8\" (UID: \"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8\") " May 26 15:41:38.919193 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:38.919039 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-ui-tls\") pod \"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8\" (UID: \"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8\") " May 26 15:41:38.919305 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:38.919190 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-feast-data\") pod \"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8\" (UID: \"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8\") " May 26 15:41:38.919357 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:38.919306 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xq5fn\" (UniqueName: \"kubernetes.io/projected/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-kube-api-access-xq5fn\") pod \"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8\" (UID: \"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8\") " May 26 15:41:38.919357 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:38.919342 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-online-tls\") pod \"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8\" (UID: \"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8\") " May 26 15:41:38.919706 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:38.919678 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-feast-data" (OuterVolumeSpecName: "feast-data") pod "8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" (UID: "8a3ad2b2-91f0-46f9-90a5-e962b8163ee8"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 26 15:41:38.921454 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:38.921391 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" (UID: "8a3ad2b2-91f0-46f9-90a5-e962b8163ee8"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 26 15:41:38.921615 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:38.921590 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" (UID: "8a3ad2b2-91f0-46f9-90a5-e962b8163ee8"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 26 15:41:38.921681 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:38.921590 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-kube-api-access-xq5fn" (OuterVolumeSpecName: "kube-api-access-xq5fn") pod "8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" (UID: "8a3ad2b2-91f0-46f9-90a5-e962b8163ee8"). InnerVolumeSpecName "kube-api-access-xq5fn". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 26 15:41:38.921755 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:38.921736 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-online-tls" (OuterVolumeSpecName: "online-tls") pod "8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" (UID: "8a3ad2b2-91f0-46f9-90a5-e962b8163ee8"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 26 15:41:38.921860 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:38.921841 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" (UID: "8a3ad2b2-91f0-46f9-90a5-e962b8163ee8"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 26 15:41:39.020857 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:39.020816 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xq5fn\" (UniqueName: \"kubernetes.io/projected/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-kube-api-access-xq5fn\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:41:39.020857 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:39.020848 2547 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-online-tls\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:41:39.020857 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:39.020857 2547 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-offline-tls\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:41:39.020857 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:39.020868 2547 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-registry-tls\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:41:39.021146 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:39.020876 2547 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-ui-tls\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:41:39.021146 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:39.020883 2547 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8-feast-data\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:41:39.567496 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:39.567464 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" event={"ID":"8a3ad2b2-91f0-46f9-90a5-e962b8163ee8","Type":"ContainerDied","Data":"350508946f3a6566868d8eacf3412c8797d8806d2ec384cec362f5032d6d4452"} May 26 15:41:39.567914 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:39.567510 2547 scope.go:117] "RemoveContainer" containerID="c976676a8587c60258cbad9efcd447dc686b36b2fde691b89073f8b0697b7bc2" May 26 15:41:39.567914 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:39.567511 2547 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j" May 26 15:41:39.577359 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:39.577304 2547 scope.go:117] "RemoveContainer" containerID="37b10e293c4486b7c7b174d837f180ee997c58c01c08e9b93536576a99cb37ac" May 26 15:41:39.585442 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:39.585422 2547 scope.go:117] "RemoveContainer" containerID="d734f44c36b9fccfcb010c3230856d79149b5527dba72db9b87c311d036470cf" May 26 15:41:39.592639 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:39.592605 2547 scope.go:117] "RemoveContainer" containerID="cc0ce75b3b879abaad310cbd9fcbc76c4e130b1177b8b838984878a8c6bdc940" May 26 15:41:39.594845 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:39.594825 2547 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j"] May 26 15:41:39.597859 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:39.597840 2547 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-jft6j"] May 26 15:41:39.600991 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:39.600973 2547 scope.go:117] "RemoveContainer" containerID="0d1af9fda0a71e0e1f99a457f4efbaa8e64aa6e98deb936210d919a986bfd400" May 26 15:41:39.607763 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:39.607743 2547 scope.go:117] "RemoveContainer" containerID="b3b84df9b1a852e35fddde24b0466e3cf69b8a8a15145d8bc1405b9d273c8ba6" May 26 15:41:39.913534 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:41:39.913454 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" path="/var/lib/kubelet/pods/8a3ad2b2-91f0-46f9-90a5-e962b8163ee8/volumes" May 26 15:42:30.341988 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:30.341945 2547 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg"] May 26 15:42:30.342592 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:30.342396 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" containerName="offline" May 26 15:42:30.342592 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:30.342415 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" containerName="offline" May 26 15:42:30.342592 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:30.342432 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" containerName="feast-init" May 26 15:42:30.342592 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:30.342442 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" containerName="feast-init" May 26 15:42:30.342592 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:30.342454 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" containerName="feast-apply" May 26 15:42:30.342592 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:30.342461 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" containerName="feast-apply" May 26 15:42:30.342592 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:30.342476 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" containerName="registry" May 26 15:42:30.342592 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:30.342484 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" containerName="registry" May 26 15:42:30.342592 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:30.342500 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" containerName="ui" May 26 15:42:30.342592 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:30.342508 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" containerName="ui" May 26 15:42:30.342592 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:30.342520 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" containerName="online" May 26 15:42:30.342592 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:30.342528 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" containerName="online" May 26 15:42:30.343174 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:30.342636 2547 memory_manager.go:356] "RemoveStaleState removing state" podUID="8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" containerName="ui" May 26 15:42:30.343174 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:30.342655 2547 memory_manager.go:356] "RemoveStaleState removing state" podUID="8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" containerName="registry" May 26 15:42:30.343174 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:30.342666 2547 memory_manager.go:356] "RemoveStaleState removing state" podUID="8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" containerName="offline" May 26 15:42:30.343174 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:30.342677 2547 memory_manager.go:356] "RemoveStaleState removing state" podUID="8a3ad2b2-91f0-46f9-90a5-e962b8163ee8" containerName="online" May 26 15:42:30.346332 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:30.346315 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" May 26 15:42:30.348458 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:30.348435 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" May 26 15:42:30.348650 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:30.348627 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" May 26 15:42:30.348776 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:30.348489 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" May 26 15:42:30.348776 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:30.348491 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" May 26 15:42:30.348776 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:30.348506 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-g5z7j\"" May 26 15:42:30.348944 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:30.348533 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" May 26 15:42:30.349010 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:30.348996 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" May 26 15:42:30.353965 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:30.353940 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg"] May 26 15:42:30.525485 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:30.525448 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/ce2fb726-f446-4bce-9eb8-072f241d3e7d-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-zltrg\" (UID: \"ce2fb726-f446-4bce-9eb8-072f241d3e7d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" May 26 15:42:30.525485 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:30.525483 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ce2fb726-f446-4bce-9eb8-072f241d3e7d-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-zltrg\" (UID: \"ce2fb726-f446-4bce-9eb8-072f241d3e7d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" May 26 15:42:30.525711 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:30.525510 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p99vb\" (UniqueName: \"kubernetes.io/projected/ce2fb726-f446-4bce-9eb8-072f241d3e7d-kube-api-access-p99vb\") pod \"feast-simple-feast-remote-setup-5cff94746b-zltrg\" (UID: \"ce2fb726-f446-4bce-9eb8-072f241d3e7d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" May 26 15:42:30.525711 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:30.525538 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ce2fb726-f446-4bce-9eb8-072f241d3e7d-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-zltrg\" (UID: \"ce2fb726-f446-4bce-9eb8-072f241d3e7d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" May 26 15:42:30.525711 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:30.525642 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ce2fb726-f446-4bce-9eb8-072f241d3e7d-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-zltrg\" (UID: \"ce2fb726-f446-4bce-9eb8-072f241d3e7d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" May 26 15:42:30.525858 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:30.525756 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ce2fb726-f446-4bce-9eb8-072f241d3e7d-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-zltrg\" (UID: \"ce2fb726-f446-4bce-9eb8-072f241d3e7d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" May 26 15:42:30.627178 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:30.627105 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/ce2fb726-f446-4bce-9eb8-072f241d3e7d-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-zltrg\" (UID: \"ce2fb726-f446-4bce-9eb8-072f241d3e7d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" May 26 15:42:30.627178 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:30.627137 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ce2fb726-f446-4bce-9eb8-072f241d3e7d-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-zltrg\" (UID: \"ce2fb726-f446-4bce-9eb8-072f241d3e7d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" May 26 15:42:30.627178 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:30.627162 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p99vb\" (UniqueName: \"kubernetes.io/projected/ce2fb726-f446-4bce-9eb8-072f241d3e7d-kube-api-access-p99vb\") pod \"feast-simple-feast-remote-setup-5cff94746b-zltrg\" (UID: \"ce2fb726-f446-4bce-9eb8-072f241d3e7d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" May 26 15:42:30.627452 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:30.627182 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ce2fb726-f446-4bce-9eb8-072f241d3e7d-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-zltrg\" (UID: \"ce2fb726-f446-4bce-9eb8-072f241d3e7d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" May 26 15:42:30.627452 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:30.627213 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ce2fb726-f446-4bce-9eb8-072f241d3e7d-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-zltrg\" (UID: \"ce2fb726-f446-4bce-9eb8-072f241d3e7d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" May 26 15:42:30.627452 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:30.627252 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ce2fb726-f446-4bce-9eb8-072f241d3e7d-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-zltrg\" (UID: \"ce2fb726-f446-4bce-9eb8-072f241d3e7d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" May 26 15:42:30.627452 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:42:30.627270 2547 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-offline-tls: secret "feast-simple-feast-remote-setup-offline-tls" not found May 26 15:42:30.627452 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:42:30.627356 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce2fb726-f446-4bce-9eb8-072f241d3e7d-offline-tls podName:ce2fb726-f446-4bce-9eb8-072f241d3e7d nodeName:}" failed. No retries permitted until 2026-05-26 15:42:31.127333617 +0000 UTC m=+1299.854008531 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/ce2fb726-f446-4bce-9eb8-072f241d3e7d-offline-tls") pod "feast-simple-feast-remote-setup-5cff94746b-zltrg" (UID: "ce2fb726-f446-4bce-9eb8-072f241d3e7d") : secret "feast-simple-feast-remote-setup-offline-tls" not found May 26 15:42:30.627747 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:42:30.627520 2547 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-online-tls: secret "feast-simple-feast-remote-setup-online-tls" not found May 26 15:42:30.627747 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:42:30.627556 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce2fb726-f446-4bce-9eb8-072f241d3e7d-online-tls podName:ce2fb726-f446-4bce-9eb8-072f241d3e7d nodeName:}" failed. No retries permitted until 2026-05-26 15:42:31.127543799 +0000 UTC m=+1299.854218708 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/ce2fb726-f446-4bce-9eb8-072f241d3e7d-online-tls") pod "feast-simple-feast-remote-setup-5cff94746b-zltrg" (UID: "ce2fb726-f446-4bce-9eb8-072f241d3e7d") : secret "feast-simple-feast-remote-setup-online-tls" not found May 26 15:42:30.627747 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:30.627672 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ce2fb726-f446-4bce-9eb8-072f241d3e7d-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-zltrg\" (UID: \"ce2fb726-f446-4bce-9eb8-072f241d3e7d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" May 26 15:42:30.627963 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:30.627942 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/ce2fb726-f446-4bce-9eb8-072f241d3e7d-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-zltrg\" (UID: \"ce2fb726-f446-4bce-9eb8-072f241d3e7d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" May 26 15:42:30.629705 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:30.629679 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ce2fb726-f446-4bce-9eb8-072f241d3e7d-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-zltrg\" (UID: \"ce2fb726-f446-4bce-9eb8-072f241d3e7d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" May 26 15:42:30.639309 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:30.639287 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p99vb\" (UniqueName: \"kubernetes.io/projected/ce2fb726-f446-4bce-9eb8-072f241d3e7d-kube-api-access-p99vb\") pod \"feast-simple-feast-remote-setup-5cff94746b-zltrg\" (UID: \"ce2fb726-f446-4bce-9eb8-072f241d3e7d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" May 26 15:42:31.131887 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:31.131859 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ce2fb726-f446-4bce-9eb8-072f241d3e7d-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-zltrg\" (UID: \"ce2fb726-f446-4bce-9eb8-072f241d3e7d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" May 26 15:42:31.132038 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:31.131906 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ce2fb726-f446-4bce-9eb8-072f241d3e7d-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-zltrg\" (UID: \"ce2fb726-f446-4bce-9eb8-072f241d3e7d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" May 26 15:42:31.132038 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:42:31.131996 2547 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-online-tls: secret "feast-simple-feast-remote-setup-online-tls" not found May 26 15:42:31.132154 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:42:31.132073 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce2fb726-f446-4bce-9eb8-072f241d3e7d-online-tls podName:ce2fb726-f446-4bce-9eb8-072f241d3e7d nodeName:}" failed. No retries permitted until 2026-05-26 15:42:32.132040355 +0000 UTC m=+1300.858715255 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/ce2fb726-f446-4bce-9eb8-072f241d3e7d-online-tls") pod "feast-simple-feast-remote-setup-5cff94746b-zltrg" (UID: "ce2fb726-f446-4bce-9eb8-072f241d3e7d") : secret "feast-simple-feast-remote-setup-online-tls" not found May 26 15:42:31.134197 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:31.134173 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ce2fb726-f446-4bce-9eb8-072f241d3e7d-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-zltrg\" (UID: \"ce2fb726-f446-4bce-9eb8-072f241d3e7d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" May 26 15:42:32.138476 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:32.138445 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ce2fb726-f446-4bce-9eb8-072f241d3e7d-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-zltrg\" (UID: \"ce2fb726-f446-4bce-9eb8-072f241d3e7d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" May 26 15:42:32.140706 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:32.140688 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ce2fb726-f446-4bce-9eb8-072f241d3e7d-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-zltrg\" (UID: \"ce2fb726-f446-4bce-9eb8-072f241d3e7d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" May 26 15:42:32.159103 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:32.159069 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" May 26 15:42:32.289572 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:32.289548 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg"] May 26 15:42:32.291508 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:42:32.291481 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podce2fb726_f446_4bce_9eb8_072f241d3e7d.slice/crio-0a170ab41fe48167c51792ea021fb35d10cb95b796d4024e55e49e9772d72dfb WatchSource:0}: Error finding container 0a170ab41fe48167c51792ea021fb35d10cb95b796d4024e55e49e9772d72dfb: Status 404 returned error can't find the container with id 0a170ab41fe48167c51792ea021fb35d10cb95b796d4024e55e49e9772d72dfb May 26 15:42:32.726030 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:32.725994 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" event={"ID":"ce2fb726-f446-4bce-9eb8-072f241d3e7d","Type":"ContainerStarted","Data":"c6c58b6146501c09748c45a6fdb555acc9ebfa003fbe7dad0ece0f537efe8000"} May 26 15:42:32.726218 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:32.726036 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" event={"ID":"ce2fb726-f446-4bce-9eb8-072f241d3e7d","Type":"ContainerStarted","Data":"0a170ab41fe48167c51792ea021fb35d10cb95b796d4024e55e49e9772d72dfb"} May 26 15:42:37.743142 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:37.743108 2547 generic.go:358] "Generic (PLEG): container finished" podID="ce2fb726-f446-4bce-9eb8-072f241d3e7d" containerID="c6c58b6146501c09748c45a6fdb555acc9ebfa003fbe7dad0ece0f537efe8000" exitCode=0 May 26 15:42:37.743540 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:37.743187 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" event={"ID":"ce2fb726-f446-4bce-9eb8-072f241d3e7d","Type":"ContainerDied","Data":"c6c58b6146501c09748c45a6fdb555acc9ebfa003fbe7dad0ece0f537efe8000"} May 26 15:42:38.747315 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:38.747272 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" event={"ID":"ce2fb726-f446-4bce-9eb8-072f241d3e7d","Type":"ContainerStarted","Data":"4273edbcc5ed358a8a558666a5aff946d4ad2ab65e3a20524ad1ae51947acb6d"} May 26 15:42:43.768019 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:43.767986 2547 generic.go:358] "Generic (PLEG): container finished" podID="ce2fb726-f446-4bce-9eb8-072f241d3e7d" containerID="4273edbcc5ed358a8a558666a5aff946d4ad2ab65e3a20524ad1ae51947acb6d" exitCode=0 May 26 15:42:43.768475 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:43.768076 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" event={"ID":"ce2fb726-f446-4bce-9eb8-072f241d3e7d","Type":"ContainerDied","Data":"4273edbcc5ed358a8a558666a5aff946d4ad2ab65e3a20524ad1ae51947acb6d"} May 26 15:42:44.774590 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:44.774547 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" event={"ID":"ce2fb726-f446-4bce-9eb8-072f241d3e7d","Type":"ContainerStarted","Data":"354b7a6a4c30fa9657fdfad123c6390daf3c5ff798237675d052e8cd1bd4953d"} May 26 15:42:44.775006 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:44.774605 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" event={"ID":"ce2fb726-f446-4bce-9eb8-072f241d3e7d","Type":"ContainerStarted","Data":"39cfff21a5ebcdaebd020650298ffc2b76566cf8e45f84b1a9ec12bcdbee2a10"} May 26 15:42:44.775006 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:44.774624 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" event={"ID":"ce2fb726-f446-4bce-9eb8-072f241d3e7d","Type":"ContainerStarted","Data":"d4c3e3f3ef4c5eac08b6b334acd8c2f56774ca0a534d41b72a2c0127a0d2abe2"} May 26 15:42:44.797881 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:44.797793 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" podStartSLOduration=14.797771922999999 podStartE2EDuration="14.797771923s" podCreationTimestamp="2026-05-26 15:42:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 15:42:44.796222608 +0000 UTC m=+1313.522897529" watchObservedRunningTime="2026-05-26 15:42:44.797771923 +0000 UTC m=+1313.524446845" May 26 15:42:47.159871 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:47.159831 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" May 26 15:42:47.160345 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:47.159888 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" May 26 15:42:47.160345 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:47.159903 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" May 26 15:42:47.161781 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:47.161741 2547 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" podUID="ce2fb726-f446-4bce-9eb8-072f241d3e7d" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.33:8443: connect: connection refused" May 26 15:42:47.161781 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:47.161759 2547 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" podUID="ce2fb726-f446-4bce-9eb8-072f241d3e7d" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.33:8816: connect: connection refused" May 26 15:42:47.161974 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:47.161803 2547 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" podUID="ce2fb726-f446-4bce-9eb8-072f241d3e7d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused" May 26 15:42:50.160249 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:50.160212 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" May 26 15:42:50.160644 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:50.160385 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" May 26 15:42:50.160848 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:50.160817 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" May 26 15:42:50.160984 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:50.160855 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" May 26 15:42:50.161119 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:50.161100 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" May 26 15:42:50.161195 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:50.161160 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" May 26 15:42:50.164441 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:50.164422 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" May 26 15:42:50.795419 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:50.795389 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" May 26 15:42:50.798271 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:42:50.798253 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" May 26 15:43:53.602136 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:43:53.602097 2547 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg"] May 26 15:43:53.612109 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:43:53.612021 2547 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" podUID="ce2fb726-f446-4bce-9eb8-072f241d3e7d" containerName="online" containerID="cri-o://d4c3e3f3ef4c5eac08b6b334acd8c2f56774ca0a534d41b72a2c0127a0d2abe2" gracePeriod=30 May 26 15:43:53.612281 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:43:53.612129 2547 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" podUID="ce2fb726-f446-4bce-9eb8-072f241d3e7d" containerName="ui" containerID="cri-o://354b7a6a4c30fa9657fdfad123c6390daf3c5ff798237675d052e8cd1bd4953d" gracePeriod=30 May 26 15:43:53.612376 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:43:53.612330 2547 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" podUID="ce2fb726-f446-4bce-9eb8-072f241d3e7d" containerName="offline" containerID="cri-o://39cfff21a5ebcdaebd020650298ffc2b76566cf8e45f84b1a9ec12bcdbee2a10" gracePeriod=30 May 26 15:43:53.744031 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:43:53.743990 2547 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5"] May 26 15:43:53.744392 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:43:53.744339 2547 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" podUID="7548eb6f-39bf-4632-9bd9-8e0780a40550" containerName="registry" containerID="cri-o://f138312b178c4064fcd07588b119dc4daa960f2ab45dbbb8433d32804fe6e406" gracePeriod=30 May 26 15:43:53.744492 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:43:53.744392 2547 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" podUID="7548eb6f-39bf-4632-9bd9-8e0780a40550" containerName="offline" containerID="cri-o://985d9dc0497ab53ca1c88f93280cf218790ffe688f02d2f6b0148401e6a8ec03" gracePeriod=30 May 26 15:43:53.744492 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:43:53.744396 2547 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" podUID="7548eb6f-39bf-4632-9bd9-8e0780a40550" containerName="ui" containerID="cri-o://48de061a279017064ec4474ba7ebcade035496a2d19fde74ff969612f3a9332f" gracePeriod=30 May 26 15:43:53.744492 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:43:53.744488 2547 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" podUID="7548eb6f-39bf-4632-9bd9-8e0780a40550" containerName="online" containerID="cri-o://a45507e726e6f853a163d180b304f73a03d566351b72dfa2a7e43a9fd57aa35e" gracePeriod=30 May 26 15:43:54.981443 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:43:54.981410 2547 generic.go:358] "Generic (PLEG): container finished" podID="7548eb6f-39bf-4632-9bd9-8e0780a40550" containerID="48de061a279017064ec4474ba7ebcade035496a2d19fde74ff969612f3a9332f" exitCode=0 May 26 15:43:54.981443 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:43:54.981435 2547 generic.go:358] "Generic (PLEG): container finished" podID="7548eb6f-39bf-4632-9bd9-8e0780a40550" containerID="a45507e726e6f853a163d180b304f73a03d566351b72dfa2a7e43a9fd57aa35e" exitCode=0 May 26 15:43:54.981903 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:43:54.981486 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" event={"ID":"7548eb6f-39bf-4632-9bd9-8e0780a40550","Type":"ContainerDied","Data":"48de061a279017064ec4474ba7ebcade035496a2d19fde74ff969612f3a9332f"} May 26 15:43:54.981903 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:43:54.981528 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" event={"ID":"7548eb6f-39bf-4632-9bd9-8e0780a40550","Type":"ContainerDied","Data":"a45507e726e6f853a163d180b304f73a03d566351b72dfa2a7e43a9fd57aa35e"} May 26 15:43:54.983663 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:43:54.983640 2547 generic.go:358] "Generic (PLEG): container finished" podID="ce2fb726-f446-4bce-9eb8-072f241d3e7d" containerID="354b7a6a4c30fa9657fdfad123c6390daf3c5ff798237675d052e8cd1bd4953d" exitCode=0 May 26 15:43:54.983663 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:43:54.983658 2547 generic.go:358] "Generic (PLEG): container finished" podID="ce2fb726-f446-4bce-9eb8-072f241d3e7d" containerID="d4c3e3f3ef4c5eac08b6b334acd8c2f56774ca0a534d41b72a2c0127a0d2abe2" exitCode=0 May 26 15:43:54.983818 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:43:54.983693 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" event={"ID":"ce2fb726-f446-4bce-9eb8-072f241d3e7d","Type":"ContainerDied","Data":"354b7a6a4c30fa9657fdfad123c6390daf3c5ff798237675d052e8cd1bd4953d"} May 26 15:43:54.983818 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:43:54.983714 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" event={"ID":"ce2fb726-f446-4bce-9eb8-072f241d3e7d","Type":"ContainerDied","Data":"d4c3e3f3ef4c5eac08b6b334acd8c2f56774ca0a534d41b72a2c0127a0d2abe2"} May 26 15:43:57.543982 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:43:57.543934 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" podUID="7548eb6f-39bf-4632-9bd9-8e0780a40550" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.32:8443: connect: connection refused" May 26 15:43:58.533770 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:43:58.533727 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" podUID="7548eb6f-39bf-4632-9bd9-8e0780a40550" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused" May 26 15:44:00.161126 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:00.161082 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" podUID="ce2fb726-f446-4bce-9eb8-072f241d3e7d" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.33:8443: connect: connection refused" May 26 15:44:00.795950 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:00.795913 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" podUID="ce2fb726-f446-4bce-9eb8-072f241d3e7d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused" May 26 15:44:07.544697 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:07.544652 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" podUID="7548eb6f-39bf-4632-9bd9-8e0780a40550" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.32:8443: connect: connection refused" May 26 15:44:08.533894 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:08.533850 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" podUID="7548eb6f-39bf-4632-9bd9-8e0780a40550" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused" May 26 15:44:10.161825 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:10.161770 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" podUID="ce2fb726-f446-4bce-9eb8-072f241d3e7d" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.33:8443: connect: connection refused" May 26 15:44:10.796142 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:10.796105 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" podUID="ce2fb726-f446-4bce-9eb8-072f241d3e7d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused" May 26 15:44:17.544146 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:17.544092 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" podUID="7548eb6f-39bf-4632-9bd9-8e0780a40550" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.32:8443: connect: connection refused" May 26 15:44:17.544556 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:17.544268 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" May 26 15:44:18.533633 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:18.533586 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" podUID="7548eb6f-39bf-4632-9bd9-8e0780a40550" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused" May 26 15:44:18.533814 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:18.533729 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" May 26 15:44:20.161607 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:20.161561 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" podUID="ce2fb726-f446-4bce-9eb8-072f241d3e7d" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.33:8443: connect: connection refused" May 26 15:44:20.161999 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:20.161694 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" May 26 15:44:20.796475 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:20.796427 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" podUID="ce2fb726-f446-4bce-9eb8-072f241d3e7d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused" May 26 15:44:20.796645 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:20.796608 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" May 26 15:44:23.865604 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:23.865580 2547 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" May 26 15:44:23.941314 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:23.941282 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ce2fb726-f446-4bce-9eb8-072f241d3e7d-online-tls\") pod \"ce2fb726-f446-4bce-9eb8-072f241d3e7d\" (UID: \"ce2fb726-f446-4bce-9eb8-072f241d3e7d\") " May 26 15:44:23.941500 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:23.941333 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p99vb\" (UniqueName: \"kubernetes.io/projected/ce2fb726-f446-4bce-9eb8-072f241d3e7d-kube-api-access-p99vb\") pod \"ce2fb726-f446-4bce-9eb8-072f241d3e7d\" (UID: \"ce2fb726-f446-4bce-9eb8-072f241d3e7d\") " May 26 15:44:23.941500 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:23.941360 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ce2fb726-f446-4bce-9eb8-072f241d3e7d-feast-data\") pod \"ce2fb726-f446-4bce-9eb8-072f241d3e7d\" (UID: \"ce2fb726-f446-4bce-9eb8-072f241d3e7d\") " May 26 15:44:23.941500 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:23.941396 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/ce2fb726-f446-4bce-9eb8-072f241d3e7d-registry-tls\") pod \"ce2fb726-f446-4bce-9eb8-072f241d3e7d\" (UID: \"ce2fb726-f446-4bce-9eb8-072f241d3e7d\") " May 26 15:44:23.941500 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:23.941430 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ce2fb726-f446-4bce-9eb8-072f241d3e7d-offline-tls\") pod \"ce2fb726-f446-4bce-9eb8-072f241d3e7d\" (UID: \"ce2fb726-f446-4bce-9eb8-072f241d3e7d\") " May 26 15:44:23.941500 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:23.941446 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ce2fb726-f446-4bce-9eb8-072f241d3e7d-ui-tls\") pod \"ce2fb726-f446-4bce-9eb8-072f241d3e7d\" (UID: \"ce2fb726-f446-4bce-9eb8-072f241d3e7d\") " May 26 15:44:23.941818 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:23.941787 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce2fb726-f446-4bce-9eb8-072f241d3e7d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ce2fb726-f446-4bce-9eb8-072f241d3e7d" (UID: "ce2fb726-f446-4bce-9eb8-072f241d3e7d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 26 15:44:23.941950 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:23.941887 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce2fb726-f446-4bce-9eb8-072f241d3e7d-feast-data" (OuterVolumeSpecName: "feast-data") pod "ce2fb726-f446-4bce-9eb8-072f241d3e7d" (UID: "ce2fb726-f446-4bce-9eb8-072f241d3e7d"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 26 15:44:23.943729 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:23.943694 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce2fb726-f446-4bce-9eb8-072f241d3e7d-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "ce2fb726-f446-4bce-9eb8-072f241d3e7d" (UID: "ce2fb726-f446-4bce-9eb8-072f241d3e7d"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 26 15:44:23.943729 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:23.943706 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce2fb726-f446-4bce-9eb8-072f241d3e7d-online-tls" (OuterVolumeSpecName: "online-tls") pod "ce2fb726-f446-4bce-9eb8-072f241d3e7d" (UID: "ce2fb726-f446-4bce-9eb8-072f241d3e7d"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 26 15:44:23.943848 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:23.943728 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce2fb726-f446-4bce-9eb8-072f241d3e7d-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "ce2fb726-f446-4bce-9eb8-072f241d3e7d" (UID: "ce2fb726-f446-4bce-9eb8-072f241d3e7d"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 26 15:44:23.943848 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:23.943796 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce2fb726-f446-4bce-9eb8-072f241d3e7d-kube-api-access-p99vb" (OuterVolumeSpecName: "kube-api-access-p99vb") pod "ce2fb726-f446-4bce-9eb8-072f241d3e7d" (UID: "ce2fb726-f446-4bce-9eb8-072f241d3e7d"). InnerVolumeSpecName "kube-api-access-p99vb". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 26 15:44:24.042494 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.042446 2547 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ce2fb726-f446-4bce-9eb8-072f241d3e7d-online-tls\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:44:24.042494 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.042488 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p99vb\" (UniqueName: \"kubernetes.io/projected/ce2fb726-f446-4bce-9eb8-072f241d3e7d-kube-api-access-p99vb\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:44:24.042494 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.042499 2547 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ce2fb726-f446-4bce-9eb8-072f241d3e7d-feast-data\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:44:24.042494 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.042509 2547 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/ce2fb726-f446-4bce-9eb8-072f241d3e7d-registry-tls\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:44:24.042793 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.042521 2547 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ce2fb726-f446-4bce-9eb8-072f241d3e7d-offline-tls\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:44:24.042793 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.042533 2547 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ce2fb726-f446-4bce-9eb8-072f241d3e7d-ui-tls\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:44:24.062092 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.062037 2547 generic.go:358] "Generic (PLEG): container finished" podID="ce2fb726-f446-4bce-9eb8-072f241d3e7d" containerID="39cfff21a5ebcdaebd020650298ffc2b76566cf8e45f84b1a9ec12bcdbee2a10" exitCode=137 May 26 15:44:24.062228 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.062090 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" event={"ID":"ce2fb726-f446-4bce-9eb8-072f241d3e7d","Type":"ContainerDied","Data":"39cfff21a5ebcdaebd020650298ffc2b76566cf8e45f84b1a9ec12bcdbee2a10"} May 26 15:44:24.062228 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.062130 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" event={"ID":"ce2fb726-f446-4bce-9eb8-072f241d3e7d","Type":"ContainerDied","Data":"0a170ab41fe48167c51792ea021fb35d10cb95b796d4024e55e49e9772d72dfb"} May 26 15:44:24.062228 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.062132 2547 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg" May 26 15:44:24.062228 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.062153 2547 scope.go:117] "RemoveContainer" containerID="354b7a6a4c30fa9657fdfad123c6390daf3c5ff798237675d052e8cd1bd4953d" May 26 15:44:24.065184 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.065160 2547 generic.go:358] "Generic (PLEG): container finished" podID="7548eb6f-39bf-4632-9bd9-8e0780a40550" containerID="985d9dc0497ab53ca1c88f93280cf218790ffe688f02d2f6b0148401e6a8ec03" exitCode=137 May 26 15:44:24.065184 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.065183 2547 generic.go:358] "Generic (PLEG): container finished" podID="7548eb6f-39bf-4632-9bd9-8e0780a40550" containerID="f138312b178c4064fcd07588b119dc4daa960f2ab45dbbb8433d32804fe6e406" exitCode=137 May 26 15:44:24.065322 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.065227 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" event={"ID":"7548eb6f-39bf-4632-9bd9-8e0780a40550","Type":"ContainerDied","Data":"985d9dc0497ab53ca1c88f93280cf218790ffe688f02d2f6b0148401e6a8ec03"} May 26 15:44:24.065322 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.065258 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" event={"ID":"7548eb6f-39bf-4632-9bd9-8e0780a40550","Type":"ContainerDied","Data":"f138312b178c4064fcd07588b119dc4daa960f2ab45dbbb8433d32804fe6e406"} May 26 15:44:24.070822 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.070805 2547 scope.go:117] "RemoveContainer" containerID="39cfff21a5ebcdaebd020650298ffc2b76566cf8e45f84b1a9ec12bcdbee2a10" May 26 15:44:24.078560 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.078543 2547 scope.go:117] "RemoveContainer" containerID="d4c3e3f3ef4c5eac08b6b334acd8c2f56774ca0a534d41b72a2c0127a0d2abe2" May 26 15:44:24.083878 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.083850 2547 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg"] May 26 15:44:24.087106 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.087082 2547 scope.go:117] "RemoveContainer" containerID="4273edbcc5ed358a8a558666a5aff946d4ad2ab65e3a20524ad1ae51947acb6d" May 26 15:44:24.087218 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.087154 2547 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-zltrg"] May 26 15:44:24.094115 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.094094 2547 scope.go:117] "RemoveContainer" containerID="c6c58b6146501c09748c45a6fdb555acc9ebfa003fbe7dad0ece0f537efe8000" May 26 15:44:24.101158 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.101139 2547 scope.go:117] "RemoveContainer" containerID="354b7a6a4c30fa9657fdfad123c6390daf3c5ff798237675d052e8cd1bd4953d" May 26 15:44:24.101428 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:44:24.101407 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"354b7a6a4c30fa9657fdfad123c6390daf3c5ff798237675d052e8cd1bd4953d\": container with ID starting with 354b7a6a4c30fa9657fdfad123c6390daf3c5ff798237675d052e8cd1bd4953d not found: ID does not exist" containerID="354b7a6a4c30fa9657fdfad123c6390daf3c5ff798237675d052e8cd1bd4953d" May 26 15:44:24.101473 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.101439 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"354b7a6a4c30fa9657fdfad123c6390daf3c5ff798237675d052e8cd1bd4953d"} err="failed to get container status \"354b7a6a4c30fa9657fdfad123c6390daf3c5ff798237675d052e8cd1bd4953d\": rpc error: code = NotFound desc = could not find container \"354b7a6a4c30fa9657fdfad123c6390daf3c5ff798237675d052e8cd1bd4953d\": container with ID starting with 354b7a6a4c30fa9657fdfad123c6390daf3c5ff798237675d052e8cd1bd4953d not found: ID does not exist" May 26 15:44:24.101473 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.101459 2547 scope.go:117] "RemoveContainer" containerID="39cfff21a5ebcdaebd020650298ffc2b76566cf8e45f84b1a9ec12bcdbee2a10" May 26 15:44:24.101673 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:44:24.101657 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39cfff21a5ebcdaebd020650298ffc2b76566cf8e45f84b1a9ec12bcdbee2a10\": container with ID starting with 39cfff21a5ebcdaebd020650298ffc2b76566cf8e45f84b1a9ec12bcdbee2a10 not found: ID does not exist" containerID="39cfff21a5ebcdaebd020650298ffc2b76566cf8e45f84b1a9ec12bcdbee2a10" May 26 15:44:24.101718 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.101678 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39cfff21a5ebcdaebd020650298ffc2b76566cf8e45f84b1a9ec12bcdbee2a10"} err="failed to get container status \"39cfff21a5ebcdaebd020650298ffc2b76566cf8e45f84b1a9ec12bcdbee2a10\": rpc error: code = NotFound desc = could not find container \"39cfff21a5ebcdaebd020650298ffc2b76566cf8e45f84b1a9ec12bcdbee2a10\": container with ID starting with 39cfff21a5ebcdaebd020650298ffc2b76566cf8e45f84b1a9ec12bcdbee2a10 not found: ID does not exist" May 26 15:44:24.101718 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.101692 2547 scope.go:117] "RemoveContainer" containerID="d4c3e3f3ef4c5eac08b6b334acd8c2f56774ca0a534d41b72a2c0127a0d2abe2" May 26 15:44:24.101897 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:44:24.101878 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4c3e3f3ef4c5eac08b6b334acd8c2f56774ca0a534d41b72a2c0127a0d2abe2\": container with ID starting with d4c3e3f3ef4c5eac08b6b334acd8c2f56774ca0a534d41b72a2c0127a0d2abe2 not found: ID does not exist" containerID="d4c3e3f3ef4c5eac08b6b334acd8c2f56774ca0a534d41b72a2c0127a0d2abe2" May 26 15:44:24.101959 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.101905 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4c3e3f3ef4c5eac08b6b334acd8c2f56774ca0a534d41b72a2c0127a0d2abe2"} err="failed to get container status \"d4c3e3f3ef4c5eac08b6b334acd8c2f56774ca0a534d41b72a2c0127a0d2abe2\": rpc error: code = NotFound desc = could not find container \"d4c3e3f3ef4c5eac08b6b334acd8c2f56774ca0a534d41b72a2c0127a0d2abe2\": container with ID starting with d4c3e3f3ef4c5eac08b6b334acd8c2f56774ca0a534d41b72a2c0127a0d2abe2 not found: ID does not exist" May 26 15:44:24.101959 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.101931 2547 scope.go:117] "RemoveContainer" containerID="4273edbcc5ed358a8a558666a5aff946d4ad2ab65e3a20524ad1ae51947acb6d" May 26 15:44:24.102183 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:44:24.102167 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4273edbcc5ed358a8a558666a5aff946d4ad2ab65e3a20524ad1ae51947acb6d\": container with ID starting with 4273edbcc5ed358a8a558666a5aff946d4ad2ab65e3a20524ad1ae51947acb6d not found: ID does not exist" containerID="4273edbcc5ed358a8a558666a5aff946d4ad2ab65e3a20524ad1ae51947acb6d" May 26 15:44:24.102233 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.102186 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4273edbcc5ed358a8a558666a5aff946d4ad2ab65e3a20524ad1ae51947acb6d"} err="failed to get container status \"4273edbcc5ed358a8a558666a5aff946d4ad2ab65e3a20524ad1ae51947acb6d\": rpc error: code = NotFound desc = could not find container \"4273edbcc5ed358a8a558666a5aff946d4ad2ab65e3a20524ad1ae51947acb6d\": container with ID starting with 4273edbcc5ed358a8a558666a5aff946d4ad2ab65e3a20524ad1ae51947acb6d not found: ID does not exist" May 26 15:44:24.102233 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.102199 2547 scope.go:117] "RemoveContainer" containerID="c6c58b6146501c09748c45a6fdb555acc9ebfa003fbe7dad0ece0f537efe8000" May 26 15:44:24.102381 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:44:24.102366 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6c58b6146501c09748c45a6fdb555acc9ebfa003fbe7dad0ece0f537efe8000\": container with ID starting with c6c58b6146501c09748c45a6fdb555acc9ebfa003fbe7dad0ece0f537efe8000 not found: ID does not exist" containerID="c6c58b6146501c09748c45a6fdb555acc9ebfa003fbe7dad0ece0f537efe8000" May 26 15:44:24.102426 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.102383 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6c58b6146501c09748c45a6fdb555acc9ebfa003fbe7dad0ece0f537efe8000"} err="failed to get container status \"c6c58b6146501c09748c45a6fdb555acc9ebfa003fbe7dad0ece0f537efe8000\": rpc error: code = NotFound desc = could not find container \"c6c58b6146501c09748c45a6fdb555acc9ebfa003fbe7dad0ece0f537efe8000\": container with ID starting with c6c58b6146501c09748c45a6fdb555acc9ebfa003fbe7dad0ece0f537efe8000 not found: ID does not exist" May 26 15:44:24.388030 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.388008 2547 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" May 26 15:44:24.547941 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.547852 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7548eb6f-39bf-4632-9bd9-8e0780a40550-feast-data\") pod \"7548eb6f-39bf-4632-9bd9-8e0780a40550\" (UID: \"7548eb6f-39bf-4632-9bd9-8e0780a40550\") " May 26 15:44:24.547941 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.547903 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7548eb6f-39bf-4632-9bd9-8e0780a40550-online-tls\") pod \"7548eb6f-39bf-4632-9bd9-8e0780a40550\" (UID: \"7548eb6f-39bf-4632-9bd9-8e0780a40550\") " May 26 15:44:24.547941 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.547925 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7548eb6f-39bf-4632-9bd9-8e0780a40550-ui-tls\") pod \"7548eb6f-39bf-4632-9bd9-8e0780a40550\" (UID: \"7548eb6f-39bf-4632-9bd9-8e0780a40550\") " May 26 15:44:24.548237 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.548045 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7548eb6f-39bf-4632-9bd9-8e0780a40550-offline-tls\") pod \"7548eb6f-39bf-4632-9bd9-8e0780a40550\" (UID: \"7548eb6f-39bf-4632-9bd9-8e0780a40550\") " May 26 15:44:24.548237 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.548112 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tc9qp\" (UniqueName: \"kubernetes.io/projected/7548eb6f-39bf-4632-9bd9-8e0780a40550-kube-api-access-tc9qp\") pod \"7548eb6f-39bf-4632-9bd9-8e0780a40550\" (UID: \"7548eb6f-39bf-4632-9bd9-8e0780a40550\") " May 26 15:44:24.548237 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.548181 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7548eb6f-39bf-4632-9bd9-8e0780a40550-registry-tls\") pod \"7548eb6f-39bf-4632-9bd9-8e0780a40550\" (UID: \"7548eb6f-39bf-4632-9bd9-8e0780a40550\") " May 26 15:44:24.548500 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.548473 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7548eb6f-39bf-4632-9bd9-8e0780a40550-feast-data" (OuterVolumeSpecName: "feast-data") pod "7548eb6f-39bf-4632-9bd9-8e0780a40550" (UID: "7548eb6f-39bf-4632-9bd9-8e0780a40550"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 26 15:44:24.550202 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.550173 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7548eb6f-39bf-4632-9bd9-8e0780a40550-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "7548eb6f-39bf-4632-9bd9-8e0780a40550" (UID: "7548eb6f-39bf-4632-9bd9-8e0780a40550"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 26 15:44:24.550295 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.550200 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7548eb6f-39bf-4632-9bd9-8e0780a40550-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "7548eb6f-39bf-4632-9bd9-8e0780a40550" (UID: "7548eb6f-39bf-4632-9bd9-8e0780a40550"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 26 15:44:24.550295 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.550201 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7548eb6f-39bf-4632-9bd9-8e0780a40550-kube-api-access-tc9qp" (OuterVolumeSpecName: "kube-api-access-tc9qp") pod "7548eb6f-39bf-4632-9bd9-8e0780a40550" (UID: "7548eb6f-39bf-4632-9bd9-8e0780a40550"). InnerVolumeSpecName "kube-api-access-tc9qp". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 26 15:44:24.550393 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.550371 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7548eb6f-39bf-4632-9bd9-8e0780a40550-online-tls" (OuterVolumeSpecName: "online-tls") pod "7548eb6f-39bf-4632-9bd9-8e0780a40550" (UID: "7548eb6f-39bf-4632-9bd9-8e0780a40550"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 26 15:44:24.550451 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.550432 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7548eb6f-39bf-4632-9bd9-8e0780a40550-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "7548eb6f-39bf-4632-9bd9-8e0780a40550" (UID: "7548eb6f-39bf-4632-9bd9-8e0780a40550"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 26 15:44:24.649374 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.649335 2547 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7548eb6f-39bf-4632-9bd9-8e0780a40550-registry-tls\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:44:24.649374 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.649368 2547 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7548eb6f-39bf-4632-9bd9-8e0780a40550-feast-data\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:44:24.649374 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.649377 2547 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7548eb6f-39bf-4632-9bd9-8e0780a40550-online-tls\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:44:24.649689 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.649386 2547 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7548eb6f-39bf-4632-9bd9-8e0780a40550-ui-tls\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:44:24.649689 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.649395 2547 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7548eb6f-39bf-4632-9bd9-8e0780a40550-offline-tls\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:44:24.649689 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:24.649404 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tc9qp\" (UniqueName: \"kubernetes.io/projected/7548eb6f-39bf-4632-9bd9-8e0780a40550-kube-api-access-tc9qp\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:44:25.071742 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:25.071707 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" event={"ID":"7548eb6f-39bf-4632-9bd9-8e0780a40550","Type":"ContainerDied","Data":"4bbc834aa2cbaa29e7f4c2e94a0b5a7b4735d32d4cbcce51fb8c49bf0cede95b"} May 26 15:44:25.071742 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:25.071736 2547 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5" May 26 15:44:25.072398 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:25.071752 2547 scope.go:117] "RemoveContainer" containerID="48de061a279017064ec4474ba7ebcade035496a2d19fde74ff969612f3a9332f" May 26 15:44:25.080618 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:25.080597 2547 scope.go:117] "RemoveContainer" containerID="985d9dc0497ab53ca1c88f93280cf218790ffe688f02d2f6b0148401e6a8ec03" May 26 15:44:25.087592 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:25.087565 2547 scope.go:117] "RemoveContainer" containerID="a45507e726e6f853a163d180b304f73a03d566351b72dfa2a7e43a9fd57aa35e" May 26 15:44:25.093595 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:25.093568 2547 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5"] May 26 15:44:25.094957 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:25.094936 2547 scope.go:117] "RemoveContainer" containerID="f138312b178c4064fcd07588b119dc4daa960f2ab45dbbb8433d32804fe6e406" May 26 15:44:25.100408 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:25.100380 2547 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-r2gm5"] May 26 15:44:25.102297 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:25.102283 2547 scope.go:117] "RemoveContainer" containerID="a9861e6366b29d47d55036c555b673856746cfc1291a2c5dacca8b22f98c6b37" May 26 15:44:25.109172 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:25.109144 2547 scope.go:117] "RemoveContainer" containerID="0e4daa7f6ce8f09a66d07af7503d0a8e5409b09fe80cf1d137c82879404e9740" May 26 15:44:25.914415 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:25.914382 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7548eb6f-39bf-4632-9bd9-8e0780a40550" path="/var/lib/kubelet/pods/7548eb6f-39bf-4632-9bd9-8e0780a40550/volumes" May 26 15:44:25.914982 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:25.914968 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce2fb726-f446-4bce-9eb8-072f241d3e7d" path="/var/lib/kubelet/pods/ce2fb726-f446-4bce-9eb8-072f241d3e7d/volumes" May 26 15:44:42.193781 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:42.193748 2547 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-cf99db75-9cw24"] May 26 15:44:42.194254 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:42.193975 2547 kuberuntime_container.go:864] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-9cw24" podUID="620ffd1c-3511-4e5f-8c5f-19ec9f3aaaa8" containerName="manager" containerID="cri-o://05e1019496d2e5d6e975393b39112e74e1723892acc2cf19a76e078c34c885bb" gracePeriod=10 May 26 15:44:42.439095 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:42.439071 2547 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-9cw24" May 26 15:44:42.484333 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:42.484259 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-48x29\" (UniqueName: \"kubernetes.io/projected/620ffd1c-3511-4e5f-8c5f-19ec9f3aaaa8-kube-api-access-48x29\") pod \"620ffd1c-3511-4e5f-8c5f-19ec9f3aaaa8\" (UID: \"620ffd1c-3511-4e5f-8c5f-19ec9f3aaaa8\") " May 26 15:44:42.486367 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:42.486334 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/620ffd1c-3511-4e5f-8c5f-19ec9f3aaaa8-kube-api-access-48x29" (OuterVolumeSpecName: "kube-api-access-48x29") pod "620ffd1c-3511-4e5f-8c5f-19ec9f3aaaa8" (UID: "620ffd1c-3511-4e5f-8c5f-19ec9f3aaaa8"). InnerVolumeSpecName "kube-api-access-48x29". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 26 15:44:42.584819 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:42.584784 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48x29\" (UniqueName: \"kubernetes.io/projected/620ffd1c-3511-4e5f-8c5f-19ec9f3aaaa8-kube-api-access-48x29\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:44:43.123719 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:43.123684 2547 generic.go:358] "Generic (PLEG): container finished" podID="620ffd1c-3511-4e5f-8c5f-19ec9f3aaaa8" containerID="05e1019496d2e5d6e975393b39112e74e1723892acc2cf19a76e078c34c885bb" exitCode=0 May 26 15:44:43.123907 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:43.123745 2547 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-9cw24" May 26 15:44:43.123907 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:43.123773 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-9cw24" event={"ID":"620ffd1c-3511-4e5f-8c5f-19ec9f3aaaa8","Type":"ContainerDied","Data":"05e1019496d2e5d6e975393b39112e74e1723892acc2cf19a76e078c34c885bb"} May 26 15:44:43.123907 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:43.123809 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-9cw24" event={"ID":"620ffd1c-3511-4e5f-8c5f-19ec9f3aaaa8","Type":"ContainerDied","Data":"239a6a844b72a53c165fd62c3c18b65498d2942e758d5bd43326e480f6161257"} May 26 15:44:43.123907 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:43.123835 2547 scope.go:117] "RemoveContainer" containerID="05e1019496d2e5d6e975393b39112e74e1723892acc2cf19a76e078c34c885bb" May 26 15:44:43.131808 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:43.131788 2547 scope.go:117] "RemoveContainer" containerID="05e1019496d2e5d6e975393b39112e74e1723892acc2cf19a76e078c34c885bb" May 26 15:44:43.132086 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:44:43.132046 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05e1019496d2e5d6e975393b39112e74e1723892acc2cf19a76e078c34c885bb\": container with ID starting with 05e1019496d2e5d6e975393b39112e74e1723892acc2cf19a76e078c34c885bb not found: ID does not exist" containerID="05e1019496d2e5d6e975393b39112e74e1723892acc2cf19a76e078c34c885bb" May 26 15:44:43.132165 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:43.132094 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05e1019496d2e5d6e975393b39112e74e1723892acc2cf19a76e078c34c885bb"} err="failed to get container status \"05e1019496d2e5d6e975393b39112e74e1723892acc2cf19a76e078c34c885bb\": rpc error: code = NotFound desc = could not find container \"05e1019496d2e5d6e975393b39112e74e1723892acc2cf19a76e078c34c885bb\": container with ID starting with 05e1019496d2e5d6e975393b39112e74e1723892acc2cf19a76e078c34c885bb not found: ID does not exist" May 26 15:44:43.150134 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:43.150105 2547 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-cf99db75-9cw24"] May 26 15:44:43.151881 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:43.151860 2547 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-cf99db75-9cw24"] May 26 15:44:43.918633 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:43.918593 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="620ffd1c-3511-4e5f-8c5f-19ec9f3aaaa8" path="/var/lib/kubelet/pods/620ffd1c-3511-4e5f-8c5f-19ec9f3aaaa8/volumes" May 26 15:44:46.526733 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.526694 2547 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-cf99db75-7w8xg"] May 26 15:44:46.527193 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.527098 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce2fb726-f446-4bce-9eb8-072f241d3e7d" containerName="ui" May 26 15:44:46.527193 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.527116 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce2fb726-f446-4bce-9eb8-072f241d3e7d" containerName="ui" May 26 15:44:46.527193 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.527134 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce2fb726-f446-4bce-9eb8-072f241d3e7d" containerName="feast-init" May 26 15:44:46.527193 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.527142 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce2fb726-f446-4bce-9eb8-072f241d3e7d" containerName="feast-init" May 26 15:44:46.527193 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.527150 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7548eb6f-39bf-4632-9bd9-8e0780a40550" containerName="ui" May 26 15:44:46.527193 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.527157 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="7548eb6f-39bf-4632-9bd9-8e0780a40550" containerName="ui" May 26 15:44:46.527193 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.527166 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce2fb726-f446-4bce-9eb8-072f241d3e7d" containerName="offline" May 26 15:44:46.527193 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.527174 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce2fb726-f446-4bce-9eb8-072f241d3e7d" containerName="offline" May 26 15:44:46.527193 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.527191 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7548eb6f-39bf-4632-9bd9-8e0780a40550" containerName="feast-apply" May 26 15:44:46.527193 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.527199 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="7548eb6f-39bf-4632-9bd9-8e0780a40550" containerName="feast-apply" May 26 15:44:46.527667 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.527211 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="620ffd1c-3511-4e5f-8c5f-19ec9f3aaaa8" containerName="manager" May 26 15:44:46.527667 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.527219 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="620ffd1c-3511-4e5f-8c5f-19ec9f3aaaa8" containerName="manager" May 26 15:44:46.527667 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.527228 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7548eb6f-39bf-4632-9bd9-8e0780a40550" containerName="online" May 26 15:44:46.527667 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.527236 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="7548eb6f-39bf-4632-9bd9-8e0780a40550" containerName="online" May 26 15:44:46.527667 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.527248 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7548eb6f-39bf-4632-9bd9-8e0780a40550" containerName="offline" May 26 15:44:46.527667 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.527258 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="7548eb6f-39bf-4632-9bd9-8e0780a40550" containerName="offline" May 26 15:44:46.527667 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.527271 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce2fb726-f446-4bce-9eb8-072f241d3e7d" containerName="online" May 26 15:44:46.527667 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.527280 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce2fb726-f446-4bce-9eb8-072f241d3e7d" containerName="online" May 26 15:44:46.527667 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.527294 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce2fb726-f446-4bce-9eb8-072f241d3e7d" containerName="feast-apply" May 26 15:44:46.527667 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.527303 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce2fb726-f446-4bce-9eb8-072f241d3e7d" containerName="feast-apply" May 26 15:44:46.527667 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.527311 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7548eb6f-39bf-4632-9bd9-8e0780a40550" containerName="feast-init" May 26 15:44:46.527667 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.527321 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="7548eb6f-39bf-4632-9bd9-8e0780a40550" containerName="feast-init" May 26 15:44:46.527667 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.527329 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7548eb6f-39bf-4632-9bd9-8e0780a40550" containerName="registry" May 26 15:44:46.527667 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.527337 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="7548eb6f-39bf-4632-9bd9-8e0780a40550" containerName="registry" May 26 15:44:46.527667 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.527415 2547 memory_manager.go:356] "RemoveStaleState removing state" podUID="ce2fb726-f446-4bce-9eb8-072f241d3e7d" containerName="online" May 26 15:44:46.527667 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.527427 2547 memory_manager.go:356] "RemoveStaleState removing state" podUID="7548eb6f-39bf-4632-9bd9-8e0780a40550" containerName="registry" May 26 15:44:46.527667 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.527438 2547 memory_manager.go:356] "RemoveStaleState removing state" podUID="7548eb6f-39bf-4632-9bd9-8e0780a40550" containerName="offline" May 26 15:44:46.527667 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.527446 2547 memory_manager.go:356] "RemoveStaleState removing state" podUID="ce2fb726-f446-4bce-9eb8-072f241d3e7d" containerName="offline" May 26 15:44:46.527667 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.527456 2547 memory_manager.go:356] "RemoveStaleState removing state" podUID="ce2fb726-f446-4bce-9eb8-072f241d3e7d" containerName="ui" May 26 15:44:46.527667 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.527466 2547 memory_manager.go:356] "RemoveStaleState removing state" podUID="620ffd1c-3511-4e5f-8c5f-19ec9f3aaaa8" containerName="manager" May 26 15:44:46.527667 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.527474 2547 memory_manager.go:356] "RemoveStaleState removing state" podUID="7548eb6f-39bf-4632-9bd9-8e0780a40550" containerName="ui" May 26 15:44:46.527667 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.527484 2547 memory_manager.go:356] "RemoveStaleState removing state" podUID="7548eb6f-39bf-4632-9bd9-8e0780a40550" containerName="online" May 26 15:44:46.530364 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.530345 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-7w8xg" May 26 15:44:46.532625 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.532599 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" May 26 15:44:46.532731 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.532607 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-vqksw\"" May 26 15:44:46.532731 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.532609 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" May 26 15:44:46.536124 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.536101 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-cf99db75-7w8xg"] May 26 15:44:46.615634 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.615613 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpjts\" (UniqueName: \"kubernetes.io/projected/3a8f3905-86d6-467c-90fb-31185223b991-kube-api-access-jpjts\") pod \"feast-operator-controller-manager-cf99db75-7w8xg\" (UID: \"3a8f3905-86d6-467c-90fb-31185223b991\") " pod="feast-operator-system/feast-operator-controller-manager-cf99db75-7w8xg" May 26 15:44:46.716473 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.716438 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jpjts\" (UniqueName: \"kubernetes.io/projected/3a8f3905-86d6-467c-90fb-31185223b991-kube-api-access-jpjts\") pod \"feast-operator-controller-manager-cf99db75-7w8xg\" (UID: \"3a8f3905-86d6-467c-90fb-31185223b991\") " pod="feast-operator-system/feast-operator-controller-manager-cf99db75-7w8xg" May 26 15:44:46.726181 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.726143 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpjts\" (UniqueName: \"kubernetes.io/projected/3a8f3905-86d6-467c-90fb-31185223b991-kube-api-access-jpjts\") pod \"feast-operator-controller-manager-cf99db75-7w8xg\" (UID: \"3a8f3905-86d6-467c-90fb-31185223b991\") " pod="feast-operator-system/feast-operator-controller-manager-cf99db75-7w8xg" May 26 15:44:46.841392 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.841310 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-7w8xg" May 26 15:44:46.966435 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.966410 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-cf99db75-7w8xg"] May 26 15:44:46.968824 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:44:46.968794 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a8f3905_86d6_467c_90fb_31185223b991.slice/crio-054eff4c27058bdebf40ce79ab2b130c25bceb984e07426c1f097ed9df6e8ade WatchSource:0}: Error finding container 054eff4c27058bdebf40ce79ab2b130c25bceb984e07426c1f097ed9df6e8ade: Status 404 returned error can't find the container with id 054eff4c27058bdebf40ce79ab2b130c25bceb984e07426c1f097ed9df6e8ade May 26 15:44:46.971016 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:46.970991 2547 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 26 15:44:47.136845 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:47.136755 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-7w8xg" event={"ID":"3a8f3905-86d6-467c-90fb-31185223b991","Type":"ContainerStarted","Data":"b1e9dcdca33ea70a673241a844ac9e74a909d6c3bfabc983670b5c4fb844eaee"} May 26 15:44:47.136845 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:47.136795 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-7w8xg" event={"ID":"3a8f3905-86d6-467c-90fb-31185223b991","Type":"ContainerStarted","Data":"054eff4c27058bdebf40ce79ab2b130c25bceb984e07426c1f097ed9df6e8ade"} May 26 15:44:47.136845 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:47.136832 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-7w8xg" May 26 15:44:47.153277 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:47.153231 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-7w8xg" podStartSLOduration=1.153219453 podStartE2EDuration="1.153219453s" podCreationTimestamp="2026-05-26 15:44:46 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 15:44:47.151048622 +0000 UTC m=+1435.877723543" watchObservedRunningTime="2026-05-26 15:44:47.153219453 +0000 UTC m=+1435.879894376" May 26 15:44:58.141704 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:44:58.141667 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-7w8xg" May 26 15:45:03.440126 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:03.440090 2547 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2"] May 26 15:45:03.445517 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:03.445496 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" May 26 15:45:03.447597 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:03.447312 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" May 26 15:45:03.447597 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:03.447373 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-djjvw\"" May 26 15:45:03.447597 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:03.447420 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" May 26 15:45:03.447597 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:03.447480 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" May 26 15:45:03.447597 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:03.447421 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" May 26 15:45:03.447942 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:03.447713 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" May 26 15:45:03.447942 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:03.447755 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" May 26 15:45:03.454886 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:03.454864 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2"] May 26 15:45:03.537236 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:03.537203 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ac6e1e9f-b11d-4382-a264-17e5ff81e84b-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-xt8r2\" (UID: \"ac6e1e9f-b11d-4382-a264-17e5ff81e84b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" May 26 15:45:03.537400 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:03.537255 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfcmv\" (UniqueName: \"kubernetes.io/projected/ac6e1e9f-b11d-4382-a264-17e5ff81e84b-kube-api-access-mfcmv\") pod \"feast-simple-feast-setup-664fcfcc58-xt8r2\" (UID: \"ac6e1e9f-b11d-4382-a264-17e5ff81e84b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" May 26 15:45:03.537400 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:03.537324 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ac6e1e9f-b11d-4382-a264-17e5ff81e84b-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-xt8r2\" (UID: \"ac6e1e9f-b11d-4382-a264-17e5ff81e84b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" May 26 15:45:03.537400 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:03.537386 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ac6e1e9f-b11d-4382-a264-17e5ff81e84b-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-xt8r2\" (UID: \"ac6e1e9f-b11d-4382-a264-17e5ff81e84b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" May 26 15:45:03.537557 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:03.537418 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ac6e1e9f-b11d-4382-a264-17e5ff81e84b-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-xt8r2\" (UID: \"ac6e1e9f-b11d-4382-a264-17e5ff81e84b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" May 26 15:45:03.537557 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:03.537435 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ac6e1e9f-b11d-4382-a264-17e5ff81e84b-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-xt8r2\" (UID: \"ac6e1e9f-b11d-4382-a264-17e5ff81e84b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" May 26 15:45:03.638251 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:03.638220 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ac6e1e9f-b11d-4382-a264-17e5ff81e84b-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-xt8r2\" (UID: \"ac6e1e9f-b11d-4382-a264-17e5ff81e84b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" May 26 15:45:03.638251 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:03.638251 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ac6e1e9f-b11d-4382-a264-17e5ff81e84b-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-xt8r2\" (UID: \"ac6e1e9f-b11d-4382-a264-17e5ff81e84b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" May 26 15:45:03.638439 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:03.638271 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ac6e1e9f-b11d-4382-a264-17e5ff81e84b-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-xt8r2\" (UID: \"ac6e1e9f-b11d-4382-a264-17e5ff81e84b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" May 26 15:45:03.638439 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:03.638395 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ac6e1e9f-b11d-4382-a264-17e5ff81e84b-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-xt8r2\" (UID: \"ac6e1e9f-b11d-4382-a264-17e5ff81e84b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" May 26 15:45:03.638636 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:03.638439 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mfcmv\" (UniqueName: \"kubernetes.io/projected/ac6e1e9f-b11d-4382-a264-17e5ff81e84b-kube-api-access-mfcmv\") pod \"feast-simple-feast-setup-664fcfcc58-xt8r2\" (UID: \"ac6e1e9f-b11d-4382-a264-17e5ff81e84b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" May 26 15:45:03.638636 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:03.638480 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ac6e1e9f-b11d-4382-a264-17e5ff81e84b-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-xt8r2\" (UID: \"ac6e1e9f-b11d-4382-a264-17e5ff81e84b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" May 26 15:45:03.638878 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:03.638853 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ac6e1e9f-b11d-4382-a264-17e5ff81e84b-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-xt8r2\" (UID: \"ac6e1e9f-b11d-4382-a264-17e5ff81e84b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" May 26 15:45:03.640772 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:03.640756 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ac6e1e9f-b11d-4382-a264-17e5ff81e84b-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-xt8r2\" (UID: \"ac6e1e9f-b11d-4382-a264-17e5ff81e84b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" May 26 15:45:03.640900 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:03.640879 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ac6e1e9f-b11d-4382-a264-17e5ff81e84b-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-xt8r2\" (UID: \"ac6e1e9f-b11d-4382-a264-17e5ff81e84b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" May 26 15:45:03.640962 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:03.640943 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ac6e1e9f-b11d-4382-a264-17e5ff81e84b-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-xt8r2\" (UID: \"ac6e1e9f-b11d-4382-a264-17e5ff81e84b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" May 26 15:45:03.641004 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:03.640943 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ac6e1e9f-b11d-4382-a264-17e5ff81e84b-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-xt8r2\" (UID: \"ac6e1e9f-b11d-4382-a264-17e5ff81e84b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" May 26 15:45:03.647164 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:03.647145 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfcmv\" (UniqueName: \"kubernetes.io/projected/ac6e1e9f-b11d-4382-a264-17e5ff81e84b-kube-api-access-mfcmv\") pod \"feast-simple-feast-setup-664fcfcc58-xt8r2\" (UID: \"ac6e1e9f-b11d-4382-a264-17e5ff81e84b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" May 26 15:45:03.767369 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:03.767298 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" May 26 15:45:03.897312 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:03.897276 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2"] May 26 15:45:03.900213 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:45:03.900190 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podac6e1e9f_b11d_4382_a264_17e5ff81e84b.slice/crio-83e8e9db091bf4700f8ceea53ed4a8093fe7f581ff4e8200b0439f3155f906dc WatchSource:0}: Error finding container 83e8e9db091bf4700f8ceea53ed4a8093fe7f581ff4e8200b0439f3155f906dc: Status 404 returned error can't find the container with id 83e8e9db091bf4700f8ceea53ed4a8093fe7f581ff4e8200b0439f3155f906dc May 26 15:45:04.189344 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:04.188619 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" event={"ID":"ac6e1e9f-b11d-4382-a264-17e5ff81e84b","Type":"ContainerStarted","Data":"aa58cd5297f269ea69c74dd04960c1ea771b9f5d36bf7fd50bd83c0dc5127341"} May 26 15:45:04.189344 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:04.188661 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" event={"ID":"ac6e1e9f-b11d-4382-a264-17e5ff81e84b","Type":"ContainerStarted","Data":"83e8e9db091bf4700f8ceea53ed4a8093fe7f581ff4e8200b0439f3155f906dc"} May 26 15:45:09.204519 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:09.204485 2547 generic.go:358] "Generic (PLEG): container finished" podID="ac6e1e9f-b11d-4382-a264-17e5ff81e84b" containerID="aa58cd5297f269ea69c74dd04960c1ea771b9f5d36bf7fd50bd83c0dc5127341" exitCode=0 May 26 15:45:09.204875 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:09.204565 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" event={"ID":"ac6e1e9f-b11d-4382-a264-17e5ff81e84b","Type":"ContainerDied","Data":"aa58cd5297f269ea69c74dd04960c1ea771b9f5d36bf7fd50bd83c0dc5127341"} May 26 15:45:10.208908 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:10.208871 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" event={"ID":"ac6e1e9f-b11d-4382-a264-17e5ff81e84b","Type":"ContainerStarted","Data":"78293cbad3dff8bf8ede5d94ec9455826cec83f3165318ffa2cb470bb3068b4e"} May 26 15:45:15.224536 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:15.224498 2547 generic.go:358] "Generic (PLEG): container finished" podID="ac6e1e9f-b11d-4382-a264-17e5ff81e84b" containerID="78293cbad3dff8bf8ede5d94ec9455826cec83f3165318ffa2cb470bb3068b4e" exitCode=0 May 26 15:45:15.224896 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:15.224552 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" event={"ID":"ac6e1e9f-b11d-4382-a264-17e5ff81e84b","Type":"ContainerDied","Data":"78293cbad3dff8bf8ede5d94ec9455826cec83f3165318ffa2cb470bb3068b4e"} May 26 15:45:16.232106 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:16.232041 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" event={"ID":"ac6e1e9f-b11d-4382-a264-17e5ff81e84b","Type":"ContainerStarted","Data":"13e77ea2b92273539db63645b7ca7ca4c1613b7919594ed79bf85a7b77469201"} May 26 15:45:16.232587 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:16.232115 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" event={"ID":"ac6e1e9f-b11d-4382-a264-17e5ff81e84b","Type":"ContainerStarted","Data":"38d31b3a2a8097e3558cf799a9ce5ec63cf9b9b10bfa67b472c03b82a54331a5"} May 26 15:45:16.232587 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:16.232130 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" event={"ID":"ac6e1e9f-b11d-4382-a264-17e5ff81e84b","Type":"ContainerStarted","Data":"7817b708d480d307896e2b8f7d94f0d6250ca66392cb2ec4ee7597a18bf369b7"} May 26 15:45:16.232587 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:16.232144 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" event={"ID":"ac6e1e9f-b11d-4382-a264-17e5ff81e84b","Type":"ContainerStarted","Data":"d313025ab27ac54fcff49f2dfafc70f09c82cec7c7630465cf735f5dade50626"} May 26 15:45:16.255644 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:16.255585 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" podStartSLOduration=13.255565626 podStartE2EDuration="13.255565626s" podCreationTimestamp="2026-05-26 15:45:03 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 15:45:16.253616244 +0000 UTC m=+1464.980291168" watchObservedRunningTime="2026-05-26 15:45:16.255565626 +0000 UTC m=+1464.982240550" May 26 15:45:18.768308 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:18.768262 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" May 26 15:45:18.768308 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:18.768314 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" May 26 15:45:18.768829 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:18.768331 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" May 26 15:45:18.768829 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:18.768343 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" May 26 15:45:18.770739 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:18.770701 2547 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" podUID="ac6e1e9f-b11d-4382-a264-17e5ff81e84b" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.35:8443: connect: connection refused" May 26 15:45:18.770849 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:18.770772 2547 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" podUID="ac6e1e9f-b11d-4382-a264-17e5ff81e84b" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.35:6571: connect: connection refused" May 26 15:45:18.770849 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:18.770793 2547 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" podUID="ac6e1e9f-b11d-4382-a264-17e5ff81e84b" containerName="online" probeResult="failure" output="Get \"https://10.134.0.35:6567/health\": dial tcp 10.134.0.35:6567: connect: connection refused" May 26 15:45:18.770972 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:18.770855 2547 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" podUID="ac6e1e9f-b11d-4382-a264-17e5ff81e84b" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.35:8816: connect: connection refused" May 26 15:45:21.768432 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:21.768395 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" May 26 15:45:21.768826 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:21.768568 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" May 26 15:45:21.768917 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:21.768896 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" May 26 15:45:21.769293 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:21.769266 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" May 26 15:45:21.769414 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:21.769304 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" May 26 15:45:21.769414 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:21.769318 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" May 26 15:45:21.769414 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:21.769381 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" May 26 15:45:21.769520 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:21.769422 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" May 26 15:45:21.772968 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:21.772952 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" May 26 15:45:22.252878 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:22.252842 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" May 26 15:45:22.253119 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:22.253099 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" May 26 15:45:22.255822 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:22.255803 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" May 26 15:45:51.893675 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:51.893644 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-qd2tx_dd56d3c2-38ff-4bd8-9893-6b643455a976/console-operator/2.log" May 26 15:45:51.897313 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:45:51.897289 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-qd2tx_dd56d3c2-38ff-4bd8-9893-6b643455a976/console-operator/2.log" May 26 15:46:24.513713 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:24.513622 2547 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2"] May 26 15:46:24.514234 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:24.514104 2547 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" podUID="ac6e1e9f-b11d-4382-a264-17e5ff81e84b" containerName="registry" containerID="cri-o://d313025ab27ac54fcff49f2dfafc70f09c82cec7c7630465cf735f5dade50626" gracePeriod=30 May 26 15:46:24.514322 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:24.514254 2547 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" podUID="ac6e1e9f-b11d-4382-a264-17e5ff81e84b" containerName="offline" containerID="cri-o://38d31b3a2a8097e3558cf799a9ce5ec63cf9b9b10bfa67b472c03b82a54331a5" gracePeriod=30 May 26 15:46:24.514322 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:24.514242 2547 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" podUID="ac6e1e9f-b11d-4382-a264-17e5ff81e84b" containerName="ui" containerID="cri-o://13e77ea2b92273539db63645b7ca7ca4c1613b7919594ed79bf85a7b77469201" gracePeriod=30 May 26 15:46:24.514422 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:24.514359 2547 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" podUID="ac6e1e9f-b11d-4382-a264-17e5ff81e84b" containerName="online" containerID="cri-o://7817b708d480d307896e2b8f7d94f0d6250ca66392cb2ec4ee7597a18bf369b7" gracePeriod=30 May 26 15:46:24.900646 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:24.900561 2547 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96"] May 26 15:46:24.905423 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:24.905398 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:46:24.908008 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:24.907976 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-vmhgb\"" May 26 15:46:24.934086 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:24.934042 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96"] May 26 15:46:24.988832 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:24.988787 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgkj8\" (UniqueName: \"kubernetes.io/projected/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-kube-api-access-xgkj8\") pod \"feast-simple-feast-setup-664fcfcc58-g2x96\" (UID: \"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:46:24.989017 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:24.988845 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-g2x96\" (UID: \"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:46:24.989017 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:24.988924 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-g2x96\" (UID: \"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:46:24.989017 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:24.988968 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-g2x96\" (UID: \"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:46:24.989223 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:24.989016 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-g2x96\" (UID: \"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:46:24.989223 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:24.989084 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-g2x96\" (UID: \"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:46:25.090083 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:25.090030 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-g2x96\" (UID: \"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:46:25.090288 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:25.090104 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xgkj8\" (UniqueName: \"kubernetes.io/projected/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-kube-api-access-xgkj8\") pod \"feast-simple-feast-setup-664fcfcc58-g2x96\" (UID: \"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:46:25.090288 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:25.090141 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-g2x96\" (UID: \"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:46:25.090288 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:25.090174 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-g2x96\" (UID: \"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:46:25.090288 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:25.090199 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-g2x96\" (UID: \"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:46:25.090288 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:25.090242 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-g2x96\" (UID: \"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:46:25.092312 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:46:25.090932 2547 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found May 26 15:46:25.092312 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:46:25.091038 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-registry-tls podName:5bca7cd5-f29c-44fb-ba0e-a4ab0268051d nodeName:}" failed. No retries permitted until 2026-05-26 15:46:25.591016007 +0000 UTC m=+1534.317690919 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-registry-tls") pod "feast-simple-feast-setup-664fcfcc58-g2x96" (UID: "5bca7cd5-f29c-44fb-ba0e-a4ab0268051d") : secret "feast-simple-feast-setup-registry-tls" not found May 26 15:46:25.092312 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:46:25.091171 2547 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found May 26 15:46:25.092312 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:25.091228 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-g2x96\" (UID: \"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:46:25.092312 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:46:25.091242 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-ui-tls podName:5bca7cd5-f29c-44fb-ba0e-a4ab0268051d nodeName:}" failed. No retries permitted until 2026-05-26 15:46:25.591222714 +0000 UTC m=+1534.317897631 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-ui-tls") pod "feast-simple-feast-setup-664fcfcc58-g2x96" (UID: "5bca7cd5-f29c-44fb-ba0e-a4ab0268051d") : secret "feast-simple-feast-setup-ui-tls" not found May 26 15:46:25.092312 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:46:25.091320 2547 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found May 26 15:46:25.092312 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:46:25.091368 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-online-tls podName:5bca7cd5-f29c-44fb-ba0e-a4ab0268051d nodeName:}" failed. No retries permitted until 2026-05-26 15:46:25.591356563 +0000 UTC m=+1534.318031477 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-online-tls") pod "feast-simple-feast-setup-664fcfcc58-g2x96" (UID: "5bca7cd5-f29c-44fb-ba0e-a4ab0268051d") : secret "feast-simple-feast-setup-online-tls" not found May 26 15:46:25.102728 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:25.102518 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgkj8\" (UniqueName: \"kubernetes.io/projected/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-kube-api-access-xgkj8\") pod \"feast-simple-feast-setup-664fcfcc58-g2x96\" (UID: \"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:46:25.102996 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:25.102971 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-g2x96\" (UID: \"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:46:25.428634 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:25.428598 2547 generic.go:358] "Generic (PLEG): container finished" podID="ac6e1e9f-b11d-4382-a264-17e5ff81e84b" containerID="13e77ea2b92273539db63645b7ca7ca4c1613b7919594ed79bf85a7b77469201" exitCode=0 May 26 15:46:25.428634 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:25.428624 2547 generic.go:358] "Generic (PLEG): container finished" podID="ac6e1e9f-b11d-4382-a264-17e5ff81e84b" containerID="7817b708d480d307896e2b8f7d94f0d6250ca66392cb2ec4ee7597a18bf369b7" exitCode=0 May 26 15:46:25.428842 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:25.428675 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" event={"ID":"ac6e1e9f-b11d-4382-a264-17e5ff81e84b","Type":"ContainerDied","Data":"13e77ea2b92273539db63645b7ca7ca4c1613b7919594ed79bf85a7b77469201"} May 26 15:46:25.428842 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:25.428722 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" event={"ID":"ac6e1e9f-b11d-4382-a264-17e5ff81e84b","Type":"ContainerDied","Data":"7817b708d480d307896e2b8f7d94f0d6250ca66392cb2ec4ee7597a18bf369b7"} May 26 15:46:25.594202 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:25.594169 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-g2x96\" (UID: \"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:46:25.594202 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:25.594208 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-g2x96\" (UID: \"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:46:25.594716 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:25.594231 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-g2x96\" (UID: \"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:46:25.596615 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:25.596583 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-g2x96\" (UID: \"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:46:25.596737 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:25.596666 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-g2x96\" (UID: \"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:46:25.596737 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:25.596717 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-g2x96\" (UID: \"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:46:25.818051 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:25.817977 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:46:25.943126 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:25.943025 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96"] May 26 15:46:25.945259 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:46:25.945233 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5bca7cd5_f29c_44fb_ba0e_a4ab0268051d.slice/crio-fc975233db16f3591bad49b1f4c26bb58f8464327f8ee25e364f376d159225be WatchSource:0}: Error finding container fc975233db16f3591bad49b1f4c26bb58f8464327f8ee25e364f376d159225be: Status 404 returned error can't find the container with id fc975233db16f3591bad49b1f4c26bb58f8464327f8ee25e364f376d159225be May 26 15:46:26.433700 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:26.433660 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" event={"ID":"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d","Type":"ContainerStarted","Data":"3e063ecf3a67e550619cdef234f7b613646c4cd3c0983b9d79376decc5801f07"} May 26 15:46:26.433700 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:26.433697 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" event={"ID":"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d","Type":"ContainerStarted","Data":"fc975233db16f3591bad49b1f4c26bb58f8464327f8ee25e364f376d159225be"} May 26 15:46:31.450107 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:31.450047 2547 generic.go:358] "Generic (PLEG): container finished" podID="5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" containerID="3e063ecf3a67e550619cdef234f7b613646c4cd3c0983b9d79376decc5801f07" exitCode=0 May 26 15:46:31.450513 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:31.450125 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" event={"ID":"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d","Type":"ContainerDied","Data":"3e063ecf3a67e550619cdef234f7b613646c4cd3c0983b9d79376decc5801f07"} May 26 15:46:31.769331 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:31.769242 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" podUID="ac6e1e9f-b11d-4382-a264-17e5ff81e84b" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.35:8443: connect: connection refused" May 26 15:46:32.253143 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:32.253102 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" podUID="ac6e1e9f-b11d-4382-a264-17e5ff81e84b" containerName="online" probeResult="failure" output="Get \"https://10.134.0.35:6567/health\": dial tcp 10.134.0.35:6567: connect: connection refused" May 26 15:46:32.454242 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:32.454205 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" event={"ID":"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d","Type":"ContainerStarted","Data":"ca6b0629cca9577652e34cd366e7f8c01e85e3c05024466c8aa2422e7cf125e0"} May 26 15:46:37.470041 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:37.470001 2547 generic.go:358] "Generic (PLEG): container finished" podID="5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" containerID="ca6b0629cca9577652e34cd366e7f8c01e85e3c05024466c8aa2422e7cf125e0" exitCode=0 May 26 15:46:37.470442 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:37.470088 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" event={"ID":"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d","Type":"ContainerDied","Data":"ca6b0629cca9577652e34cd366e7f8c01e85e3c05024466c8aa2422e7cf125e0"} May 26 15:46:38.478184 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:38.478128 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" event={"ID":"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d","Type":"ContainerStarted","Data":"bb77927413d6f15fd52063433d19be435be78e3fcf44ef46276ce687cc334c44"} May 26 15:46:38.478666 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:38.478192 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" event={"ID":"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d","Type":"ContainerStarted","Data":"087a0794daf43a6309b7818411bbd03ee8dfd341668aa2b5d348dbbafb00d32c"} May 26 15:46:38.478666 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:38.478213 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" event={"ID":"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d","Type":"ContainerStarted","Data":"072cfe9e4fd5b6cbccd1e3d4d82579423d896c3effaf0425d33e11ca208cc9a4"} May 26 15:46:38.478666 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:38.478226 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" event={"ID":"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d","Type":"ContainerStarted","Data":"a32f2d21fb5cc6a4a46e5ead67e84a228bddc13c8e32fb53b30c83e4dd048e09"} May 26 15:46:38.501796 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:38.501732 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" podStartSLOduration=14.501712572 podStartE2EDuration="14.501712572s" podCreationTimestamp="2026-05-26 15:46:24 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 15:46:38.500043354 +0000 UTC m=+1547.226718278" watchObservedRunningTime="2026-05-26 15:46:38.501712572 +0000 UTC m=+1547.228387494" May 26 15:46:40.819849 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:40.819107 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:46:40.819849 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:40.819169 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:46:40.819849 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:40.819186 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:46:40.819849 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:40.819316 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:46:40.822596 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:40.821650 2547 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" podUID="5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.36:6571: connect: connection refused" May 26 15:46:40.822596 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:40.821670 2547 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" podUID="5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.36:8443: connect: connection refused" May 26 15:46:40.822596 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:40.821707 2547 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" podUID="5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.36:6567/health\": dial tcp 10.134.0.36:6567: connect: connection refused" May 26 15:46:40.822596 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:40.821796 2547 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" podUID="5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.36:8816: connect: connection refused" May 26 15:46:41.769897 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:41.769851 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" podUID="ac6e1e9f-b11d-4382-a264-17e5ff81e84b" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.35:8443: connect: connection refused" May 26 15:46:42.253912 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:42.253841 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" podUID="ac6e1e9f-b11d-4382-a264-17e5ff81e84b" containerName="online" probeResult="failure" output="Get \"https://10.134.0.35:6567/health\": dial tcp 10.134.0.35:6567: connect: connection refused" May 26 15:46:43.819378 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:43.819344 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:46:43.819835 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:43.819470 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:46:43.819835 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:43.819765 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:46:43.820184 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:43.820159 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:46:43.820327 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:43.820193 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:46:43.820327 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:43.820206 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:46:43.820455 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:43.820368 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:46:43.820648 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:43.820625 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:46:43.820748 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:43.820733 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:46:43.824030 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:43.824010 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:46:44.500746 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:44.500717 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:46:44.503588 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:44.503569 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:46:51.769763 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:51.769725 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" podUID="ac6e1e9f-b11d-4382-a264-17e5ff81e84b" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.35:8443: connect: connection refused" May 26 15:46:51.770145 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:51.769836 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" May 26 15:46:52.253572 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:52.253530 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" podUID="ac6e1e9f-b11d-4382-a264-17e5ff81e84b" containerName="online" probeResult="failure" output="Get \"https://10.134.0.35:6567/health\": dial tcp 10.134.0.35:6567: connect: connection refused" May 26 15:46:52.253773 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:52.253657 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" May 26 15:46:55.155275 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.155247 2547 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" May 26 15:46:55.253521 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.253489 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ac6e1e9f-b11d-4382-a264-17e5ff81e84b-online-tls\") pod \"ac6e1e9f-b11d-4382-a264-17e5ff81e84b\" (UID: \"ac6e1e9f-b11d-4382-a264-17e5ff81e84b\") " May 26 15:46:55.253670 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.253528 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ac6e1e9f-b11d-4382-a264-17e5ff81e84b-registry-tls\") pod \"ac6e1e9f-b11d-4382-a264-17e5ff81e84b\" (UID: \"ac6e1e9f-b11d-4382-a264-17e5ff81e84b\") " May 26 15:46:55.253670 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.253546 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ac6e1e9f-b11d-4382-a264-17e5ff81e84b-ui-tls\") pod \"ac6e1e9f-b11d-4382-a264-17e5ff81e84b\" (UID: \"ac6e1e9f-b11d-4382-a264-17e5ff81e84b\") " May 26 15:46:55.253670 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.253590 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mfcmv\" (UniqueName: \"kubernetes.io/projected/ac6e1e9f-b11d-4382-a264-17e5ff81e84b-kube-api-access-mfcmv\") pod \"ac6e1e9f-b11d-4382-a264-17e5ff81e84b\" (UID: \"ac6e1e9f-b11d-4382-a264-17e5ff81e84b\") " May 26 15:46:55.253670 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.253611 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ac6e1e9f-b11d-4382-a264-17e5ff81e84b-offline-tls\") pod \"ac6e1e9f-b11d-4382-a264-17e5ff81e84b\" (UID: \"ac6e1e9f-b11d-4382-a264-17e5ff81e84b\") " May 26 15:46:55.253670 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.253641 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ac6e1e9f-b11d-4382-a264-17e5ff81e84b-feast-data\") pod \"ac6e1e9f-b11d-4382-a264-17e5ff81e84b\" (UID: \"ac6e1e9f-b11d-4382-a264-17e5ff81e84b\") " May 26 15:46:55.254421 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.254357 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac6e1e9f-b11d-4382-a264-17e5ff81e84b-feast-data" (OuterVolumeSpecName: "feast-data") pod "ac6e1e9f-b11d-4382-a264-17e5ff81e84b" (UID: "ac6e1e9f-b11d-4382-a264-17e5ff81e84b"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 26 15:46:55.255849 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.255819 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac6e1e9f-b11d-4382-a264-17e5ff81e84b-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "ac6e1e9f-b11d-4382-a264-17e5ff81e84b" (UID: "ac6e1e9f-b11d-4382-a264-17e5ff81e84b"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 26 15:46:55.255849 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.255831 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac6e1e9f-b11d-4382-a264-17e5ff81e84b-online-tls" (OuterVolumeSpecName: "online-tls") pod "ac6e1e9f-b11d-4382-a264-17e5ff81e84b" (UID: "ac6e1e9f-b11d-4382-a264-17e5ff81e84b"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 26 15:46:55.256019 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.255887 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac6e1e9f-b11d-4382-a264-17e5ff81e84b-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "ac6e1e9f-b11d-4382-a264-17e5ff81e84b" (UID: "ac6e1e9f-b11d-4382-a264-17e5ff81e84b"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 26 15:46:55.256155 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.256140 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac6e1e9f-b11d-4382-a264-17e5ff81e84b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ac6e1e9f-b11d-4382-a264-17e5ff81e84b" (UID: "ac6e1e9f-b11d-4382-a264-17e5ff81e84b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 26 15:46:55.256218 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.256203 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac6e1e9f-b11d-4382-a264-17e5ff81e84b-kube-api-access-mfcmv" (OuterVolumeSpecName: "kube-api-access-mfcmv") pod "ac6e1e9f-b11d-4382-a264-17e5ff81e84b" (UID: "ac6e1e9f-b11d-4382-a264-17e5ff81e84b"). InnerVolumeSpecName "kube-api-access-mfcmv". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 26 15:46:55.354974 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.354906 2547 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ac6e1e9f-b11d-4382-a264-17e5ff81e84b-feast-data\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:46:55.354974 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.354929 2547 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ac6e1e9f-b11d-4382-a264-17e5ff81e84b-online-tls\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:46:55.354974 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.354938 2547 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ac6e1e9f-b11d-4382-a264-17e5ff81e84b-registry-tls\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:46:55.354974 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.354946 2547 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ac6e1e9f-b11d-4382-a264-17e5ff81e84b-ui-tls\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:46:55.354974 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.354955 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mfcmv\" (UniqueName: \"kubernetes.io/projected/ac6e1e9f-b11d-4382-a264-17e5ff81e84b-kube-api-access-mfcmv\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:46:55.354974 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.354964 2547 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ac6e1e9f-b11d-4382-a264-17e5ff81e84b-offline-tls\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:46:55.538445 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.538413 2547 generic.go:358] "Generic (PLEG): container finished" podID="ac6e1e9f-b11d-4382-a264-17e5ff81e84b" containerID="38d31b3a2a8097e3558cf799a9ce5ec63cf9b9b10bfa67b472c03b82a54331a5" exitCode=137 May 26 15:46:55.538445 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.538439 2547 generic.go:358] "Generic (PLEG): container finished" podID="ac6e1e9f-b11d-4382-a264-17e5ff81e84b" containerID="d313025ab27ac54fcff49f2dfafc70f09c82cec7c7630465cf735f5dade50626" exitCode=137 May 26 15:46:55.538682 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.538486 2547 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" May 26 15:46:55.538682 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.538487 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" event={"ID":"ac6e1e9f-b11d-4382-a264-17e5ff81e84b","Type":"ContainerDied","Data":"38d31b3a2a8097e3558cf799a9ce5ec63cf9b9b10bfa67b472c03b82a54331a5"} May 26 15:46:55.538682 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.538594 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" event={"ID":"ac6e1e9f-b11d-4382-a264-17e5ff81e84b","Type":"ContainerDied","Data":"d313025ab27ac54fcff49f2dfafc70f09c82cec7c7630465cf735f5dade50626"} May 26 15:46:55.538682 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.538605 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2" event={"ID":"ac6e1e9f-b11d-4382-a264-17e5ff81e84b","Type":"ContainerDied","Data":"83e8e9db091bf4700f8ceea53ed4a8093fe7f581ff4e8200b0439f3155f906dc"} May 26 15:46:55.538682 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.538619 2547 scope.go:117] "RemoveContainer" containerID="13e77ea2b92273539db63645b7ca7ca4c1613b7919594ed79bf85a7b77469201" May 26 15:46:55.546971 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.546954 2547 scope.go:117] "RemoveContainer" containerID="38d31b3a2a8097e3558cf799a9ce5ec63cf9b9b10bfa67b472c03b82a54331a5" May 26 15:46:55.556125 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.556103 2547 scope.go:117] "RemoveContainer" containerID="7817b708d480d307896e2b8f7d94f0d6250ca66392cb2ec4ee7597a18bf369b7" May 26 15:46:55.567235 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.564499 2547 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2"] May 26 15:46:55.568173 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.568150 2547 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-xt8r2"] May 26 15:46:55.571902 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.571881 2547 scope.go:117] "RemoveContainer" containerID="d313025ab27ac54fcff49f2dfafc70f09c82cec7c7630465cf735f5dade50626" May 26 15:46:55.579564 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.579548 2547 scope.go:117] "RemoveContainer" containerID="78293cbad3dff8bf8ede5d94ec9455826cec83f3165318ffa2cb470bb3068b4e" May 26 15:46:55.586950 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.586929 2547 scope.go:117] "RemoveContainer" containerID="aa58cd5297f269ea69c74dd04960c1ea771b9f5d36bf7fd50bd83c0dc5127341" May 26 15:46:55.595871 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.595854 2547 scope.go:117] "RemoveContainer" containerID="13e77ea2b92273539db63645b7ca7ca4c1613b7919594ed79bf85a7b77469201" May 26 15:46:55.596174 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:46:55.596146 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13e77ea2b92273539db63645b7ca7ca4c1613b7919594ed79bf85a7b77469201\": container with ID starting with 13e77ea2b92273539db63645b7ca7ca4c1613b7919594ed79bf85a7b77469201 not found: ID does not exist" containerID="13e77ea2b92273539db63645b7ca7ca4c1613b7919594ed79bf85a7b77469201" May 26 15:46:55.596247 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.596183 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13e77ea2b92273539db63645b7ca7ca4c1613b7919594ed79bf85a7b77469201"} err="failed to get container status \"13e77ea2b92273539db63645b7ca7ca4c1613b7919594ed79bf85a7b77469201\": rpc error: code = NotFound desc = could not find container \"13e77ea2b92273539db63645b7ca7ca4c1613b7919594ed79bf85a7b77469201\": container with ID starting with 13e77ea2b92273539db63645b7ca7ca4c1613b7919594ed79bf85a7b77469201 not found: ID does not exist" May 26 15:46:55.596247 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.596202 2547 scope.go:117] "RemoveContainer" containerID="38d31b3a2a8097e3558cf799a9ce5ec63cf9b9b10bfa67b472c03b82a54331a5" May 26 15:46:55.596444 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:46:55.596426 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38d31b3a2a8097e3558cf799a9ce5ec63cf9b9b10bfa67b472c03b82a54331a5\": container with ID starting with 38d31b3a2a8097e3558cf799a9ce5ec63cf9b9b10bfa67b472c03b82a54331a5 not found: ID does not exist" containerID="38d31b3a2a8097e3558cf799a9ce5ec63cf9b9b10bfa67b472c03b82a54331a5" May 26 15:46:55.596483 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.596449 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38d31b3a2a8097e3558cf799a9ce5ec63cf9b9b10bfa67b472c03b82a54331a5"} err="failed to get container status \"38d31b3a2a8097e3558cf799a9ce5ec63cf9b9b10bfa67b472c03b82a54331a5\": rpc error: code = NotFound desc = could not find container \"38d31b3a2a8097e3558cf799a9ce5ec63cf9b9b10bfa67b472c03b82a54331a5\": container with ID starting with 38d31b3a2a8097e3558cf799a9ce5ec63cf9b9b10bfa67b472c03b82a54331a5 not found: ID does not exist" May 26 15:46:55.596483 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.596466 2547 scope.go:117] "RemoveContainer" containerID="7817b708d480d307896e2b8f7d94f0d6250ca66392cb2ec4ee7597a18bf369b7" May 26 15:46:55.596714 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:46:55.596697 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7817b708d480d307896e2b8f7d94f0d6250ca66392cb2ec4ee7597a18bf369b7\": container with ID starting with 7817b708d480d307896e2b8f7d94f0d6250ca66392cb2ec4ee7597a18bf369b7 not found: ID does not exist" containerID="7817b708d480d307896e2b8f7d94f0d6250ca66392cb2ec4ee7597a18bf369b7" May 26 15:46:55.596757 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.596718 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7817b708d480d307896e2b8f7d94f0d6250ca66392cb2ec4ee7597a18bf369b7"} err="failed to get container status \"7817b708d480d307896e2b8f7d94f0d6250ca66392cb2ec4ee7597a18bf369b7\": rpc error: code = NotFound desc = could not find container \"7817b708d480d307896e2b8f7d94f0d6250ca66392cb2ec4ee7597a18bf369b7\": container with ID starting with 7817b708d480d307896e2b8f7d94f0d6250ca66392cb2ec4ee7597a18bf369b7 not found: ID does not exist" May 26 15:46:55.596757 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.596732 2547 scope.go:117] "RemoveContainer" containerID="d313025ab27ac54fcff49f2dfafc70f09c82cec7c7630465cf735f5dade50626" May 26 15:46:55.596961 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:46:55.596938 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d313025ab27ac54fcff49f2dfafc70f09c82cec7c7630465cf735f5dade50626\": container with ID starting with d313025ab27ac54fcff49f2dfafc70f09c82cec7c7630465cf735f5dade50626 not found: ID does not exist" containerID="d313025ab27ac54fcff49f2dfafc70f09c82cec7c7630465cf735f5dade50626" May 26 15:46:55.597010 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.596966 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d313025ab27ac54fcff49f2dfafc70f09c82cec7c7630465cf735f5dade50626"} err="failed to get container status \"d313025ab27ac54fcff49f2dfafc70f09c82cec7c7630465cf735f5dade50626\": rpc error: code = NotFound desc = could not find container \"d313025ab27ac54fcff49f2dfafc70f09c82cec7c7630465cf735f5dade50626\": container with ID starting with d313025ab27ac54fcff49f2dfafc70f09c82cec7c7630465cf735f5dade50626 not found: ID does not exist" May 26 15:46:55.597010 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.596979 2547 scope.go:117] "RemoveContainer" containerID="78293cbad3dff8bf8ede5d94ec9455826cec83f3165318ffa2cb470bb3068b4e" May 26 15:46:55.597252 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:46:55.597225 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78293cbad3dff8bf8ede5d94ec9455826cec83f3165318ffa2cb470bb3068b4e\": container with ID starting with 78293cbad3dff8bf8ede5d94ec9455826cec83f3165318ffa2cb470bb3068b4e not found: ID does not exist" containerID="78293cbad3dff8bf8ede5d94ec9455826cec83f3165318ffa2cb470bb3068b4e" May 26 15:46:55.597340 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.597257 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78293cbad3dff8bf8ede5d94ec9455826cec83f3165318ffa2cb470bb3068b4e"} err="failed to get container status \"78293cbad3dff8bf8ede5d94ec9455826cec83f3165318ffa2cb470bb3068b4e\": rpc error: code = NotFound desc = could not find container \"78293cbad3dff8bf8ede5d94ec9455826cec83f3165318ffa2cb470bb3068b4e\": container with ID starting with 78293cbad3dff8bf8ede5d94ec9455826cec83f3165318ffa2cb470bb3068b4e not found: ID does not exist" May 26 15:46:55.597340 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.597271 2547 scope.go:117] "RemoveContainer" containerID="aa58cd5297f269ea69c74dd04960c1ea771b9f5d36bf7fd50bd83c0dc5127341" May 26 15:46:55.597506 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:46:55.597480 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa58cd5297f269ea69c74dd04960c1ea771b9f5d36bf7fd50bd83c0dc5127341\": container with ID starting with aa58cd5297f269ea69c74dd04960c1ea771b9f5d36bf7fd50bd83c0dc5127341 not found: ID does not exist" containerID="aa58cd5297f269ea69c74dd04960c1ea771b9f5d36bf7fd50bd83c0dc5127341" May 26 15:46:55.597558 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.597509 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa58cd5297f269ea69c74dd04960c1ea771b9f5d36bf7fd50bd83c0dc5127341"} err="failed to get container status \"aa58cd5297f269ea69c74dd04960c1ea771b9f5d36bf7fd50bd83c0dc5127341\": rpc error: code = NotFound desc = could not find container \"aa58cd5297f269ea69c74dd04960c1ea771b9f5d36bf7fd50bd83c0dc5127341\": container with ID starting with aa58cd5297f269ea69c74dd04960c1ea771b9f5d36bf7fd50bd83c0dc5127341 not found: ID does not exist" May 26 15:46:55.597558 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.597522 2547 scope.go:117] "RemoveContainer" containerID="13e77ea2b92273539db63645b7ca7ca4c1613b7919594ed79bf85a7b77469201" May 26 15:46:55.597750 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.597727 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13e77ea2b92273539db63645b7ca7ca4c1613b7919594ed79bf85a7b77469201"} err="failed to get container status \"13e77ea2b92273539db63645b7ca7ca4c1613b7919594ed79bf85a7b77469201\": rpc error: code = NotFound desc = could not find container \"13e77ea2b92273539db63645b7ca7ca4c1613b7919594ed79bf85a7b77469201\": container with ID starting with 13e77ea2b92273539db63645b7ca7ca4c1613b7919594ed79bf85a7b77469201 not found: ID does not exist" May 26 15:46:55.597793 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.597753 2547 scope.go:117] "RemoveContainer" containerID="38d31b3a2a8097e3558cf799a9ce5ec63cf9b9b10bfa67b472c03b82a54331a5" May 26 15:46:55.598007 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.597985 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38d31b3a2a8097e3558cf799a9ce5ec63cf9b9b10bfa67b472c03b82a54331a5"} err="failed to get container status \"38d31b3a2a8097e3558cf799a9ce5ec63cf9b9b10bfa67b472c03b82a54331a5\": rpc error: code = NotFound desc = could not find container \"38d31b3a2a8097e3558cf799a9ce5ec63cf9b9b10bfa67b472c03b82a54331a5\": container with ID starting with 38d31b3a2a8097e3558cf799a9ce5ec63cf9b9b10bfa67b472c03b82a54331a5 not found: ID does not exist" May 26 15:46:55.598114 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.598010 2547 scope.go:117] "RemoveContainer" containerID="7817b708d480d307896e2b8f7d94f0d6250ca66392cb2ec4ee7597a18bf369b7" May 26 15:46:55.598273 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.598257 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7817b708d480d307896e2b8f7d94f0d6250ca66392cb2ec4ee7597a18bf369b7"} err="failed to get container status \"7817b708d480d307896e2b8f7d94f0d6250ca66392cb2ec4ee7597a18bf369b7\": rpc error: code = NotFound desc = could not find container \"7817b708d480d307896e2b8f7d94f0d6250ca66392cb2ec4ee7597a18bf369b7\": container with ID starting with 7817b708d480d307896e2b8f7d94f0d6250ca66392cb2ec4ee7597a18bf369b7 not found: ID does not exist" May 26 15:46:55.598318 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.598273 2547 scope.go:117] "RemoveContainer" containerID="d313025ab27ac54fcff49f2dfafc70f09c82cec7c7630465cf735f5dade50626" May 26 15:46:55.598480 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.598461 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d313025ab27ac54fcff49f2dfafc70f09c82cec7c7630465cf735f5dade50626"} err="failed to get container status \"d313025ab27ac54fcff49f2dfafc70f09c82cec7c7630465cf735f5dade50626\": rpc error: code = NotFound desc = could not find container \"d313025ab27ac54fcff49f2dfafc70f09c82cec7c7630465cf735f5dade50626\": container with ID starting with d313025ab27ac54fcff49f2dfafc70f09c82cec7c7630465cf735f5dade50626 not found: ID does not exist" May 26 15:46:55.598525 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.598481 2547 scope.go:117] "RemoveContainer" containerID="78293cbad3dff8bf8ede5d94ec9455826cec83f3165318ffa2cb470bb3068b4e" May 26 15:46:55.598678 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.598659 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78293cbad3dff8bf8ede5d94ec9455826cec83f3165318ffa2cb470bb3068b4e"} err="failed to get container status \"78293cbad3dff8bf8ede5d94ec9455826cec83f3165318ffa2cb470bb3068b4e\": rpc error: code = NotFound desc = could not find container \"78293cbad3dff8bf8ede5d94ec9455826cec83f3165318ffa2cb470bb3068b4e\": container with ID starting with 78293cbad3dff8bf8ede5d94ec9455826cec83f3165318ffa2cb470bb3068b4e not found: ID does not exist" May 26 15:46:55.598733 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.598679 2547 scope.go:117] "RemoveContainer" containerID="aa58cd5297f269ea69c74dd04960c1ea771b9f5d36bf7fd50bd83c0dc5127341" May 26 15:46:55.598884 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.598866 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa58cd5297f269ea69c74dd04960c1ea771b9f5d36bf7fd50bd83c0dc5127341"} err="failed to get container status \"aa58cd5297f269ea69c74dd04960c1ea771b9f5d36bf7fd50bd83c0dc5127341\": rpc error: code = NotFound desc = could not find container \"aa58cd5297f269ea69c74dd04960c1ea771b9f5d36bf7fd50bd83c0dc5127341\": container with ID starting with aa58cd5297f269ea69c74dd04960c1ea771b9f5d36bf7fd50bd83c0dc5127341 not found: ID does not exist" May 26 15:46:55.913596 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:46:55.913556 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac6e1e9f-b11d-4382-a264-17e5ff81e84b" path="/var/lib/kubelet/pods/ac6e1e9f-b11d-4382-a264-17e5ff81e84b/volumes" May 26 15:47:46.227918 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.227882 2547 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt"] May 26 15:47:46.228537 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.228214 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac6e1e9f-b11d-4382-a264-17e5ff81e84b" containerName="offline" May 26 15:47:46.228537 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.228226 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac6e1e9f-b11d-4382-a264-17e5ff81e84b" containerName="offline" May 26 15:47:46.228537 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.228238 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac6e1e9f-b11d-4382-a264-17e5ff81e84b" containerName="online" May 26 15:47:46.228537 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.228244 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac6e1e9f-b11d-4382-a264-17e5ff81e84b" containerName="online" May 26 15:47:46.228537 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.228255 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac6e1e9f-b11d-4382-a264-17e5ff81e84b" containerName="registry" May 26 15:47:46.228537 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.228264 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac6e1e9f-b11d-4382-a264-17e5ff81e84b" containerName="registry" May 26 15:47:46.228537 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.228271 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac6e1e9f-b11d-4382-a264-17e5ff81e84b" containerName="ui" May 26 15:47:46.228537 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.228277 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac6e1e9f-b11d-4382-a264-17e5ff81e84b" containerName="ui" May 26 15:47:46.228537 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.228291 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac6e1e9f-b11d-4382-a264-17e5ff81e84b" containerName="feast-init" May 26 15:47:46.228537 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.228300 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac6e1e9f-b11d-4382-a264-17e5ff81e84b" containerName="feast-init" May 26 15:47:46.228537 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.228310 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac6e1e9f-b11d-4382-a264-17e5ff81e84b" containerName="feast-apply" May 26 15:47:46.228537 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.228319 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac6e1e9f-b11d-4382-a264-17e5ff81e84b" containerName="feast-apply" May 26 15:47:46.228537 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.228403 2547 memory_manager.go:356] "RemoveStaleState removing state" podUID="ac6e1e9f-b11d-4382-a264-17e5ff81e84b" containerName="ui" May 26 15:47:46.228537 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.228412 2547 memory_manager.go:356] "RemoveStaleState removing state" podUID="ac6e1e9f-b11d-4382-a264-17e5ff81e84b" containerName="registry" May 26 15:47:46.228537 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.228421 2547 memory_manager.go:356] "RemoveStaleState removing state" podUID="ac6e1e9f-b11d-4382-a264-17e5ff81e84b" containerName="online" May 26 15:47:46.228537 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.228428 2547 memory_manager.go:356] "RemoveStaleState removing state" podUID="ac6e1e9f-b11d-4382-a264-17e5ff81e84b" containerName="offline" May 26 15:47:46.231720 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.231696 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" May 26 15:47:46.233913 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.233888 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" May 26 15:47:46.234021 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.233907 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-dsqk4\"" May 26 15:47:46.234021 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.233890 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" May 26 15:47:46.234021 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.234010 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" May 26 15:47:46.234247 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.234217 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" May 26 15:47:46.234412 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.234393 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" May 26 15:47:46.234515 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.234476 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" May 26 15:47:46.240999 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.240976 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt"] May 26 15:47:46.356295 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.356267 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/72afb19e-266e-4371-ae05-6f2a221ff11b-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lxrgt\" (UID: \"72afb19e-266e-4371-ae05-6f2a221ff11b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" May 26 15:47:46.356455 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.356317 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/72afb19e-266e-4371-ae05-6f2a221ff11b-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lxrgt\" (UID: \"72afb19e-266e-4371-ae05-6f2a221ff11b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" May 26 15:47:46.356455 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.356343 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/72afb19e-266e-4371-ae05-6f2a221ff11b-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lxrgt\" (UID: \"72afb19e-266e-4371-ae05-6f2a221ff11b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" May 26 15:47:46.356455 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.356399 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/72afb19e-266e-4371-ae05-6f2a221ff11b-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lxrgt\" (UID: \"72afb19e-266e-4371-ae05-6f2a221ff11b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" May 26 15:47:46.356455 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.356427 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqjxc\" (UniqueName: \"kubernetes.io/projected/72afb19e-266e-4371-ae05-6f2a221ff11b-kube-api-access-bqjxc\") pod \"feast-simple-feast-remote-setup-5cff94746b-lxrgt\" (UID: \"72afb19e-266e-4371-ae05-6f2a221ff11b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" May 26 15:47:46.356455 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.356455 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/72afb19e-266e-4371-ae05-6f2a221ff11b-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-lxrgt\" (UID: \"72afb19e-266e-4371-ae05-6f2a221ff11b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" May 26 15:47:46.456839 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.456809 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/72afb19e-266e-4371-ae05-6f2a221ff11b-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lxrgt\" (UID: \"72afb19e-266e-4371-ae05-6f2a221ff11b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" May 26 15:47:46.456839 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.456838 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/72afb19e-266e-4371-ae05-6f2a221ff11b-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lxrgt\" (UID: \"72afb19e-266e-4371-ae05-6f2a221ff11b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" May 26 15:47:46.457023 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.456873 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/72afb19e-266e-4371-ae05-6f2a221ff11b-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lxrgt\" (UID: \"72afb19e-266e-4371-ae05-6f2a221ff11b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" May 26 15:47:46.457023 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.456901 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bqjxc\" (UniqueName: \"kubernetes.io/projected/72afb19e-266e-4371-ae05-6f2a221ff11b-kube-api-access-bqjxc\") pod \"feast-simple-feast-remote-setup-5cff94746b-lxrgt\" (UID: \"72afb19e-266e-4371-ae05-6f2a221ff11b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" May 26 15:47:46.457162 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.457106 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/72afb19e-266e-4371-ae05-6f2a221ff11b-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-lxrgt\" (UID: \"72afb19e-266e-4371-ae05-6f2a221ff11b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" May 26 15:47:46.457229 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.457211 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/72afb19e-266e-4371-ae05-6f2a221ff11b-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lxrgt\" (UID: \"72afb19e-266e-4371-ae05-6f2a221ff11b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" May 26 15:47:46.457538 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.457514 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/72afb19e-266e-4371-ae05-6f2a221ff11b-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-lxrgt\" (UID: \"72afb19e-266e-4371-ae05-6f2a221ff11b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" May 26 15:47:46.458022 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.457997 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/72afb19e-266e-4371-ae05-6f2a221ff11b-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lxrgt\" (UID: \"72afb19e-266e-4371-ae05-6f2a221ff11b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" May 26 15:47:46.459360 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.459342 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/72afb19e-266e-4371-ae05-6f2a221ff11b-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lxrgt\" (UID: \"72afb19e-266e-4371-ae05-6f2a221ff11b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" May 26 15:47:46.459517 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.459500 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/72afb19e-266e-4371-ae05-6f2a221ff11b-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lxrgt\" (UID: \"72afb19e-266e-4371-ae05-6f2a221ff11b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" May 26 15:47:46.459608 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.459589 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/72afb19e-266e-4371-ae05-6f2a221ff11b-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lxrgt\" (UID: \"72afb19e-266e-4371-ae05-6f2a221ff11b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" May 26 15:47:46.464763 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.464741 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqjxc\" (UniqueName: \"kubernetes.io/projected/72afb19e-266e-4371-ae05-6f2a221ff11b-kube-api-access-bqjxc\") pod \"feast-simple-feast-remote-setup-5cff94746b-lxrgt\" (UID: \"72afb19e-266e-4371-ae05-6f2a221ff11b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" May 26 15:47:46.542705 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.542643 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" May 26 15:47:46.673443 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.673410 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt"] May 26 15:47:46.677166 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:47:46.677136 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod72afb19e_266e_4371_ae05_6f2a221ff11b.slice/crio-1fee7c3ef47d8ecd4913253246031bd1266ba6a2b7086df9f2861b43f54c3aef WatchSource:0}: Error finding container 1fee7c3ef47d8ecd4913253246031bd1266ba6a2b7086df9f2861b43f54c3aef: Status 404 returned error can't find the container with id 1fee7c3ef47d8ecd4913253246031bd1266ba6a2b7086df9f2861b43f54c3aef May 26 15:47:46.688855 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:46.688830 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" event={"ID":"72afb19e-266e-4371-ae05-6f2a221ff11b","Type":"ContainerStarted","Data":"1fee7c3ef47d8ecd4913253246031bd1266ba6a2b7086df9f2861b43f54c3aef"} May 26 15:47:47.693745 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:47.693701 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" event={"ID":"72afb19e-266e-4371-ae05-6f2a221ff11b","Type":"ContainerStarted","Data":"0798733dfa1d93eb50409f2b6ccdc0324179bf9c820d5b665417e068fbc611ac"} May 26 15:47:51.710741 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:51.710710 2547 generic.go:358] "Generic (PLEG): container finished" podID="72afb19e-266e-4371-ae05-6f2a221ff11b" containerID="0798733dfa1d93eb50409f2b6ccdc0324179bf9c820d5b665417e068fbc611ac" exitCode=0 May 26 15:47:51.711114 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:51.710786 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" event={"ID":"72afb19e-266e-4371-ae05-6f2a221ff11b","Type":"ContainerDied","Data":"0798733dfa1d93eb50409f2b6ccdc0324179bf9c820d5b665417e068fbc611ac"} May 26 15:47:52.715182 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:52.715147 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" event={"ID":"72afb19e-266e-4371-ae05-6f2a221ff11b","Type":"ContainerStarted","Data":"a2dcdc289f321259b8b13baf003f766780661f163bb45cd7a37b3d8da1d90313"} May 26 15:47:57.731654 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:57.731617 2547 generic.go:358] "Generic (PLEG): container finished" podID="72afb19e-266e-4371-ae05-6f2a221ff11b" containerID="a2dcdc289f321259b8b13baf003f766780661f163bb45cd7a37b3d8da1d90313" exitCode=0 May 26 15:47:57.732091 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:57.731687 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" event={"ID":"72afb19e-266e-4371-ae05-6f2a221ff11b","Type":"ContainerDied","Data":"a2dcdc289f321259b8b13baf003f766780661f163bb45cd7a37b3d8da1d90313"} May 26 15:47:58.737796 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:58.737755 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" event={"ID":"72afb19e-266e-4371-ae05-6f2a221ff11b","Type":"ContainerStarted","Data":"9fbd7c95773fa4ed84937cfb02b7e0cb387bea0facc64acff5bf23c3906d9572"} May 26 15:47:58.738220 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:58.737809 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" event={"ID":"72afb19e-266e-4371-ae05-6f2a221ff11b","Type":"ContainerStarted","Data":"4cef81af5930740a0a877790856242a0f20468c016f1dd3152ac4c20c3d4690c"} May 26 15:47:58.738220 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:58.737825 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" event={"ID":"72afb19e-266e-4371-ae05-6f2a221ff11b","Type":"ContainerStarted","Data":"4e830ab295e0655bd6407c1dfbb2f33014381438e857680d57ba294d46a9a9a7"} May 26 15:47:58.763639 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:47:58.763591 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" podStartSLOduration=12.763576037 podStartE2EDuration="12.763576037s" podCreationTimestamp="2026-05-26 15:47:46 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 15:47:58.762301974 +0000 UTC m=+1627.488976897" watchObservedRunningTime="2026-05-26 15:47:58.763576037 +0000 UTC m=+1627.490250986" May 26 15:48:01.543837 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:48:01.543788 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" May 26 15:48:01.544337 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:48:01.543864 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" May 26 15:48:01.544337 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:48:01.543880 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" May 26 15:48:01.545416 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:48:01.545373 2547 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" podUID="72afb19e-266e-4371-ae05-6f2a221ff11b" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.37:8816: connect: connection refused" May 26 15:48:01.545416 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:48:01.545390 2547 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" podUID="72afb19e-266e-4371-ae05-6f2a221ff11b" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.37:8443: connect: connection refused" May 26 15:48:01.545614 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:48:01.545421 2547 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" podUID="72afb19e-266e-4371-ae05-6f2a221ff11b" containerName="online" probeResult="failure" output="Get \"https://10.134.0.37:6567/health\": dial tcp 10.134.0.37:6567: connect: connection refused" May 26 15:48:04.543847 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:48:04.543818 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" May 26 15:48:04.544355 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:48:04.543932 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" May 26 15:48:04.544355 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:48:04.544270 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" May 26 15:48:04.544355 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:48:04.544302 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" May 26 15:48:04.544651 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:48:04.544626 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" May 26 15:48:04.544832 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:48:04.544816 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" May 26 15:48:04.548006 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:48:04.547985 2547 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" May 26 15:48:04.758757 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:48:04.758725 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" May 26 15:48:04.761539 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:48:04.761513 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" May 26 15:49:07.348667 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:07.348571 2547 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt"] May 26 15:49:07.349220 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:07.349001 2547 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" podUID="72afb19e-266e-4371-ae05-6f2a221ff11b" containerName="online" containerID="cri-o://4e830ab295e0655bd6407c1dfbb2f33014381438e857680d57ba294d46a9a9a7" gracePeriod=30 May 26 15:49:07.349220 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:07.349111 2547 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" podUID="72afb19e-266e-4371-ae05-6f2a221ff11b" containerName="ui" containerID="cri-o://9fbd7c95773fa4ed84937cfb02b7e0cb387bea0facc64acff5bf23c3906d9572" gracePeriod=30 May 26 15:49:07.349348 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:07.349215 2547 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" podUID="72afb19e-266e-4371-ae05-6f2a221ff11b" containerName="offline" containerID="cri-o://4cef81af5930740a0a877790856242a0f20468c016f1dd3152ac4c20c3d4690c" gracePeriod=30 May 26 15:49:07.567703 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:07.567672 2547 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96"] May 26 15:49:07.568072 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:07.568006 2547 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" podUID="5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" containerName="registry" containerID="cri-o://a32f2d21fb5cc6a4a46e5ead67e84a228bddc13c8e32fb53b30c83e4dd048e09" gracePeriod=30 May 26 15:49:07.568306 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:07.568042 2547 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" podUID="5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" containerName="offline" containerID="cri-o://087a0794daf43a6309b7818411bbd03ee8dfd341668aa2b5d348dbbafb00d32c" gracePeriod=30 May 26 15:49:07.568432 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:07.568090 2547 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" podUID="5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" containerName="ui" containerID="cri-o://bb77927413d6f15fd52063433d19be435be78e3fcf44ef46276ce687cc334c44" gracePeriod=30 May 26 15:49:07.568432 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:07.568149 2547 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" podUID="5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" containerName="online" containerID="cri-o://072cfe9e4fd5b6cbccd1e3d4d82579423d896c3effaf0425d33e11ca208cc9a4" gracePeriod=30 May 26 15:49:07.946281 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:07.946244 2547 generic.go:358] "Generic (PLEG): container finished" podID="72afb19e-266e-4371-ae05-6f2a221ff11b" containerID="9fbd7c95773fa4ed84937cfb02b7e0cb387bea0facc64acff5bf23c3906d9572" exitCode=0 May 26 15:49:07.946460 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:07.946322 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" event={"ID":"72afb19e-266e-4371-ae05-6f2a221ff11b","Type":"ContainerDied","Data":"9fbd7c95773fa4ed84937cfb02b7e0cb387bea0facc64acff5bf23c3906d9572"} May 26 15:49:08.951663 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:08.951633 2547 generic.go:358] "Generic (PLEG): container finished" podID="5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" containerID="bb77927413d6f15fd52063433d19be435be78e3fcf44ef46276ce687cc334c44" exitCode=0 May 26 15:49:08.951663 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:08.951658 2547 generic.go:358] "Generic (PLEG): container finished" podID="5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" containerID="072cfe9e4fd5b6cbccd1e3d4d82579423d896c3effaf0425d33e11ca208cc9a4" exitCode=0 May 26 15:49:08.952187 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:08.951711 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" event={"ID":"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d","Type":"ContainerDied","Data":"bb77927413d6f15fd52063433d19be435be78e3fcf44ef46276ce687cc334c44"} May 26 15:49:08.952187 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:08.951745 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" event={"ID":"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d","Type":"ContainerDied","Data":"072cfe9e4fd5b6cbccd1e3d4d82579423d896c3effaf0425d33e11ca208cc9a4"} May 26 15:49:08.953905 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:08.953885 2547 generic.go:358] "Generic (PLEG): container finished" podID="72afb19e-266e-4371-ae05-6f2a221ff11b" containerID="4e830ab295e0655bd6407c1dfbb2f33014381438e857680d57ba294d46a9a9a7" exitCode=0 May 26 15:49:08.954009 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:08.953958 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" event={"ID":"72afb19e-266e-4371-ae05-6f2a221ff11b","Type":"ContainerDied","Data":"4e830ab295e0655bd6407c1dfbb2f33014381438e857680d57ba294d46a9a9a7"} May 26 15:49:13.821184 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:13.821140 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" podUID="5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.36:8443: connect: connection refused" May 26 15:49:14.501235 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:14.501190 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" podUID="5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.36:6567/health\": dial tcp 10.134.0.36:6567: connect: connection refused" May 26 15:49:14.545167 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:14.545125 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" podUID="72afb19e-266e-4371-ae05-6f2a221ff11b" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.37:8443: connect: connection refused" May 26 15:49:14.759787 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:14.759693 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" podUID="72afb19e-266e-4371-ae05-6f2a221ff11b" containerName="online" probeResult="failure" output="Get \"https://10.134.0.37:6567/health\": dial tcp 10.134.0.37:6567: connect: connection refused" May 26 15:49:23.820840 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:23.820791 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" podUID="5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.36:8443: connect: connection refused" May 26 15:49:24.501494 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:24.501453 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" podUID="5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.36:6567/health\": dial tcp 10.134.0.36:6567: connect: connection refused" May 26 15:49:24.544569 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:24.544518 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" podUID="72afb19e-266e-4371-ae05-6f2a221ff11b" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.37:8443: connect: connection refused" May 26 15:49:24.759398 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:24.759309 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" podUID="72afb19e-266e-4371-ae05-6f2a221ff11b" containerName="online" probeResult="failure" output="Get \"https://10.134.0.37:6567/health\": dial tcp 10.134.0.37:6567: connect: connection refused" May 26 15:49:33.820564 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:33.820519 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" podUID="5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.36:8443: connect: connection refused" May 26 15:49:33.820963 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:33.820648 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:49:34.501393 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:34.501349 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" podUID="5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.36:6567/health\": dial tcp 10.134.0.36:6567: connect: connection refused" May 26 15:49:34.501557 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:34.501484 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:49:34.544531 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:34.544490 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" podUID="72afb19e-266e-4371-ae05-6f2a221ff11b" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.37:8443: connect: connection refused" May 26 15:49:34.544693 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:34.544597 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" May 26 15:49:34.759132 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:34.759012 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" podUID="72afb19e-266e-4371-ae05-6f2a221ff11b" containerName="online" probeResult="failure" output="Get \"https://10.134.0.37:6567/health\": dial tcp 10.134.0.37:6567: connect: connection refused" May 26 15:49:34.759301 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:34.759152 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" May 26 15:49:37.520438 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:37.520414 2547 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" May 26 15:49:37.660354 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:37.660280 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/72afb19e-266e-4371-ae05-6f2a221ff11b-registry-tls\") pod \"72afb19e-266e-4371-ae05-6f2a221ff11b\" (UID: \"72afb19e-266e-4371-ae05-6f2a221ff11b\") " May 26 15:49:37.660354 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:37.660323 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/72afb19e-266e-4371-ae05-6f2a221ff11b-ui-tls\") pod \"72afb19e-266e-4371-ae05-6f2a221ff11b\" (UID: \"72afb19e-266e-4371-ae05-6f2a221ff11b\") " May 26 15:49:37.660354 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:37.660350 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/72afb19e-266e-4371-ae05-6f2a221ff11b-online-tls\") pod \"72afb19e-266e-4371-ae05-6f2a221ff11b\" (UID: \"72afb19e-266e-4371-ae05-6f2a221ff11b\") " May 26 15:49:37.660623 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:37.660463 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/72afb19e-266e-4371-ae05-6f2a221ff11b-offline-tls\") pod \"72afb19e-266e-4371-ae05-6f2a221ff11b\" (UID: \"72afb19e-266e-4371-ae05-6f2a221ff11b\") " May 26 15:49:37.660623 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:37.660518 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bqjxc\" (UniqueName: \"kubernetes.io/projected/72afb19e-266e-4371-ae05-6f2a221ff11b-kube-api-access-bqjxc\") pod \"72afb19e-266e-4371-ae05-6f2a221ff11b\" (UID: \"72afb19e-266e-4371-ae05-6f2a221ff11b\") " May 26 15:49:37.660623 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:37.660566 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/72afb19e-266e-4371-ae05-6f2a221ff11b-feast-data\") pod \"72afb19e-266e-4371-ae05-6f2a221ff11b\" (UID: \"72afb19e-266e-4371-ae05-6f2a221ff11b\") " May 26 15:49:37.660773 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:37.660680 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72afb19e-266e-4371-ae05-6f2a221ff11b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "72afb19e-266e-4371-ae05-6f2a221ff11b" (UID: "72afb19e-266e-4371-ae05-6f2a221ff11b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 26 15:49:37.660931 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:37.660909 2547 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/72afb19e-266e-4371-ae05-6f2a221ff11b-registry-tls\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:49:37.661139 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:37.661112 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72afb19e-266e-4371-ae05-6f2a221ff11b-feast-data" (OuterVolumeSpecName: "feast-data") pod "72afb19e-266e-4371-ae05-6f2a221ff11b" (UID: "72afb19e-266e-4371-ae05-6f2a221ff11b"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 26 15:49:37.663020 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:37.662995 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72afb19e-266e-4371-ae05-6f2a221ff11b-online-tls" (OuterVolumeSpecName: "online-tls") pod "72afb19e-266e-4371-ae05-6f2a221ff11b" (UID: "72afb19e-266e-4371-ae05-6f2a221ff11b"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 26 15:49:37.663020 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:37.663007 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72afb19e-266e-4371-ae05-6f2a221ff11b-kube-api-access-bqjxc" (OuterVolumeSpecName: "kube-api-access-bqjxc") pod "72afb19e-266e-4371-ae05-6f2a221ff11b" (UID: "72afb19e-266e-4371-ae05-6f2a221ff11b"). InnerVolumeSpecName "kube-api-access-bqjxc". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 26 15:49:37.663143 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:37.663041 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72afb19e-266e-4371-ae05-6f2a221ff11b-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "72afb19e-266e-4371-ae05-6f2a221ff11b" (UID: "72afb19e-266e-4371-ae05-6f2a221ff11b"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 26 15:49:37.663143 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:37.663080 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72afb19e-266e-4371-ae05-6f2a221ff11b-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "72afb19e-266e-4371-ae05-6f2a221ff11b" (UID: "72afb19e-266e-4371-ae05-6f2a221ff11b"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 26 15:49:37.761733 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:37.761700 2547 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/72afb19e-266e-4371-ae05-6f2a221ff11b-offline-tls\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:49:37.761733 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:37.761728 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqjxc\" (UniqueName: \"kubernetes.io/projected/72afb19e-266e-4371-ae05-6f2a221ff11b-kube-api-access-bqjxc\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:49:37.761733 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:37.761739 2547 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/72afb19e-266e-4371-ae05-6f2a221ff11b-feast-data\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:49:37.761936 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:37.761748 2547 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/72afb19e-266e-4371-ae05-6f2a221ff11b-ui-tls\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:49:37.761936 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:37.761757 2547 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/72afb19e-266e-4371-ae05-6f2a221ff11b-online-tls\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:49:38.038853 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.038821 2547 generic.go:358] "Generic (PLEG): container finished" podID="5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" containerID="087a0794daf43a6309b7818411bbd03ee8dfd341668aa2b5d348dbbafb00d32c" exitCode=137 May 26 15:49:38.038853 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.038845 2547 generic.go:358] "Generic (PLEG): container finished" podID="5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" containerID="a32f2d21fb5cc6a4a46e5ead67e84a228bddc13c8e32fb53b30c83e4dd048e09" exitCode=137 May 26 15:49:38.039040 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.038890 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" event={"ID":"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d","Type":"ContainerDied","Data":"087a0794daf43a6309b7818411bbd03ee8dfd341668aa2b5d348dbbafb00d32c"} May 26 15:49:38.039040 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.038924 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" event={"ID":"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d","Type":"ContainerDied","Data":"a32f2d21fb5cc6a4a46e5ead67e84a228bddc13c8e32fb53b30c83e4dd048e09"} May 26 15:49:38.040994 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.040971 2547 generic.go:358] "Generic (PLEG): container finished" podID="72afb19e-266e-4371-ae05-6f2a221ff11b" containerID="4cef81af5930740a0a877790856242a0f20468c016f1dd3152ac4c20c3d4690c" exitCode=137 May 26 15:49:38.041143 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.041037 2547 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" May 26 15:49:38.041143 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.041047 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" event={"ID":"72afb19e-266e-4371-ae05-6f2a221ff11b","Type":"ContainerDied","Data":"4cef81af5930740a0a877790856242a0f20468c016f1dd3152ac4c20c3d4690c"} May 26 15:49:38.041143 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.041094 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt" event={"ID":"72afb19e-266e-4371-ae05-6f2a221ff11b","Type":"ContainerDied","Data":"1fee7c3ef47d8ecd4913253246031bd1266ba6a2b7086df9f2861b43f54c3aef"} May 26 15:49:38.041143 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.041115 2547 scope.go:117] "RemoveContainer" containerID="9fbd7c95773fa4ed84937cfb02b7e0cb387bea0facc64acff5bf23c3906d9572" May 26 15:49:38.049428 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.049413 2547 scope.go:117] "RemoveContainer" containerID="4cef81af5930740a0a877790856242a0f20468c016f1dd3152ac4c20c3d4690c" May 26 15:49:38.056819 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.056795 2547 scope.go:117] "RemoveContainer" containerID="4e830ab295e0655bd6407c1dfbb2f33014381438e857680d57ba294d46a9a9a7" May 26 15:49:38.057296 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.057274 2547 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt"] May 26 15:49:38.061209 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.061188 2547 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lxrgt"] May 26 15:49:38.064865 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.064851 2547 scope.go:117] "RemoveContainer" containerID="a2dcdc289f321259b8b13baf003f766780661f163bb45cd7a37b3d8da1d90313" May 26 15:49:38.071608 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.071592 2547 scope.go:117] "RemoveContainer" containerID="0798733dfa1d93eb50409f2b6ccdc0324179bf9c820d5b665417e068fbc611ac" May 26 15:49:38.078014 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.077999 2547 scope.go:117] "RemoveContainer" containerID="9fbd7c95773fa4ed84937cfb02b7e0cb387bea0facc64acff5bf23c3906d9572" May 26 15:49:38.078296 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:49:38.078277 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fbd7c95773fa4ed84937cfb02b7e0cb387bea0facc64acff5bf23c3906d9572\": container with ID starting with 9fbd7c95773fa4ed84937cfb02b7e0cb387bea0facc64acff5bf23c3906d9572 not found: ID does not exist" containerID="9fbd7c95773fa4ed84937cfb02b7e0cb387bea0facc64acff5bf23c3906d9572" May 26 15:49:38.078365 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.078304 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fbd7c95773fa4ed84937cfb02b7e0cb387bea0facc64acff5bf23c3906d9572"} err="failed to get container status \"9fbd7c95773fa4ed84937cfb02b7e0cb387bea0facc64acff5bf23c3906d9572\": rpc error: code = NotFound desc = could not find container \"9fbd7c95773fa4ed84937cfb02b7e0cb387bea0facc64acff5bf23c3906d9572\": container with ID starting with 9fbd7c95773fa4ed84937cfb02b7e0cb387bea0facc64acff5bf23c3906d9572 not found: ID does not exist" May 26 15:49:38.078365 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.078323 2547 scope.go:117] "RemoveContainer" containerID="4cef81af5930740a0a877790856242a0f20468c016f1dd3152ac4c20c3d4690c" May 26 15:49:38.078563 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:49:38.078537 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4cef81af5930740a0a877790856242a0f20468c016f1dd3152ac4c20c3d4690c\": container with ID starting with 4cef81af5930740a0a877790856242a0f20468c016f1dd3152ac4c20c3d4690c not found: ID does not exist" containerID="4cef81af5930740a0a877790856242a0f20468c016f1dd3152ac4c20c3d4690c" May 26 15:49:38.078629 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.078565 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4cef81af5930740a0a877790856242a0f20468c016f1dd3152ac4c20c3d4690c"} err="failed to get container status \"4cef81af5930740a0a877790856242a0f20468c016f1dd3152ac4c20c3d4690c\": rpc error: code = NotFound desc = could not find container \"4cef81af5930740a0a877790856242a0f20468c016f1dd3152ac4c20c3d4690c\": container with ID starting with 4cef81af5930740a0a877790856242a0f20468c016f1dd3152ac4c20c3d4690c not found: ID does not exist" May 26 15:49:38.078629 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.078589 2547 scope.go:117] "RemoveContainer" containerID="4e830ab295e0655bd6407c1dfbb2f33014381438e857680d57ba294d46a9a9a7" May 26 15:49:38.078793 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:49:38.078777 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e830ab295e0655bd6407c1dfbb2f33014381438e857680d57ba294d46a9a9a7\": container with ID starting with 4e830ab295e0655bd6407c1dfbb2f33014381438e857680d57ba294d46a9a9a7 not found: ID does not exist" containerID="4e830ab295e0655bd6407c1dfbb2f33014381438e857680d57ba294d46a9a9a7" May 26 15:49:38.078832 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.078799 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e830ab295e0655bd6407c1dfbb2f33014381438e857680d57ba294d46a9a9a7"} err="failed to get container status \"4e830ab295e0655bd6407c1dfbb2f33014381438e857680d57ba294d46a9a9a7\": rpc error: code = NotFound desc = could not find container \"4e830ab295e0655bd6407c1dfbb2f33014381438e857680d57ba294d46a9a9a7\": container with ID starting with 4e830ab295e0655bd6407c1dfbb2f33014381438e857680d57ba294d46a9a9a7 not found: ID does not exist" May 26 15:49:38.078832 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.078814 2547 scope.go:117] "RemoveContainer" containerID="a2dcdc289f321259b8b13baf003f766780661f163bb45cd7a37b3d8da1d90313" May 26 15:49:38.079017 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:49:38.079000 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2dcdc289f321259b8b13baf003f766780661f163bb45cd7a37b3d8da1d90313\": container with ID starting with a2dcdc289f321259b8b13baf003f766780661f163bb45cd7a37b3d8da1d90313 not found: ID does not exist" containerID="a2dcdc289f321259b8b13baf003f766780661f163bb45cd7a37b3d8da1d90313" May 26 15:49:38.079120 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.079017 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2dcdc289f321259b8b13baf003f766780661f163bb45cd7a37b3d8da1d90313"} err="failed to get container status \"a2dcdc289f321259b8b13baf003f766780661f163bb45cd7a37b3d8da1d90313\": rpc error: code = NotFound desc = could not find container \"a2dcdc289f321259b8b13baf003f766780661f163bb45cd7a37b3d8da1d90313\": container with ID starting with a2dcdc289f321259b8b13baf003f766780661f163bb45cd7a37b3d8da1d90313 not found: ID does not exist" May 26 15:49:38.079120 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.079033 2547 scope.go:117] "RemoveContainer" containerID="0798733dfa1d93eb50409f2b6ccdc0324179bf9c820d5b665417e068fbc611ac" May 26 15:49:38.079306 ip-10-0-139-198 kubenswrapper[2547]: E0526 15:49:38.079288 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0798733dfa1d93eb50409f2b6ccdc0324179bf9c820d5b665417e068fbc611ac\": container with ID starting with 0798733dfa1d93eb50409f2b6ccdc0324179bf9c820d5b665417e068fbc611ac not found: ID does not exist" containerID="0798733dfa1d93eb50409f2b6ccdc0324179bf9c820d5b665417e068fbc611ac" May 26 15:49:38.079387 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.079310 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0798733dfa1d93eb50409f2b6ccdc0324179bf9c820d5b665417e068fbc611ac"} err="failed to get container status \"0798733dfa1d93eb50409f2b6ccdc0324179bf9c820d5b665417e068fbc611ac\": rpc error: code = NotFound desc = could not find container \"0798733dfa1d93eb50409f2b6ccdc0324179bf9c820d5b665417e068fbc611ac\": container with ID starting with 0798733dfa1d93eb50409f2b6ccdc0324179bf9c820d5b665417e068fbc611ac not found: ID does not exist" May 26 15:49:38.200602 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.200580 2547 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:49:38.264644 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.264614 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-feast-data\") pod \"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d\" (UID: \"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d\") " May 26 15:49:38.264821 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.264660 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-ui-tls\") pod \"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d\" (UID: \"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d\") " May 26 15:49:38.264821 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.264720 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-registry-tls\") pod \"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d\" (UID: \"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d\") " May 26 15:49:38.264821 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.264735 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-offline-tls\") pod \"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d\" (UID: \"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d\") " May 26 15:49:38.264821 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.264775 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-online-tls\") pod \"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d\" (UID: \"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d\") " May 26 15:49:38.265078 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.264905 2547 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xgkj8\" (UniqueName: \"kubernetes.io/projected/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-kube-api-access-xgkj8\") pod \"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d\" (UID: \"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d\") " May 26 15:49:38.265231 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.265204 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-feast-data" (OuterVolumeSpecName: "feast-data") pod "5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" (UID: "5bca7cd5-f29c-44fb-ba0e-a4ab0268051d"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 26 15:49:38.266986 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.266962 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" (UID: "5bca7cd5-f29c-44fb-ba0e-a4ab0268051d"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 26 15:49:38.266986 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.266978 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" (UID: "5bca7cd5-f29c-44fb-ba0e-a4ab0268051d"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 26 15:49:38.267146 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.266960 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-kube-api-access-xgkj8" (OuterVolumeSpecName: "kube-api-access-xgkj8") pod "5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" (UID: "5bca7cd5-f29c-44fb-ba0e-a4ab0268051d"). InnerVolumeSpecName "kube-api-access-xgkj8". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 26 15:49:38.267252 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.267230 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-online-tls" (OuterVolumeSpecName: "online-tls") pod "5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" (UID: "5bca7cd5-f29c-44fb-ba0e-a4ab0268051d"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 26 15:49:38.267291 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.267236 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" (UID: "5bca7cd5-f29c-44fb-ba0e-a4ab0268051d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 26 15:49:38.365807 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.365779 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgkj8\" (UniqueName: \"kubernetes.io/projected/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-kube-api-access-xgkj8\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:49:38.365807 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.365807 2547 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-feast-data\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:49:38.365966 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.365817 2547 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-ui-tls\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:49:38.365966 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.365827 2547 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-registry-tls\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:49:38.365966 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.365835 2547 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-offline-tls\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:49:38.365966 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:38.365845 2547 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d-online-tls\") on node \"ip-10-0-139-198.ec2.internal\" DevicePath \"\"" May 26 15:49:39.046879 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:39.046853 2547 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" May 26 15:49:39.047433 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:39.046847 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96" event={"ID":"5bca7cd5-f29c-44fb-ba0e-a4ab0268051d","Type":"ContainerDied","Data":"fc975233db16f3591bad49b1f4c26bb58f8464327f8ee25e364f376d159225be"} May 26 15:49:39.047433 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:39.046970 2547 scope.go:117] "RemoveContainer" containerID="bb77927413d6f15fd52063433d19be435be78e3fcf44ef46276ce687cc334c44" May 26 15:49:39.055374 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:39.055352 2547 scope.go:117] "RemoveContainer" containerID="087a0794daf43a6309b7818411bbd03ee8dfd341668aa2b5d348dbbafb00d32c" May 26 15:49:39.062283 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:39.062266 2547 scope.go:117] "RemoveContainer" containerID="072cfe9e4fd5b6cbccd1e3d4d82579423d896c3effaf0425d33e11ca208cc9a4" May 26 15:49:39.067467 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:39.067441 2547 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96"] May 26 15:49:39.069752 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:39.069705 2547 scope.go:117] "RemoveContainer" containerID="a32f2d21fb5cc6a4a46e5ead67e84a228bddc13c8e32fb53b30c83e4dd048e09" May 26 15:49:39.071417 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:39.071394 2547 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-g2x96"] May 26 15:49:39.076453 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:39.076439 2547 scope.go:117] "RemoveContainer" containerID="ca6b0629cca9577652e34cd366e7f8c01e85e3c05024466c8aa2422e7cf125e0" May 26 15:49:39.083052 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:39.083038 2547 scope.go:117] "RemoveContainer" containerID="3e063ecf3a67e550619cdef234f7b613646c4cd3c0983b9d79376decc5801f07" May 26 15:49:39.913981 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:39.913945 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" path="/var/lib/kubelet/pods/5bca7cd5-f29c-44fb-ba0e-a4ab0268051d/volumes" May 26 15:49:39.914589 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:49:39.914574 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72afb19e-266e-4371-ae05-6f2a221ff11b" path="/var/lib/kubelet/pods/72afb19e-266e-4371-ae05-6f2a221ff11b/volumes" May 26 15:50:14.947289 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:14.947257 2547 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8ln6p/must-gather-2cht6"] May 26 15:50:14.947763 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:14.947532 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" containerName="registry" May 26 15:50:14.947763 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:14.947543 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" containerName="registry" May 26 15:50:14.947763 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:14.947554 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72afb19e-266e-4371-ae05-6f2a221ff11b" containerName="feast-init" May 26 15:50:14.947763 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:14.947559 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="72afb19e-266e-4371-ae05-6f2a221ff11b" containerName="feast-init" May 26 15:50:14.947763 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:14.947565 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" containerName="feast-init" May 26 15:50:14.947763 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:14.947570 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" containerName="feast-init" May 26 15:50:14.947763 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:14.947576 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" containerName="ui" May 26 15:50:14.947763 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:14.947581 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" containerName="ui" May 26 15:50:14.947763 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:14.947589 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" containerName="offline" May 26 15:50:14.947763 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:14.947594 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" containerName="offline" May 26 15:50:14.947763 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:14.947602 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" containerName="online" May 26 15:50:14.947763 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:14.947607 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" containerName="online" May 26 15:50:14.947763 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:14.947613 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72afb19e-266e-4371-ae05-6f2a221ff11b" containerName="online" May 26 15:50:14.947763 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:14.947618 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="72afb19e-266e-4371-ae05-6f2a221ff11b" containerName="online" May 26 15:50:14.947763 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:14.947625 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72afb19e-266e-4371-ae05-6f2a221ff11b" containerName="offline" May 26 15:50:14.947763 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:14.947629 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="72afb19e-266e-4371-ae05-6f2a221ff11b" containerName="offline" May 26 15:50:14.947763 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:14.947638 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72afb19e-266e-4371-ae05-6f2a221ff11b" containerName="feast-apply" May 26 15:50:14.947763 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:14.947642 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="72afb19e-266e-4371-ae05-6f2a221ff11b" containerName="feast-apply" May 26 15:50:14.947763 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:14.947653 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" containerName="feast-apply" May 26 15:50:14.947763 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:14.947658 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" containerName="feast-apply" May 26 15:50:14.947763 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:14.947663 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72afb19e-266e-4371-ae05-6f2a221ff11b" containerName="ui" May 26 15:50:14.947763 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:14.947668 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="72afb19e-266e-4371-ae05-6f2a221ff11b" containerName="ui" May 26 15:50:14.947763 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:14.947711 2547 memory_manager.go:356] "RemoveStaleState removing state" podUID="5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" containerName="ui" May 26 15:50:14.947763 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:14.947719 2547 memory_manager.go:356] "RemoveStaleState removing state" podUID="5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" containerName="registry" May 26 15:50:14.947763 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:14.947724 2547 memory_manager.go:356] "RemoveStaleState removing state" podUID="72afb19e-266e-4371-ae05-6f2a221ff11b" containerName="online" May 26 15:50:14.947763 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:14.947730 2547 memory_manager.go:356] "RemoveStaleState removing state" podUID="5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" containerName="offline" May 26 15:50:14.947763 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:14.947736 2547 memory_manager.go:356] "RemoveStaleState removing state" podUID="72afb19e-266e-4371-ae05-6f2a221ff11b" containerName="offline" May 26 15:50:14.947763 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:14.947743 2547 memory_manager.go:356] "RemoveStaleState removing state" podUID="5bca7cd5-f29c-44fb-ba0e-a4ab0268051d" containerName="online" May 26 15:50:14.947763 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:14.947750 2547 memory_manager.go:356] "RemoveStaleState removing state" podUID="72afb19e-266e-4371-ae05-6f2a221ff11b" containerName="ui" May 26 15:50:14.952205 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:14.952182 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8ln6p/must-gather-2cht6" May 26 15:50:14.954511 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:14.954490 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8ln6p\"/\"kube-root-ca.crt\"" May 26 15:50:14.955223 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:14.955206 2547 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8ln6p\"/\"openshift-service-ca.crt\"" May 26 15:50:14.955277 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:14.955242 2547 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8ln6p\"/\"default-dockercfg-s9jsb\"" May 26 15:50:14.961086 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:14.961048 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8ln6p/must-gather-2cht6"] May 26 15:50:15.035410 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:15.035386 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/cfb55821-cd73-4b73-bb4c-13e7c9d53af2-must-gather-output\") pod \"must-gather-2cht6\" (UID: \"cfb55821-cd73-4b73-bb4c-13e7c9d53af2\") " pod="openshift-must-gather-8ln6p/must-gather-2cht6" May 26 15:50:15.035530 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:15.035428 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njt5d\" (UniqueName: \"kubernetes.io/projected/cfb55821-cd73-4b73-bb4c-13e7c9d53af2-kube-api-access-njt5d\") pod \"must-gather-2cht6\" (UID: \"cfb55821-cd73-4b73-bb4c-13e7c9d53af2\") " pod="openshift-must-gather-8ln6p/must-gather-2cht6" May 26 15:50:15.135808 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:15.135780 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/cfb55821-cd73-4b73-bb4c-13e7c9d53af2-must-gather-output\") pod \"must-gather-2cht6\" (UID: \"cfb55821-cd73-4b73-bb4c-13e7c9d53af2\") " pod="openshift-must-gather-8ln6p/must-gather-2cht6" May 26 15:50:15.135927 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:15.135825 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-njt5d\" (UniqueName: \"kubernetes.io/projected/cfb55821-cd73-4b73-bb4c-13e7c9d53af2-kube-api-access-njt5d\") pod \"must-gather-2cht6\" (UID: \"cfb55821-cd73-4b73-bb4c-13e7c9d53af2\") " pod="openshift-must-gather-8ln6p/must-gather-2cht6" May 26 15:50:15.136127 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:15.136110 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/cfb55821-cd73-4b73-bb4c-13e7c9d53af2-must-gather-output\") pod \"must-gather-2cht6\" (UID: \"cfb55821-cd73-4b73-bb4c-13e7c9d53af2\") " pod="openshift-must-gather-8ln6p/must-gather-2cht6" May 26 15:50:15.144743 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:15.144726 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-njt5d\" (UniqueName: \"kubernetes.io/projected/cfb55821-cd73-4b73-bb4c-13e7c9d53af2-kube-api-access-njt5d\") pod \"must-gather-2cht6\" (UID: \"cfb55821-cd73-4b73-bb4c-13e7c9d53af2\") " pod="openshift-must-gather-8ln6p/must-gather-2cht6" May 26 15:50:15.274955 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:15.274879 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8ln6p/must-gather-2cht6" May 26 15:50:15.402822 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:15.402793 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8ln6p/must-gather-2cht6"] May 26 15:50:15.405805 ip-10-0-139-198 kubenswrapper[2547]: W0526 15:50:15.405780 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcfb55821_cd73_4b73_bb4c_13e7c9d53af2.slice/crio-8b8dd7bb7dc2e9fae7bd0c3585da0efef1e5b9efd1cc1b96c4d033148db2488c WatchSource:0}: Error finding container 8b8dd7bb7dc2e9fae7bd0c3585da0efef1e5b9efd1cc1b96c4d033148db2488c: Status 404 returned error can't find the container with id 8b8dd7bb7dc2e9fae7bd0c3585da0efef1e5b9efd1cc1b96c4d033148db2488c May 26 15:50:15.407608 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:15.407591 2547 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 26 15:50:16.152759 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:16.152710 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8ln6p/must-gather-2cht6" event={"ID":"cfb55821-cd73-4b73-bb4c-13e7c9d53af2","Type":"ContainerStarted","Data":"8b8dd7bb7dc2e9fae7bd0c3585da0efef1e5b9efd1cc1b96c4d033148db2488c"} May 26 15:50:17.157095 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:17.157028 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8ln6p/must-gather-2cht6" event={"ID":"cfb55821-cd73-4b73-bb4c-13e7c9d53af2","Type":"ContainerStarted","Data":"a6106fa8e44961a54dea727b64c5462781085d22a34211e0b188194447e6193e"} May 26 15:50:17.157095 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:17.157097 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8ln6p/must-gather-2cht6" event={"ID":"cfb55821-cd73-4b73-bb4c-13e7c9d53af2","Type":"ContainerStarted","Data":"925cb926dfe263a5cc12ca940cbbbeced6c17e4d5644e5f60d05224639245e56"} May 26 15:50:17.174240 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:17.174176 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-8ln6p/must-gather-2cht6" podStartSLOduration=2.450329582 podStartE2EDuration="3.174157691s" podCreationTimestamp="2026-05-26 15:50:14 +0000 UTC" firstStartedPulling="2026-05-26 15:50:15.407718876 +0000 UTC m=+1764.134393774" lastFinishedPulling="2026-05-26 15:50:16.131546984 +0000 UTC m=+1764.858221883" observedRunningTime="2026-05-26 15:50:17.173348459 +0000 UTC m=+1765.900023381" watchObservedRunningTime="2026-05-26 15:50:17.174157691 +0000 UTC m=+1765.900832617" May 26 15:50:17.522442 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:17.522405 2547 ???:1] "http2: server: error reading preface from client 10.0.139.198:51530: read tcp 10.0.139.198:10250->10.0.139.198:51530: read: connection reset by peer" May 26 15:50:17.526515 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:17.526492 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-7d47p_ba53dd75-3f69-4a05-aad2-72d6859eeb38/global-pull-secret-syncer/0.log" May 26 15:50:17.681538 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:17.681503 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-qcrp6_eda06c33-9985-46af-ac9d-d006488f30ca/konnectivity-agent/0.log" May 26 15:50:17.736248 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:17.736223 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-198.ec2.internal_10db4583199c612d03dabeed2e12e039/haproxy/0.log" May 26 15:50:20.954426 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:20.954188 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-54fd6bc5c4-k8flp_a16892d4-26db-4f38-a60b-7a92ae470ae1/kube-state-metrics/0.log" May 26 15:50:20.978181 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:20.978141 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-54fd6bc5c4-k8flp_a16892d4-26db-4f38-a60b-7a92ae470ae1/kube-rbac-proxy-main/0.log" May 26 15:50:21.005246 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:21.005216 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-54fd6bc5c4-k8flp_a16892d4-26db-4f38-a60b-7a92ae470ae1/kube-rbac-proxy-self/0.log" May 26 15:50:21.040820 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:21.040789 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-84b5bf67c7-925fq_a4426673-24c2-480f-9a9d-6ec20a2f1ba0/metrics-server/0.log" May 26 15:50:21.070200 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:21.070171 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-665c9c8584-5gppl_812f69f2-f125-432a-8244-920534eba136/monitoring-plugin/0.log" May 26 15:50:21.187355 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:21.187320 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-j87gv_08d8a00f-3b89-4724-bce7-d3a7f395346b/node-exporter/0.log" May 26 15:50:21.223663 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:21.223575 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-j87gv_08d8a00f-3b89-4724-bce7-d3a7f395346b/kube-rbac-proxy/0.log" May 26 15:50:21.262009 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:21.261982 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-j87gv_08d8a00f-3b89-4724-bce7-d3a7f395346b/init-textfile/0.log" May 26 15:50:23.113372 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:23.113333 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-d4d8f94f9-mkzgw_a323cf3c-1a25-4e72-a4ac-1a3d45ad5a41/networking-console-plugin/0.log" May 26 15:50:23.512374 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:23.512349 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-qd2tx_dd56d3c2-38ff-4bd8-9893-6b643455a976/console-operator/2.log" May 26 15:50:23.517418 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:23.517396 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-qd2tx_dd56d3c2-38ff-4bd8-9893-6b643455a976/console-operator/3.log" May 26 15:50:23.895179 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:23.895115 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-784b5459d6-54mst_d4b31c58-7803-4692-9aa5-786b3b974e31/console/0.log" May 26 15:50:23.935247 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:23.935217 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-57bd96b888-krxkd_1e3a0692-61b5-4201-beaf-ac3df33be93b/download-server/0.log" May 26 15:50:23.950810 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:23.950774 2547 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8ln6p/perf-node-gather-daemonset-jdxll"] May 26 15:50:23.955784 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:23.955756 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8ln6p/perf-node-gather-daemonset-jdxll" May 26 15:50:23.962553 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:23.962523 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8ln6p/perf-node-gather-daemonset-jdxll"] May 26 15:50:24.023041 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:24.020909 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fb2a4e42-cbab-423b-93e8-4d1a70841cd0-sys\") pod \"perf-node-gather-daemonset-jdxll\" (UID: \"fb2a4e42-cbab-423b-93e8-4d1a70841cd0\") " pod="openshift-must-gather-8ln6p/perf-node-gather-daemonset-jdxll" May 26 15:50:24.023041 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:24.020954 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zhrf\" (UniqueName: \"kubernetes.io/projected/fb2a4e42-cbab-423b-93e8-4d1a70841cd0-kube-api-access-8zhrf\") pod \"perf-node-gather-daemonset-jdxll\" (UID: \"fb2a4e42-cbab-423b-93e8-4d1a70841cd0\") " pod="openshift-must-gather-8ln6p/perf-node-gather-daemonset-jdxll" May 26 15:50:24.023041 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:24.021116 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fb2a4e42-cbab-423b-93e8-4d1a70841cd0-proc\") pod \"perf-node-gather-daemonset-jdxll\" (UID: \"fb2a4e42-cbab-423b-93e8-4d1a70841cd0\") " pod="openshift-must-gather-8ln6p/perf-node-gather-daemonset-jdxll" May 26 15:50:24.023041 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:24.021161 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fb2a4e42-cbab-423b-93e8-4d1a70841cd0-lib-modules\") pod \"perf-node-gather-daemonset-jdxll\" (UID: \"fb2a4e42-cbab-423b-93e8-4d1a70841cd0\") " pod="openshift-must-gather-8ln6p/perf-node-gather-daemonset-jdxll" May 26 15:50:24.023041 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:24.021300 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fb2a4e42-cbab-423b-93e8-4d1a70841cd0-podres\") pod \"perf-node-gather-daemonset-jdxll\" (UID: \"fb2a4e42-cbab-423b-93e8-4d1a70841cd0\") " pod="openshift-must-gather-8ln6p/perf-node-gather-daemonset-jdxll" May 26 15:50:24.121818 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:24.121782 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fb2a4e42-cbab-423b-93e8-4d1a70841cd0-proc\") pod \"perf-node-gather-daemonset-jdxll\" (UID: \"fb2a4e42-cbab-423b-93e8-4d1a70841cd0\") " pod="openshift-must-gather-8ln6p/perf-node-gather-daemonset-jdxll" May 26 15:50:24.121818 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:24.121819 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fb2a4e42-cbab-423b-93e8-4d1a70841cd0-lib-modules\") pod \"perf-node-gather-daemonset-jdxll\" (UID: \"fb2a4e42-cbab-423b-93e8-4d1a70841cd0\") " pod="openshift-must-gather-8ln6p/perf-node-gather-daemonset-jdxll" May 26 15:50:24.122256 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:24.121859 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fb2a4e42-cbab-423b-93e8-4d1a70841cd0-podres\") pod \"perf-node-gather-daemonset-jdxll\" (UID: \"fb2a4e42-cbab-423b-93e8-4d1a70841cd0\") " pod="openshift-must-gather-8ln6p/perf-node-gather-daemonset-jdxll" May 26 15:50:24.122256 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:24.121892 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fb2a4e42-cbab-423b-93e8-4d1a70841cd0-sys\") pod \"perf-node-gather-daemonset-jdxll\" (UID: \"fb2a4e42-cbab-423b-93e8-4d1a70841cd0\") " pod="openshift-must-gather-8ln6p/perf-node-gather-daemonset-jdxll" May 26 15:50:24.122256 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:24.121908 2547 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8zhrf\" (UniqueName: \"kubernetes.io/projected/fb2a4e42-cbab-423b-93e8-4d1a70841cd0-kube-api-access-8zhrf\") pod \"perf-node-gather-daemonset-jdxll\" (UID: \"fb2a4e42-cbab-423b-93e8-4d1a70841cd0\") " pod="openshift-must-gather-8ln6p/perf-node-gather-daemonset-jdxll" May 26 15:50:24.122256 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:24.121948 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fb2a4e42-cbab-423b-93e8-4d1a70841cd0-proc\") pod \"perf-node-gather-daemonset-jdxll\" (UID: \"fb2a4e42-cbab-423b-93e8-4d1a70841cd0\") " pod="openshift-must-gather-8ln6p/perf-node-gather-daemonset-jdxll" May 26 15:50:24.122256 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:24.122039 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fb2a4e42-cbab-423b-93e8-4d1a70841cd0-lib-modules\") pod \"perf-node-gather-daemonset-jdxll\" (UID: \"fb2a4e42-cbab-423b-93e8-4d1a70841cd0\") " pod="openshift-must-gather-8ln6p/perf-node-gather-daemonset-jdxll" May 26 15:50:24.122256 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:24.122043 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fb2a4e42-cbab-423b-93e8-4d1a70841cd0-podres\") pod \"perf-node-gather-daemonset-jdxll\" (UID: \"fb2a4e42-cbab-423b-93e8-4d1a70841cd0\") " pod="openshift-must-gather-8ln6p/perf-node-gather-daemonset-jdxll" May 26 15:50:24.122256 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:24.122039 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fb2a4e42-cbab-423b-93e8-4d1a70841cd0-sys\") pod \"perf-node-gather-daemonset-jdxll\" (UID: \"fb2a4e42-cbab-423b-93e8-4d1a70841cd0\") " pod="openshift-must-gather-8ln6p/perf-node-gather-daemonset-jdxll" May 26 15:50:24.131087 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:24.131046 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zhrf\" (UniqueName: \"kubernetes.io/projected/fb2a4e42-cbab-423b-93e8-4d1a70841cd0-kube-api-access-8zhrf\") pod \"perf-node-gather-daemonset-jdxll\" (UID: \"fb2a4e42-cbab-423b-93e8-4d1a70841cd0\") " pod="openshift-must-gather-8ln6p/perf-node-gather-daemonset-jdxll" May 26 15:50:24.268939 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:24.268903 2547 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8ln6p/perf-node-gather-daemonset-jdxll" May 26 15:50:24.341849 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:24.341810 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5cf8c89d64-klhzf_463f7c87-7dc8-4b09-a855-6565923f3a8d/volume-data-source-validator/0.log" May 26 15:50:24.420109 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:24.420073 2547 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8ln6p/perf-node-gather-daemonset-jdxll"] May 26 15:50:25.043751 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:25.043725 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dzpbp_91f902eb-040a-46bf-a271-e75d7e98479f/dns/0.log" May 26 15:50:25.068450 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:25.068421 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dzpbp_91f902eb-040a-46bf-a271-e75d7e98479f/kube-rbac-proxy/0.log" May 26 15:50:25.148339 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:25.148314 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ddnhg_a1723cfd-d190-4d26-9964-3c378ba0cd3b/dns-node-resolver/0.log" May 26 15:50:25.187930 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:25.187894 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8ln6p/perf-node-gather-daemonset-jdxll" event={"ID":"fb2a4e42-cbab-423b-93e8-4d1a70841cd0","Type":"ContainerStarted","Data":"c51ba605d82c6c1a093ce11739df4fc7f56d79b8f6e82eb097913ef67eb32bdf"} May 26 15:50:25.187930 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:25.187933 2547 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8ln6p/perf-node-gather-daemonset-jdxll" event={"ID":"fb2a4e42-cbab-423b-93e8-4d1a70841cd0","Type":"ContainerStarted","Data":"3e44953ef57e2359aecc9bf913e2e6523ea984811153e6ceba35c6be9b496d16"} May 26 15:50:25.188147 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:25.188029 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8ln6p/perf-node-gather-daemonset-jdxll" May 26 15:50:25.206355 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:25.206310 2547 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-8ln6p/perf-node-gather-daemonset-jdxll" podStartSLOduration=2.206295206 podStartE2EDuration="2.206295206s" podCreationTimestamp="2026-05-26 15:50:23 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 15:50:25.205187534 +0000 UTC m=+1773.931862456" watchObservedRunningTime="2026-05-26 15:50:25.206295206 +0000 UTC m=+1773.932970126" May 26 15:50:25.601795 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:25.601757 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6b645d86d4-g9z92_125e1723-0c7c-4afd-b344-a05746f62eb7/registry/0.log" May 26 15:50:25.623766 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:25.623730 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2r56q_4f6155d9-d73c-483b-ade7-8eecd0e99883/node-ca/0.log" May 26 15:50:26.444017 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:26.443993 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-69965b4854-82hkh_f9b2322f-cdfd-43b7-8481-fe9c08706d91/router/0.log" May 26 15:50:26.761808 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:26.761739 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8sxg9_7b1e268b-9140-4f73-8abf-c1c30fc706bc/serve-healthcheck-canary/0.log" May 26 15:50:27.152564 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:27.152498 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-78bdfb888c-ht62j_90be60c1-e9f6-476d-9a27-f050b27cff15/insights-operator/1.log" May 26 15:50:27.152564 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:27.152504 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-78bdfb888c-ht62j_90be60c1-e9f6-476d-9a27-f050b27cff15/insights-operator/0.log" May 26 15:50:27.247291 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:27.247259 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mkhbh_12d2a1e5-82ec-4b4d-9eb1-fac4406e2005/kube-rbac-proxy/0.log" May 26 15:50:27.272632 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:27.272609 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mkhbh_12d2a1e5-82ec-4b4d-9eb1-fac4406e2005/exporter/0.log" May 26 15:50:27.297667 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:27.297640 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mkhbh_12d2a1e5-82ec-4b4d-9eb1-fac4406e2005/extractor/0.log" May 26 15:50:31.204421 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:31.204394 2547 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8ln6p/perf-node-gather-daemonset-jdxll" May 26 15:50:33.029756 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:33.029680 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-66nvf_f5579674-2eaf-44d9-b407-585c871463c7/kube-multus/0.log" May 26 15:50:33.273890 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:33.273863 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-knssg_c4d73d69-aaad-48c5-bfbc-75ac14352253/kube-multus-additional-cni-plugins/0.log" May 26 15:50:33.301760 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:33.301673 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-knssg_c4d73d69-aaad-48c5-bfbc-75ac14352253/egress-router-binary-copy/0.log" May 26 15:50:33.326787 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:33.326761 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-knssg_c4d73d69-aaad-48c5-bfbc-75ac14352253/cni-plugins/0.log" May 26 15:50:33.350645 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:33.350624 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-knssg_c4d73d69-aaad-48c5-bfbc-75ac14352253/bond-cni-plugin/0.log" May 26 15:50:33.376759 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:33.376738 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-knssg_c4d73d69-aaad-48c5-bfbc-75ac14352253/routeoverride-cni/0.log" May 26 15:50:33.403225 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:33.403200 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-knssg_c4d73d69-aaad-48c5-bfbc-75ac14352253/whereabouts-cni-bincopy/0.log" May 26 15:50:33.425904 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:33.425877 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-knssg_c4d73d69-aaad-48c5-bfbc-75ac14352253/whereabouts-cni/0.log" May 26 15:50:33.698654 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:33.698628 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-5xh82_fac6c416-4fef-4b1b-846f-ea63a14c9efe/network-metrics-daemon/0.log" May 26 15:50:33.721309 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:33.721286 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-5xh82_fac6c416-4fef-4b1b-846f-ea63a14c9efe/kube-rbac-proxy/0.log" May 26 15:50:35.297582 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:35.297552 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vfdvw_beeec92c-3a10-4520-a63d-984e02c0bbcf/ovn-controller/0.log" May 26 15:50:35.328244 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:35.328217 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vfdvw_beeec92c-3a10-4520-a63d-984e02c0bbcf/ovn-acl-logging/0.log" May 26 15:50:35.351831 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:35.351804 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vfdvw_beeec92c-3a10-4520-a63d-984e02c0bbcf/kube-rbac-proxy-node/0.log" May 26 15:50:35.374794 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:35.374770 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vfdvw_beeec92c-3a10-4520-a63d-984e02c0bbcf/kube-rbac-proxy-ovn-metrics/0.log" May 26 15:50:35.400932 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:35.400901 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vfdvw_beeec92c-3a10-4520-a63d-984e02c0bbcf/northd/0.log" May 26 15:50:35.426937 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:35.426910 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vfdvw_beeec92c-3a10-4520-a63d-984e02c0bbcf/nbdb/0.log" May 26 15:50:35.452080 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:35.452032 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vfdvw_beeec92c-3a10-4520-a63d-984e02c0bbcf/sbdb/0.log" May 26 15:50:35.570287 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:35.570214 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vfdvw_beeec92c-3a10-4520-a63d-984e02c0bbcf/ovnkube-controller/0.log" May 26 15:50:36.537003 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:36.536932 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-7f44b6dd87-j9222_ce0ae8cb-e08f-491d-a674-b60312b575d7/check-endpoints/0.log" May 26 15:50:36.617015 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:36.616981 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-kkq9m_033a1d5a-e393-4e8c-aa09-3786637f5a7e/network-check-target-container/0.log" May 26 15:50:37.494202 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:37.494174 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-6txjd_a402afb6-d0bc-4139-8153-4e1463fb49cc/iptables-alerter/0.log" May 26 15:50:38.232897 ip-10-0-139-198 kubenswrapper[2547]: I0526 15:50:38.232871 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-spgwf_72e9400c-a618-45b9-8b31-3fa6b713a9a2/tuned/0.log"