May 28 15:07:28.987761 ip-10-0-140-141 systemd[1]: Starting Kubernetes Kubelet... May 28 15:07:29.454604 ip-10-0-140-141 kubenswrapper[2559]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 28 15:07:29.454604 ip-10-0-140-141 kubenswrapper[2559]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. May 28 15:07:29.454604 ip-10-0-140-141 kubenswrapper[2559]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 28 15:07:29.454604 ip-10-0-140-141 kubenswrapper[2559]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. May 28 15:07:29.454604 ip-10-0-140-141 kubenswrapper[2559]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 28 15:07:29.457875 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.457783 2559 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" May 28 15:07:29.462164 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462149 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores May 28 15:07:29.462164 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462164 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 28 15:07:29.462227 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462169 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 28 15:07:29.462227 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462172 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 28 15:07:29.462227 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462175 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 28 15:07:29.462227 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462178 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 28 15:07:29.462227 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462181 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 28 15:07:29.462227 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462184 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 28 15:07:29.462227 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462187 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 28 15:07:29.462227 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462189 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 28 15:07:29.462227 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462192 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 28 15:07:29.462227 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462195 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 28 15:07:29.462227 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462197 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 28 15:07:29.462227 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462200 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 28 15:07:29.462227 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462202 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 28 15:07:29.462227 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462205 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 28 15:07:29.462227 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462207 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 28 15:07:29.462227 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462210 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 28 15:07:29.462227 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462212 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 28 15:07:29.462227 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462222 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 28 15:07:29.462227 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462225 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 28 15:07:29.462227 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462228 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 28 15:07:29.462754 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462231 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 28 15:07:29.462754 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462234 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 28 15:07:29.462754 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462237 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 28 15:07:29.462754 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462240 2559 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 28 15:07:29.462754 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462243 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 28 15:07:29.462754 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462246 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 28 15:07:29.462754 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462248 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 28 15:07:29.462754 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462251 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 28 15:07:29.462754 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462253 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 28 15:07:29.462754 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462256 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 28 15:07:29.462754 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462259 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 28 15:07:29.462754 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462261 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 28 15:07:29.462754 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462263 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability May 28 15:07:29.462754 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462266 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 28 15:07:29.462754 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462268 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 28 15:07:29.462754 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462271 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 28 15:07:29.462754 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462273 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 28 15:07:29.462754 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462276 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 28 15:07:29.462754 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462278 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 28 15:07:29.463220 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462282 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 28 15:07:29.463220 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462284 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 28 15:07:29.463220 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462289 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 28 15:07:29.463220 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462292 2559 feature_gate.go:328] unrecognized feature gate: DualReplica May 28 15:07:29.463220 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462296 2559 feature_gate.go:328] unrecognized feature gate: NewOLM May 28 15:07:29.463220 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462299 2559 feature_gate.go:328] unrecognized feature gate: Example2 May 28 15:07:29.463220 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462301 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 28 15:07:29.463220 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462304 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 28 15:07:29.463220 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462307 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 28 15:07:29.463220 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462310 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 28 15:07:29.463220 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462312 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 28 15:07:29.463220 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462315 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 28 15:07:29.463220 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462317 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 28 15:07:29.463220 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462320 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 28 15:07:29.463220 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462323 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 28 15:07:29.463220 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462326 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 28 15:07:29.463220 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462330 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 28 15:07:29.463220 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462332 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 28 15:07:29.463220 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462335 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 28 15:07:29.463697 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462337 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 28 15:07:29.463697 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462340 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 28 15:07:29.463697 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462342 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 28 15:07:29.463697 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462345 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 28 15:07:29.463697 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462348 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 28 15:07:29.463697 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462350 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 28 15:07:29.463697 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462353 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 28 15:07:29.463697 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462355 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 28 15:07:29.463697 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462358 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 28 15:07:29.463697 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462360 2559 feature_gate.go:328] unrecognized feature gate: Example May 28 15:07:29.463697 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462363 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 28 15:07:29.463697 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462365 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 28 15:07:29.463697 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462368 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 28 15:07:29.463697 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462371 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 28 15:07:29.463697 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462374 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 28 15:07:29.463697 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462376 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 28 15:07:29.463697 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462379 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 28 15:07:29.463697 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462381 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 28 15:07:29.463697 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462384 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 28 15:07:29.463697 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462388 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 28 15:07:29.464182 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462392 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 28 15:07:29.464182 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462396 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 28 15:07:29.464182 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462399 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 28 15:07:29.464182 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462402 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 28 15:07:29.464182 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462404 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages May 28 15:07:29.464182 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462407 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 28 15:07:29.464182 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462780 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 28 15:07:29.464182 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462786 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 28 15:07:29.464182 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462789 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 28 15:07:29.464182 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462792 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 28 15:07:29.464182 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462795 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 28 15:07:29.464182 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462798 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 28 15:07:29.464182 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462801 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 28 15:07:29.464182 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462803 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 28 15:07:29.464182 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462806 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages May 28 15:07:29.464182 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462809 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 28 15:07:29.464182 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462811 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 28 15:07:29.464182 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462814 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 28 15:07:29.464182 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462816 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 28 15:07:29.464650 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462819 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 28 15:07:29.464650 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462821 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 28 15:07:29.464650 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462824 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 28 15:07:29.464650 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462826 2559 feature_gate.go:328] unrecognized feature gate: Example2 May 28 15:07:29.464650 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462829 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 28 15:07:29.464650 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462832 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 28 15:07:29.464650 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462836 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 28 15:07:29.464650 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462840 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 28 15:07:29.464650 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462843 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 28 15:07:29.464650 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462846 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 28 15:07:29.464650 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462849 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 28 15:07:29.464650 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462852 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 28 15:07:29.464650 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462855 2559 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 28 15:07:29.464650 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462857 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 28 15:07:29.464650 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462860 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 28 15:07:29.464650 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462863 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 28 15:07:29.464650 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462866 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 28 15:07:29.464650 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462869 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 28 15:07:29.464650 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462872 2559 feature_gate.go:328] unrecognized feature gate: DualReplica May 28 15:07:29.465159 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462875 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 28 15:07:29.465159 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462878 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 28 15:07:29.465159 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462880 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 28 15:07:29.465159 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462883 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 28 15:07:29.465159 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462886 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 28 15:07:29.465159 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462888 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 28 15:07:29.465159 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462891 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 28 15:07:29.465159 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462893 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 28 15:07:29.465159 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462896 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 28 15:07:29.465159 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462898 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 28 15:07:29.465159 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462901 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 28 15:07:29.465159 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462903 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 28 15:07:29.465159 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462906 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 28 15:07:29.465159 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462909 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 28 15:07:29.465159 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462911 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 28 15:07:29.465159 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462914 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 28 15:07:29.465159 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462916 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 28 15:07:29.465159 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462918 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 28 15:07:29.465159 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462921 2559 feature_gate.go:328] unrecognized feature gate: Example May 28 15:07:29.465159 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462925 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 28 15:07:29.465675 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462928 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 28 15:07:29.465675 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462930 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 28 15:07:29.465675 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462933 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 28 15:07:29.465675 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462935 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 28 15:07:29.465675 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462938 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 28 15:07:29.465675 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462940 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 28 15:07:29.465675 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462943 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 28 15:07:29.465675 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462946 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 28 15:07:29.465675 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462948 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 28 15:07:29.465675 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462951 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 28 15:07:29.465675 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462954 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 28 15:07:29.465675 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462956 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 28 15:07:29.465675 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462959 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability May 28 15:07:29.465675 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462962 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 28 15:07:29.465675 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462966 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 28 15:07:29.465675 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462968 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 28 15:07:29.465675 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462971 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 28 15:07:29.465675 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462975 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 28 15:07:29.465675 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462978 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 28 15:07:29.465675 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462981 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 28 15:07:29.466252 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462984 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 28 15:07:29.466252 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462987 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 28 15:07:29.466252 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462989 2559 feature_gate.go:328] unrecognized feature gate: NewOLM May 28 15:07:29.466252 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462992 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores May 28 15:07:29.466252 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462994 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 28 15:07:29.466252 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.462997 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 28 15:07:29.466252 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.463000 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 28 15:07:29.466252 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.463003 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 28 15:07:29.466252 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.463007 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 28 15:07:29.466252 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.463010 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 28 15:07:29.466252 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.463013 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 28 15:07:29.466252 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.463015 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 28 15:07:29.466252 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.463018 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 28 15:07:29.466252 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.463021 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 28 15:07:29.466252 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463756 2559 flags.go:64] FLAG: --address="0.0.0.0" May 28 15:07:29.466252 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463766 2559 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" May 28 15:07:29.466252 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463772 2559 flags.go:64] FLAG: --anonymous-auth="true" May 28 15:07:29.466252 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463777 2559 flags.go:64] FLAG: --application-metrics-count-limit="100" May 28 15:07:29.466252 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463781 2559 flags.go:64] FLAG: --authentication-token-webhook="false" May 28 15:07:29.466252 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463784 2559 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" May 28 15:07:29.466252 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463789 2559 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" May 28 15:07:29.466774 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463793 2559 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" May 28 15:07:29.466774 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463796 2559 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" May 28 15:07:29.466774 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463799 2559 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" May 28 15:07:29.466774 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463804 2559 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" May 28 15:07:29.466774 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463807 2559 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" May 28 15:07:29.466774 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463811 2559 flags.go:64] FLAG: --cgroup-driver="cgroupfs" May 28 15:07:29.466774 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463814 2559 flags.go:64] FLAG: --cgroup-root="" May 28 15:07:29.466774 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463816 2559 flags.go:64] FLAG: --cgroups-per-qos="true" May 28 15:07:29.466774 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463820 2559 flags.go:64] FLAG: --client-ca-file="" May 28 15:07:29.466774 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463823 2559 flags.go:64] FLAG: --cloud-config="" May 28 15:07:29.466774 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463825 2559 flags.go:64] FLAG: --cloud-provider="external" May 28 15:07:29.466774 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463828 2559 flags.go:64] FLAG: --cluster-dns="[]" May 28 15:07:29.466774 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463833 2559 flags.go:64] FLAG: --cluster-domain="" May 28 15:07:29.466774 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463836 2559 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" May 28 15:07:29.466774 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463839 2559 flags.go:64] FLAG: --config-dir="" May 28 15:07:29.466774 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463843 2559 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" May 28 15:07:29.466774 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463846 2559 flags.go:64] FLAG: --container-log-max-files="5" May 28 15:07:29.466774 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463850 2559 flags.go:64] FLAG: --container-log-max-size="10Mi" May 28 15:07:29.466774 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463853 2559 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" May 28 15:07:29.466774 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463856 2559 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" May 28 15:07:29.466774 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463860 2559 flags.go:64] FLAG: --containerd-namespace="k8s.io" May 28 15:07:29.466774 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463864 2559 flags.go:64] FLAG: --contention-profiling="false" May 28 15:07:29.466774 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463866 2559 flags.go:64] FLAG: --cpu-cfs-quota="true" May 28 15:07:29.466774 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463869 2559 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" May 28 15:07:29.467358 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463879 2559 flags.go:64] FLAG: --cpu-manager-policy="none" May 28 15:07:29.467358 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463882 2559 flags.go:64] FLAG: --cpu-manager-policy-options="" May 28 15:07:29.467358 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463891 2559 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" May 28 15:07:29.467358 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463894 2559 flags.go:64] FLAG: --enable-controller-attach-detach="true" May 28 15:07:29.467358 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463901 2559 flags.go:64] FLAG: --enable-debugging-handlers="true" May 28 15:07:29.467358 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463904 2559 flags.go:64] FLAG: --enable-load-reader="false" May 28 15:07:29.467358 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463907 2559 flags.go:64] FLAG: --enable-server="true" May 28 15:07:29.467358 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463910 2559 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" May 28 15:07:29.467358 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463914 2559 flags.go:64] FLAG: --event-burst="100" May 28 15:07:29.467358 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463917 2559 flags.go:64] FLAG: --event-qps="50" May 28 15:07:29.467358 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463920 2559 flags.go:64] FLAG: --event-storage-age-limit="default=0" May 28 15:07:29.467358 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463923 2559 flags.go:64] FLAG: --event-storage-event-limit="default=0" May 28 15:07:29.467358 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463926 2559 flags.go:64] FLAG: --eviction-hard="" May 28 15:07:29.467358 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463930 2559 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" May 28 15:07:29.467358 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463933 2559 flags.go:64] FLAG: --eviction-minimum-reclaim="" May 28 15:07:29.467358 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463936 2559 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" May 28 15:07:29.467358 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463939 2559 flags.go:64] FLAG: --eviction-soft="" May 28 15:07:29.467358 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463942 2559 flags.go:64] FLAG: --eviction-soft-grace-period="" May 28 15:07:29.467358 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463944 2559 flags.go:64] FLAG: --exit-on-lock-contention="false" May 28 15:07:29.467358 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463948 2559 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" May 28 15:07:29.467358 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463951 2559 flags.go:64] FLAG: --experimental-mounter-path="" May 28 15:07:29.467358 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463954 2559 flags.go:64] FLAG: --fail-cgroupv1="false" May 28 15:07:29.467358 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463957 2559 flags.go:64] FLAG: --fail-swap-on="true" May 28 15:07:29.467358 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463960 2559 flags.go:64] FLAG: --feature-gates="" May 28 15:07:29.467358 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463963 2559 flags.go:64] FLAG: --file-check-frequency="20s" May 28 15:07:29.467968 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463967 2559 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" May 28 15:07:29.467968 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463970 2559 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" May 28 15:07:29.467968 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463973 2559 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" May 28 15:07:29.467968 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463977 2559 flags.go:64] FLAG: --healthz-port="10248" May 28 15:07:29.467968 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463980 2559 flags.go:64] FLAG: --help="false" May 28 15:07:29.467968 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463982 2559 flags.go:64] FLAG: --hostname-override="ip-10-0-140-141.ec2.internal" May 28 15:07:29.467968 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463985 2559 flags.go:64] FLAG: --housekeeping-interval="10s" May 28 15:07:29.467968 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463988 2559 flags.go:64] FLAG: --http-check-frequency="20s" May 28 15:07:29.467968 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463992 2559 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" May 28 15:07:29.467968 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463995 2559 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" May 28 15:07:29.467968 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.463998 2559 flags.go:64] FLAG: --image-gc-high-threshold="85" May 28 15:07:29.467968 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464002 2559 flags.go:64] FLAG: --image-gc-low-threshold="80" May 28 15:07:29.467968 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464005 2559 flags.go:64] FLAG: --image-service-endpoint="" May 28 15:07:29.467968 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464008 2559 flags.go:64] FLAG: --kernel-memcg-notification="false" May 28 15:07:29.467968 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464011 2559 flags.go:64] FLAG: --kube-api-burst="100" May 28 15:07:29.467968 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464014 2559 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" May 28 15:07:29.467968 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464017 2559 flags.go:64] FLAG: --kube-api-qps="50" May 28 15:07:29.467968 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464020 2559 flags.go:64] FLAG: --kube-reserved="" May 28 15:07:29.467968 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464023 2559 flags.go:64] FLAG: --kube-reserved-cgroup="" May 28 15:07:29.467968 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464026 2559 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" May 28 15:07:29.467968 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464029 2559 flags.go:64] FLAG: --kubelet-cgroups="" May 28 15:07:29.467968 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464031 2559 flags.go:64] FLAG: --local-storage-capacity-isolation="true" May 28 15:07:29.467968 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464034 2559 flags.go:64] FLAG: --lock-file="" May 28 15:07:29.467968 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464037 2559 flags.go:64] FLAG: --log-cadvisor-usage="false" May 28 15:07:29.468566 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464040 2559 flags.go:64] FLAG: --log-flush-frequency="5s" May 28 15:07:29.468566 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464043 2559 flags.go:64] FLAG: --log-json-info-buffer-size="0" May 28 15:07:29.468566 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464050 2559 flags.go:64] FLAG: --log-json-split-stream="false" May 28 15:07:29.468566 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464053 2559 flags.go:64] FLAG: --log-text-info-buffer-size="0" May 28 15:07:29.468566 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464056 2559 flags.go:64] FLAG: --log-text-split-stream="false" May 28 15:07:29.468566 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464059 2559 flags.go:64] FLAG: --logging-format="text" May 28 15:07:29.468566 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464062 2559 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" May 28 15:07:29.468566 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464065 2559 flags.go:64] FLAG: --make-iptables-util-chains="true" May 28 15:07:29.468566 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464068 2559 flags.go:64] FLAG: --manifest-url="" May 28 15:07:29.468566 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464070 2559 flags.go:64] FLAG: --manifest-url-header="" May 28 15:07:29.468566 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464075 2559 flags.go:64] FLAG: --max-housekeeping-interval="15s" May 28 15:07:29.468566 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464078 2559 flags.go:64] FLAG: --max-open-files="1000000" May 28 15:07:29.468566 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464082 2559 flags.go:64] FLAG: --max-pods="110" May 28 15:07:29.468566 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464085 2559 flags.go:64] FLAG: --maximum-dead-containers="-1" May 28 15:07:29.468566 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464088 2559 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" May 28 15:07:29.468566 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464091 2559 flags.go:64] FLAG: --memory-manager-policy="None" May 28 15:07:29.468566 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464094 2559 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" May 28 15:07:29.468566 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464097 2559 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" May 28 15:07:29.468566 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464100 2559 flags.go:64] FLAG: --node-ip="0.0.0.0" May 28 15:07:29.468566 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464105 2559 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" May 28 15:07:29.468566 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464113 2559 flags.go:64] FLAG: --node-status-max-images="50" May 28 15:07:29.468566 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464117 2559 flags.go:64] FLAG: --node-status-update-frequency="10s" May 28 15:07:29.468566 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464120 2559 flags.go:64] FLAG: --oom-score-adj="-999" May 28 15:07:29.468566 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464122 2559 flags.go:64] FLAG: --pod-cidr="" May 28 15:07:29.469157 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464125 2559 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afe88671bf163c42b467d159dc6f0b784dbe28b1ed44e171edc9c06fd99cc6fc" May 28 15:07:29.469157 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464131 2559 flags.go:64] FLAG: --pod-manifest-path="" May 28 15:07:29.469157 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464134 2559 flags.go:64] FLAG: --pod-max-pids="-1" May 28 15:07:29.469157 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464137 2559 flags.go:64] FLAG: --pods-per-core="0" May 28 15:07:29.469157 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464140 2559 flags.go:64] FLAG: --port="10250" May 28 15:07:29.469157 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464143 2559 flags.go:64] FLAG: --protect-kernel-defaults="false" May 28 15:07:29.469157 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464146 2559 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-047c3c194314588b6" May 28 15:07:29.469157 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464149 2559 flags.go:64] FLAG: --qos-reserved="" May 28 15:07:29.469157 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464152 2559 flags.go:64] FLAG: --read-only-port="10255" May 28 15:07:29.469157 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464154 2559 flags.go:64] FLAG: --register-node="true" May 28 15:07:29.469157 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464158 2559 flags.go:64] FLAG: --register-schedulable="true" May 28 15:07:29.469157 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464161 2559 flags.go:64] FLAG: --register-with-taints="" May 28 15:07:29.469157 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464165 2559 flags.go:64] FLAG: --registry-burst="10" May 28 15:07:29.469157 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464168 2559 flags.go:64] FLAG: --registry-qps="5" May 28 15:07:29.469157 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464171 2559 flags.go:64] FLAG: --reserved-cpus="" May 28 15:07:29.469157 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464174 2559 flags.go:64] FLAG: --reserved-memory="" May 28 15:07:29.469157 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464178 2559 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" May 28 15:07:29.469157 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464181 2559 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" May 28 15:07:29.469157 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464184 2559 flags.go:64] FLAG: --rotate-certificates="false" May 28 15:07:29.469157 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464187 2559 flags.go:64] FLAG: --rotate-server-certificates="false" May 28 15:07:29.469157 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464190 2559 flags.go:64] FLAG: --runonce="false" May 28 15:07:29.469157 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464193 2559 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" May 28 15:07:29.469157 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464196 2559 flags.go:64] FLAG: --runtime-request-timeout="2m0s" May 28 15:07:29.469157 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464199 2559 flags.go:64] FLAG: --seccomp-default="false" May 28 15:07:29.469157 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464202 2559 flags.go:64] FLAG: --serialize-image-pulls="true" May 28 15:07:29.469771 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464205 2559 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" May 28 15:07:29.469771 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464209 2559 flags.go:64] FLAG: --storage-driver-db="cadvisor" May 28 15:07:29.469771 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464212 2559 flags.go:64] FLAG: --storage-driver-host="localhost:8086" May 28 15:07:29.469771 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464217 2559 flags.go:64] FLAG: --storage-driver-password="root" May 28 15:07:29.469771 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464220 2559 flags.go:64] FLAG: --storage-driver-secure="false" May 28 15:07:29.469771 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464223 2559 flags.go:64] FLAG: --storage-driver-table="stats" May 28 15:07:29.469771 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464226 2559 flags.go:64] FLAG: --storage-driver-user="root" May 28 15:07:29.469771 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464229 2559 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" May 28 15:07:29.469771 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464231 2559 flags.go:64] FLAG: --sync-frequency="1m0s" May 28 15:07:29.469771 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464234 2559 flags.go:64] FLAG: --system-cgroups="" May 28 15:07:29.469771 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464237 2559 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" May 28 15:07:29.469771 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464242 2559 flags.go:64] FLAG: --system-reserved-cgroup="" May 28 15:07:29.469771 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464245 2559 flags.go:64] FLAG: --tls-cert-file="" May 28 15:07:29.469771 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464248 2559 flags.go:64] FLAG: --tls-cipher-suites="[]" May 28 15:07:29.469771 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464252 2559 flags.go:64] FLAG: --tls-min-version="" May 28 15:07:29.469771 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464255 2559 flags.go:64] FLAG: --tls-private-key-file="" May 28 15:07:29.469771 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464258 2559 flags.go:64] FLAG: --topology-manager-policy="none" May 28 15:07:29.469771 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464262 2559 flags.go:64] FLAG: --topology-manager-policy-options="" May 28 15:07:29.469771 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464265 2559 flags.go:64] FLAG: --topology-manager-scope="container" May 28 15:07:29.469771 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464268 2559 flags.go:64] FLAG: --v="2" May 28 15:07:29.469771 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464272 2559 flags.go:64] FLAG: --version="false" May 28 15:07:29.469771 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464276 2559 flags.go:64] FLAG: --vmodule="" May 28 15:07:29.469771 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464280 2559 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" May 28 15:07:29.469771 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.464283 2559 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" May 28 15:07:29.469771 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464371 2559 feature_gate.go:328] unrecognized feature gate: Example May 28 15:07:29.470381 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464375 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 28 15:07:29.470381 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464379 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores May 28 15:07:29.470381 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464381 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 28 15:07:29.470381 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464384 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 28 15:07:29.470381 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464387 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability May 28 15:07:29.470381 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464389 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 28 15:07:29.470381 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464392 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 28 15:07:29.470381 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464395 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 28 15:07:29.470381 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464397 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 28 15:07:29.470381 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464401 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 28 15:07:29.470381 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464404 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 28 15:07:29.470381 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464407 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 28 15:07:29.470381 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464411 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 28 15:07:29.470381 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464415 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 28 15:07:29.470381 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464418 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 28 15:07:29.470381 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464420 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 28 15:07:29.470381 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464423 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 28 15:07:29.470381 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464425 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 28 15:07:29.470381 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464428 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 28 15:07:29.471199 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464430 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 28 15:07:29.471199 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464433 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 28 15:07:29.471199 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464435 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 28 15:07:29.471199 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464438 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 28 15:07:29.471199 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464440 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 28 15:07:29.471199 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464445 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 28 15:07:29.471199 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464447 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 28 15:07:29.471199 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464450 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 28 15:07:29.471199 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464453 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 28 15:07:29.471199 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464455 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 28 15:07:29.471199 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464458 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 28 15:07:29.471199 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464460 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 28 15:07:29.471199 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464463 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 28 15:07:29.471199 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464466 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 28 15:07:29.471199 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464468 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 28 15:07:29.471199 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464471 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 28 15:07:29.471199 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464473 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 28 15:07:29.471199 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464476 2559 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 28 15:07:29.471199 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464493 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 28 15:07:29.472061 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464496 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 28 15:07:29.472061 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464498 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 28 15:07:29.472061 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464501 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 28 15:07:29.472061 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464505 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 28 15:07:29.472061 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464510 2559 feature_gate.go:328] unrecognized feature gate: NewOLM May 28 15:07:29.472061 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464513 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 28 15:07:29.472061 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464516 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 28 15:07:29.472061 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464518 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 28 15:07:29.472061 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464522 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 28 15:07:29.472061 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464524 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 28 15:07:29.472061 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464527 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 28 15:07:29.472061 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464529 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 28 15:07:29.472061 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464532 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 28 15:07:29.472061 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464534 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 28 15:07:29.472061 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464537 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 28 15:07:29.472061 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464539 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 28 15:07:29.472061 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464542 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 28 15:07:29.472061 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464544 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 28 15:07:29.472061 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464548 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 28 15:07:29.472061 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464551 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages May 28 15:07:29.472945 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464554 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 28 15:07:29.472945 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464556 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 28 15:07:29.472945 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464559 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 28 15:07:29.472945 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464562 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 28 15:07:29.472945 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464565 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 28 15:07:29.472945 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464567 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 28 15:07:29.472945 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464570 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 28 15:07:29.472945 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464572 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 28 15:07:29.472945 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464575 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 28 15:07:29.472945 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464578 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 28 15:07:29.472945 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464581 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 28 15:07:29.472945 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464583 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 28 15:07:29.472945 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464586 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 28 15:07:29.472945 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464588 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 28 15:07:29.472945 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464591 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 28 15:07:29.472945 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464593 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 28 15:07:29.472945 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464597 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 28 15:07:29.472945 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464601 2559 feature_gate.go:328] unrecognized feature gate: Example2 May 28 15:07:29.472945 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464603 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 28 15:07:29.473802 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464606 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 28 15:07:29.473802 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464609 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 28 15:07:29.473802 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464612 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 28 15:07:29.473802 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464614 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 28 15:07:29.473802 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464617 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 28 15:07:29.473802 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464620 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 28 15:07:29.473802 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464623 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 28 15:07:29.473802 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.464625 2559 feature_gate.go:328] unrecognized feature gate: DualReplica May 28 15:07:29.473802 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.465496 2559 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 28 15:07:29.473802 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.472475 2559 server.go:530] "Kubelet version" kubeletVersion="v1.33.11" May 28 15:07:29.473802 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.472510 2559 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" May 28 15:07:29.473802 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472578 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 28 15:07:29.473802 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472586 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 28 15:07:29.473802 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472591 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 28 15:07:29.473802 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472595 2559 feature_gate.go:328] unrecognized feature gate: Example2 May 28 15:07:29.473802 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472600 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 28 15:07:29.474228 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472604 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 28 15:07:29.474228 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472609 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages May 28 15:07:29.474228 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472614 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 28 15:07:29.474228 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472618 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 28 15:07:29.474228 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472623 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 28 15:07:29.474228 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472627 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 28 15:07:29.474228 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472631 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 28 15:07:29.474228 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472635 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 28 15:07:29.474228 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472639 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 28 15:07:29.474228 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472644 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 28 15:07:29.474228 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472648 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 28 15:07:29.474228 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472652 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 28 15:07:29.474228 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472656 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 28 15:07:29.474228 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472661 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 28 15:07:29.474228 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472666 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 28 15:07:29.474228 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472670 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 28 15:07:29.474228 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472675 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 28 15:07:29.474228 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472679 2559 feature_gate.go:328] unrecognized feature gate: NewOLM May 28 15:07:29.474228 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472683 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 28 15:07:29.474228 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472688 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 28 15:07:29.475119 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472692 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 28 15:07:29.475119 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472696 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 28 15:07:29.475119 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472702 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 28 15:07:29.475119 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472708 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 28 15:07:29.475119 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472713 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 28 15:07:29.475119 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472717 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 28 15:07:29.475119 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472723 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 28 15:07:29.475119 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472727 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 28 15:07:29.475119 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472731 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 28 15:07:29.475119 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472735 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 28 15:07:29.475119 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472740 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 28 15:07:29.475119 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472747 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 28 15:07:29.475119 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472753 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 28 15:07:29.475119 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472758 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 28 15:07:29.475119 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472763 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 28 15:07:29.475119 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472768 2559 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 28 15:07:29.475119 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472773 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 28 15:07:29.475119 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472777 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 28 15:07:29.475119 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472782 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 28 15:07:29.475898 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472787 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 28 15:07:29.475898 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472792 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 28 15:07:29.475898 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472796 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 28 15:07:29.475898 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472800 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 28 15:07:29.475898 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472805 2559 feature_gate.go:328] unrecognized feature gate: Example May 28 15:07:29.475898 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472809 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 28 15:07:29.475898 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472813 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 28 15:07:29.475898 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472817 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 28 15:07:29.475898 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472823 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 28 15:07:29.475898 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472827 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 28 15:07:29.475898 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472832 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 28 15:07:29.475898 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472837 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 28 15:07:29.475898 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472842 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 28 15:07:29.475898 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472846 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 28 15:07:29.475898 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472850 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 28 15:07:29.475898 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472854 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 28 15:07:29.475898 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472859 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 28 15:07:29.475898 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472863 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 28 15:07:29.475898 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472867 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 28 15:07:29.475898 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472872 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 28 15:07:29.476526 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472876 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores May 28 15:07:29.476526 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472881 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 28 15:07:29.476526 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472885 2559 feature_gate.go:328] unrecognized feature gate: DualReplica May 28 15:07:29.476526 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472890 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 28 15:07:29.476526 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472894 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 28 15:07:29.476526 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472898 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability May 28 15:07:29.476526 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472903 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 28 15:07:29.476526 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472907 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 28 15:07:29.476526 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472911 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 28 15:07:29.476526 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472915 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 28 15:07:29.476526 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472919 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 28 15:07:29.476526 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472924 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 28 15:07:29.476526 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472928 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 28 15:07:29.476526 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472932 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 28 15:07:29.476526 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472936 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 28 15:07:29.476526 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472940 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 28 15:07:29.476526 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472944 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 28 15:07:29.476526 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472948 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 28 15:07:29.476526 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472951 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 28 15:07:29.476526 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472955 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 28 15:07:29.477053 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472959 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 28 15:07:29.477053 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.472964 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 28 15:07:29.477053 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.472972 2559 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 28 15:07:29.477053 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473136 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 28 15:07:29.477053 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473144 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 28 15:07:29.477053 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473149 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 28 15:07:29.477053 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473154 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 28 15:07:29.477053 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473160 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 28 15:07:29.477053 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473164 2559 feature_gate.go:328] unrecognized feature gate: Example May 28 15:07:29.477053 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473169 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 28 15:07:29.477053 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473173 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 28 15:07:29.477053 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473178 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 28 15:07:29.477053 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473183 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 28 15:07:29.477053 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473187 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 28 15:07:29.477053 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473191 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 28 15:07:29.477053 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473195 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 28 15:07:29.477591 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473199 2559 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 28 15:07:29.477591 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473203 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 28 15:07:29.477591 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473208 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 28 15:07:29.477591 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473212 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 28 15:07:29.477591 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473216 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 28 15:07:29.477591 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473220 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 28 15:07:29.477591 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473224 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 28 15:07:29.477591 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473228 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 28 15:07:29.477591 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473232 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 28 15:07:29.477591 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473236 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 28 15:07:29.477591 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473240 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 28 15:07:29.477591 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473244 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores May 28 15:07:29.477591 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473248 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 28 15:07:29.477591 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473252 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 28 15:07:29.477591 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473257 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 28 15:07:29.477591 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473261 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 28 15:07:29.477591 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473265 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 28 15:07:29.477591 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473269 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 28 15:07:29.477591 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473275 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 28 15:07:29.478077 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473279 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 28 15:07:29.478077 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473283 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 28 15:07:29.478077 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473287 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 28 15:07:29.478077 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473291 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 28 15:07:29.478077 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473296 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 28 15:07:29.478077 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473300 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 28 15:07:29.478077 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473304 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 28 15:07:29.478077 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473309 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 28 15:07:29.478077 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473314 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 28 15:07:29.478077 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473318 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 28 15:07:29.478077 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473322 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 28 15:07:29.478077 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473326 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 28 15:07:29.478077 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473330 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 28 15:07:29.478077 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473334 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 28 15:07:29.478077 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473338 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 28 15:07:29.478077 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473343 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 28 15:07:29.478077 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473347 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 28 15:07:29.478077 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473351 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages May 28 15:07:29.478077 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473355 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 28 15:07:29.478077 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473362 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 28 15:07:29.478588 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473368 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 28 15:07:29.478588 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473373 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 28 15:07:29.478588 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473377 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 28 15:07:29.478588 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473383 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 28 15:07:29.478588 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473388 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability May 28 15:07:29.478588 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473393 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 28 15:07:29.478588 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473397 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 28 15:07:29.478588 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473402 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 28 15:07:29.478588 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473407 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 28 15:07:29.478588 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473411 2559 feature_gate.go:328] unrecognized feature gate: Example2 May 28 15:07:29.478588 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473415 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 28 15:07:29.478588 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473419 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 28 15:07:29.478588 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473424 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 28 15:07:29.478588 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473428 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 28 15:07:29.478588 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473432 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 28 15:07:29.478588 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473436 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 28 15:07:29.478588 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473440 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 28 15:07:29.478588 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473445 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 28 15:07:29.478588 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473449 2559 feature_gate.go:328] unrecognized feature gate: NewOLM May 28 15:07:29.479050 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473453 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 28 15:07:29.479050 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473458 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 28 15:07:29.479050 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473462 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 28 15:07:29.479050 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473466 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 28 15:07:29.479050 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473470 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 28 15:07:29.479050 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473474 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 28 15:07:29.479050 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473493 2559 feature_gate.go:328] unrecognized feature gate: DualReplica May 28 15:07:29.479050 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473498 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 28 15:07:29.479050 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473502 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 28 15:07:29.479050 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473507 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 28 15:07:29.479050 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473511 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 28 15:07:29.479050 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473515 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 28 15:07:29.479050 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473519 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 28 15:07:29.479050 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473523 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 28 15:07:29.479050 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:29.473527 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 28 15:07:29.479050 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.473535 2559 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 28 15:07:29.479454 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.473676 2559 server.go:962] "Client rotation is on, will bootstrap in background" May 28 15:07:29.479454 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.476375 2559 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" May 28 15:07:29.479454 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.477448 2559 server.go:1019] "Starting client certificate rotation" May 28 15:07:29.479454 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.477511 2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" May 28 15:07:29.479454 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.477553 2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" May 28 15:07:29.503206 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.503188 2559 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" May 28 15:07:29.508393 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.508376 2559 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" May 28 15:07:29.521318 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.521297 2559 log.go:25] "Validated CRI v1 runtime API" May 28 15:07:29.526342 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.526325 2559 log.go:25] "Validated CRI v1 image API" May 28 15:07:29.532245 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.532230 2559 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" May 28 15:07:29.535900 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.535883 2559 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" May 28 15:07:29.536382 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.536365 2559 fs.go:135] Filesystem UUIDs: map[02c3b853-252c-4c7e-87cc-9ec13c80033e:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 cedb042d-f98d-417f-a007-6b4d102f2d40:/dev/nvme0n1p3] May 28 15:07:29.536428 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.536383 2559 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] May 28 15:07:29.540711 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.540584 2559 manager.go:217] Machine: {Timestamp:2026-05-28 15:07:29.540101476 +0000 UTC m=+0.429088552 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100067 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2533a2aafa83fcbe7e704e04d19bb1 SystemUUID:ec2533a2-aafa-83fc-be7e-704e04d19bb1 BootID:11649969-9ef4-44c6-b7b9-1a8acc0002a9 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b7:97:14:10:c7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b7:97:14:10:c7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:5e:05:e3:2c:77:20 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} May 28 15:07:29.541644 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.541633 2559 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. May 28 15:07:29.541725 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.541713 2559 manager.go:233] Version: {KernelVersion:5.14.0-570.114.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260512-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} May 28 15:07:29.545125 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.545086 2559 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] May 28 15:07:29.545248 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.545125 2559 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-141.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} May 28 15:07:29.545292 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.545257 2559 topology_manager.go:138] "Creating topology manager with none policy" May 28 15:07:29.545292 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.545265 2559 container_manager_linux.go:306] "Creating device plugin manager" May 28 15:07:29.545292 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.545272 2559 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" May 28 15:07:29.546057 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.546048 2559 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" May 28 15:07:29.548103 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.548093 2559 state_mem.go:36] "Initialized new in-memory state store" May 28 15:07:29.548199 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.548191 2559 server.go:1267] "Using root directory" path="/var/lib/kubelet" May 28 15:07:29.550574 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.550555 2559 kubelet.go:491] "Attempting to sync node with API server" May 28 15:07:29.550574 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.550573 2559 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" May 28 15:07:29.550719 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.550593 2559 file.go:69] "Watching path" path="/etc/kubernetes/manifests" May 28 15:07:29.550719 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.550606 2559 kubelet.go:397] "Adding apiserver pod source" May 28 15:07:29.550719 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.550618 2559 apiserver.go:42] "Waiting for node sync before watching apiserver pods" May 28 15:07:29.551740 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.551728 2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" May 28 15:07:29.551780 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.551747 2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" May 28 15:07:29.554729 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.554705 2559 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.11-2.rhaos4.20.gitb2a8320.el9" apiVersion="v1" May 28 15:07:29.556532 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.556519 2559 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" May 28 15:07:29.557334 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.557321 2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" May 28 15:07:29.557367 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.557344 2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" May 28 15:07:29.557367 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.557354 2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" May 28 15:07:29.557367 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.557362 2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" May 28 15:07:29.557441 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.557374 2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" May 28 15:07:29.557441 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.557386 2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" May 28 15:07:29.557441 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.557394 2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" May 28 15:07:29.557441 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.557402 2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" May 28 15:07:29.557441 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.557412 2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" May 28 15:07:29.557441 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.557422 2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" May 28 15:07:29.557441 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.557433 2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" May 28 15:07:29.557662 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.557650 2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" May 28 15:07:29.558697 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.558688 2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" May 28 15:07:29.558733 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.558698 2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" May 28 15:07:29.561617 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.561605 2559 watchdog_linux.go:99] "Systemd watchdog is not enabled" May 28 15:07:29.561653 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.561639 2559 server.go:1295] "Started kubelet" May 28 15:07:29.561758 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.561709 2559 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 May 28 15:07:29.561825 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.561777 2559 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 May 28 15:07:29.561875 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.561844 2559 server_v1.go:47] "podresources" method="list" useActivePods=true May 28 15:07:29.562349 ip-10-0-140-141 systemd[1]: Started Kubernetes Kubelet. May 28 15:07:29.563002 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.562914 2559 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" May 28 15:07:29.564611 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:29.564587 2559 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" May 28 15:07:29.564705 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:29.564625 2559 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-141.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" May 28 15:07:29.564705 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.564647 2559 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-141.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope May 28 15:07:29.564705 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.564652 2559 server.go:317] "Adding debug handlers to kubelet server" May 28 15:07:29.569926 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.569899 2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" May 28 15:07:29.570727 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.570701 2559 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" May 28 15:07:29.570932 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.570820 2559 volume_manager.go:295] "The desired_state_of_world populator starts" May 28 15:07:29.570932 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.570836 2559 volume_manager.go:297] "Starting Kubelet Volume Manager" May 28 15:07:29.570932 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.570868 2559 desired_state_of_world_populator.go:150] "Desired state populator starts to run" May 28 15:07:29.571881 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.571861 2559 reconstruct.go:97] "Volume reconstruction finished" May 28 15:07:29.571881 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.571881 2559 reconciler.go:26] "Reconciler: start to sync state" May 28 15:07:29.571984 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:29.571954 2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-141.ec2.internal\" not found" May 28 15:07:29.574014 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.573929 2559 factory.go:55] Registering systemd factory May 28 15:07:29.574014 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.573946 2559 factory.go:223] Registration of the systemd container factory successfully May 28 15:07:29.574340 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.574326 2559 factory.go:153] Registering CRI-O factory May 28 15:07:29.574435 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.574424 2559 factory.go:223] Registration of the crio container factory successfully May 28 15:07:29.574693 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.574681 2559 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory May 28 15:07:29.574820 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.574809 2559 factory.go:103] Registering Raw factory May 28 15:07:29.574921 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:29.574513 2559 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" May 28 15:07:29.574921 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:29.574681 2559 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-141.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" May 28 15:07:29.574921 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.574909 2559 manager.go:1196] Started watching for new ooms in manager May 28 15:07:29.575309 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:29.574243 2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-141.ec2.internal.18b3c326e51e42bb default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-141.ec2.internal,UID:ip-10-0-140-141.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-141.ec2.internal,},FirstTimestamp:2026-05-28 15:07:29.561617083 +0000 UTC m=+0.450604158,LastTimestamp:2026-05-28 15:07:29.561617083 +0000 UTC m=+0.450604158,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-141.ec2.internal,}" May 28 15:07:29.575473 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.575459 2559 manager.go:319] Starting recovery of all containers May 28 15:07:29.576901 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:29.576874 2559 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" May 28 15:07:29.578432 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.578412 2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-f7gc2" May 28 15:07:29.584854 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.584694 2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-f7gc2" May 28 15:07:29.585024 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.585012 2559 manager.go:324] Recovery completed May 28 15:07:29.589616 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.589604 2559 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 28 15:07:29.591537 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.591523 2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-141.ec2.internal" event="NodeHasSufficientMemory" May 28 15:07:29.591592 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.591550 2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-141.ec2.internal" event="NodeHasNoDiskPressure" May 28 15:07:29.591592 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.591562 2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-141.ec2.internal" event="NodeHasSufficientPID" May 28 15:07:29.592419 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.592406 2559 cpu_manager.go:222] "Starting CPU manager" policy="none" May 28 15:07:29.592419 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.592417 2559 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" May 28 15:07:29.592532 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.592433 2559 state_mem.go:36] "Initialized new in-memory state store" May 28 15:07:29.593664 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:29.593583 2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-141.ec2.internal.18b3c326e6e6ce7d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-141.ec2.internal,UID:ip-10-0-140-141.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-141.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-141.ec2.internal,},FirstTimestamp:2026-05-28 15:07:29.591537277 +0000 UTC m=+0.480524352,LastTimestamp:2026-05-28 15:07:29.591537277 +0000 UTC m=+0.480524352,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-141.ec2.internal,}" May 28 15:07:29.594704 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.594691 2559 policy_none.go:49] "None policy: Start" May 28 15:07:29.594756 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.594707 2559 memory_manager.go:186] "Starting memorymanager" policy="None" May 28 15:07:29.594756 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.594717 2559 state_mem.go:35] "Initializing new in-memory state store" May 28 15:07:29.629069 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.629053 2559 manager.go:341] "Starting Device Plugin manager" May 28 15:07:29.644184 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:29.629084 2559 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" May 28 15:07:29.644184 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.629093 2559 server.go:85] "Starting device plugin registration server" May 28 15:07:29.644184 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.629586 2559 eviction_manager.go:189] "Eviction manager: starting control loop" May 28 15:07:29.644184 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.629604 2559 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" May 28 15:07:29.644184 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.629876 2559 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" May 28 15:07:29.644184 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.629955 2559 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" May 28 15:07:29.644184 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.629965 2559 plugin_manager.go:118] "Starting Kubelet Plugin Manager" May 28 15:07:29.644184 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:29.630694 2559 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" May 28 15:07:29.644184 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:29.630751 2559 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-141.ec2.internal\" not found" May 28 15:07:29.706636 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.706555 2559 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" May 28 15:07:29.707757 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.707741 2559 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" May 28 15:07:29.707813 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.707763 2559 status_manager.go:230] "Starting to sync pod status with apiserver" May 28 15:07:29.707813 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.707781 2559 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." May 28 15:07:29.707813 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.707788 2559 kubelet.go:2451] "Starting kubelet main sync loop" May 28 15:07:29.707927 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:29.707833 2559 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" May 28 15:07:29.710006 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.709989 2559 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" May 28 15:07:29.730743 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.730722 2559 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 28 15:07:29.731557 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.731541 2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-141.ec2.internal" event="NodeHasSufficientMemory" May 28 15:07:29.731659 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.731574 2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-141.ec2.internal" event="NodeHasNoDiskPressure" May 28 15:07:29.731659 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.731589 2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-141.ec2.internal" event="NodeHasSufficientPID" May 28 15:07:29.731659 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.731620 2559 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-141.ec2.internal" May 28 15:07:29.739881 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.739867 2559 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-141.ec2.internal" May 28 15:07:29.739945 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:29.739890 2559 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-141.ec2.internal\": node \"ip-10-0-140-141.ec2.internal\" not found" May 28 15:07:29.751646 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:29.751624 2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-141.ec2.internal\" not found" May 28 15:07:29.808831 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.808781 2559 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-141.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-141.ec2.internal"] May 28 15:07:29.808979 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.808898 2559 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 28 15:07:29.809805 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.809786 2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-141.ec2.internal" event="NodeHasSufficientMemory" May 28 15:07:29.809910 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.809817 2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-141.ec2.internal" event="NodeHasNoDiskPressure" May 28 15:07:29.809910 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.809827 2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-141.ec2.internal" event="NodeHasSufficientPID" May 28 15:07:29.811207 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.811195 2559 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 28 15:07:29.811327 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.811314 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-141.ec2.internal" May 28 15:07:29.811373 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.811343 2559 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 28 15:07:29.811944 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.811929 2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-141.ec2.internal" event="NodeHasSufficientMemory" May 28 15:07:29.812005 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.811957 2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-141.ec2.internal" event="NodeHasNoDiskPressure" May 28 15:07:29.812005 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.811967 2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-141.ec2.internal" event="NodeHasSufficientPID" May 28 15:07:29.812071 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.811929 2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-141.ec2.internal" event="NodeHasSufficientMemory" May 28 15:07:29.812071 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.812028 2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-141.ec2.internal" event="NodeHasNoDiskPressure" May 28 15:07:29.812071 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.812039 2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-141.ec2.internal" event="NodeHasSufficientPID" May 28 15:07:29.813155 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.813141 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-141.ec2.internal" May 28 15:07:29.813196 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.813165 2559 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 28 15:07:29.813838 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.813824 2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-141.ec2.internal" event="NodeHasSufficientMemory" May 28 15:07:29.813915 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.813852 2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-141.ec2.internal" event="NodeHasNoDiskPressure" May 28 15:07:29.813915 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.813865 2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-141.ec2.internal" event="NodeHasSufficientPID" May 28 15:07:29.831014 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:29.830997 2559 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-141.ec2.internal\" not found" node="ip-10-0-140-141.ec2.internal" May 28 15:07:29.834949 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:29.834932 2559 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-141.ec2.internal\" not found" node="ip-10-0-140-141.ec2.internal" May 28 15:07:29.851860 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:29.851841 2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-141.ec2.internal\" not found" May 28 15:07:29.873151 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.873124 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/764dacf5d913e4d5bcd2ceb0fe8fea04-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-141.ec2.internal\" (UID: \"764dacf5d913e4d5bcd2ceb0fe8fea04\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-141.ec2.internal" May 28 15:07:29.873241 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.873157 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/764dacf5d913e4d5bcd2ceb0fe8fea04-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-141.ec2.internal\" (UID: \"764dacf5d913e4d5bcd2ceb0fe8fea04\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-141.ec2.internal" May 28 15:07:29.873241 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.873180 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f46265c427d4cbfcaecf493db63aaa49-config\") pod \"kube-apiserver-proxy-ip-10-0-140-141.ec2.internal\" (UID: \"f46265c427d4cbfcaecf493db63aaa49\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-141.ec2.internal" May 28 15:07:29.952846 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:29.952821 2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-141.ec2.internal\" not found" May 28 15:07:29.974193 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.974137 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/764dacf5d913e4d5bcd2ceb0fe8fea04-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-141.ec2.internal\" (UID: \"764dacf5d913e4d5bcd2ceb0fe8fea04\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-141.ec2.internal" May 28 15:07:29.974193 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.974169 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/764dacf5d913e4d5bcd2ceb0fe8fea04-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-141.ec2.internal\" (UID: \"764dacf5d913e4d5bcd2ceb0fe8fea04\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-141.ec2.internal" May 28 15:07:29.974310 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.974194 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f46265c427d4cbfcaecf493db63aaa49-config\") pod \"kube-apiserver-proxy-ip-10-0-140-141.ec2.internal\" (UID: \"f46265c427d4cbfcaecf493db63aaa49\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-141.ec2.internal" May 28 15:07:29.974310 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.974225 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f46265c427d4cbfcaecf493db63aaa49-config\") pod \"kube-apiserver-proxy-ip-10-0-140-141.ec2.internal\" (UID: \"f46265c427d4cbfcaecf493db63aaa49\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-141.ec2.internal" May 28 15:07:29.974310 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.974228 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/764dacf5d913e4d5bcd2ceb0fe8fea04-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-141.ec2.internal\" (UID: \"764dacf5d913e4d5bcd2ceb0fe8fea04\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-141.ec2.internal" May 28 15:07:29.974310 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:29.974235 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/764dacf5d913e4d5bcd2ceb0fe8fea04-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-141.ec2.internal\" (UID: \"764dacf5d913e4d5bcd2ceb0fe8fea04\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-141.ec2.internal" May 28 15:07:30.053442 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:30.053405 2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-141.ec2.internal\" not found" May 28 15:07:30.133884 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:30.133860 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-141.ec2.internal" May 28 15:07:30.137615 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:30.137601 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-141.ec2.internal" May 28 15:07:30.154406 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:30.154382 2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-141.ec2.internal\" not found" May 28 15:07:30.254928 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:30.254855 2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-141.ec2.internal\" not found" May 28 15:07:30.355387 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:30.355355 2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-141.ec2.internal\" not found" May 28 15:07:30.456086 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:30.456055 2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-141.ec2.internal\" not found" May 28 15:07:30.477389 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:30.477360 2559 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" May 28 15:07:30.477553 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:30.477536 2559 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" May 28 15:07:30.557017 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:30.556964 2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-141.ec2.internal\" not found" May 28 15:07:30.570937 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:30.570803 2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" May 28 15:07:30.584297 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:30.584271 2559 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" May 28 15:07:30.587135 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:30.587106 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-27 15:02:29 +0000 UTC" deadline="2027-12-01 08:38:23.85694497 +0000 UTC" May 28 15:07:30.587135 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:30.587134 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13241h30m53.269813376s" May 28 15:07:30.603280 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:30.603256 2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-ms8cp" May 28 15:07:30.610356 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:30.610338 2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-ms8cp" May 28 15:07:30.657931 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:30.657905 2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-141.ec2.internal\" not found" May 28 15:07:30.660942 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:30.660906 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf46265c427d4cbfcaecf493db63aaa49.slice/crio-bdb028375f8fdf749dec2643d7de10d035fc65a02ee439042b74d28ba55bce70 WatchSource:0}: Error finding container bdb028375f8fdf749dec2643d7de10d035fc65a02ee439042b74d28ba55bce70: Status 404 returned error can't find the container with id bdb028375f8fdf749dec2643d7de10d035fc65a02ee439042b74d28ba55bce70 May 28 15:07:30.661191 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:30.661170 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod764dacf5d913e4d5bcd2ceb0fe8fea04.slice/crio-db824099125980caedd3bb287ec625825566d3c33ffd2e323a498170240acaf5 WatchSource:0}: Error finding container db824099125980caedd3bb287ec625825566d3c33ffd2e323a498170240acaf5: Status 404 returned error can't find the container with id db824099125980caedd3bb287ec625825566d3c33ffd2e323a498170240acaf5 May 28 15:07:30.664950 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:30.664934 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 28 15:07:30.711043 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:30.710996 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-141.ec2.internal" event={"ID":"f46265c427d4cbfcaecf493db63aaa49","Type":"ContainerStarted","Data":"bdb028375f8fdf749dec2643d7de10d035fc65a02ee439042b74d28ba55bce70"} May 28 15:07:30.711844 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:30.711827 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-141.ec2.internal" event={"ID":"764dacf5d913e4d5bcd2ceb0fe8fea04","Type":"ContainerStarted","Data":"db824099125980caedd3bb287ec625825566d3c33ffd2e323a498170240acaf5"} May 28 15:07:30.759030 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:30.758990 2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-141.ec2.internal\" not found" May 28 15:07:30.768716 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:30.768690 2559 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" May 28 15:07:30.771512 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:30.771493 2559 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-141.ec2.internal" May 28 15:07:30.784203 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:30.784183 2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" May 28 15:07:30.785992 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:30.785979 2559 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-141.ec2.internal" May 28 15:07:30.795171 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:30.795154 2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" May 28 15:07:30.858249 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:30.858190 2559 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" May 28 15:07:31.171439 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.171360 2559 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" May 28 15:07:31.552747 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.552673 2559 apiserver.go:52] "Watching apiserver" May 28 15:07:31.563629 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.563608 2559 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" May 28 15:07:31.564118 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.564093 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-141.ec2.internal","openshift-multus/multus-additional-cni-plugins-jc4zf","openshift-multus/multus-l7mvw","openshift-network-diagnostics/network-check-target-rpkl2","openshift-network-operator/iptables-alerter-l6f5d","openshift-ovn-kubernetes/ovnkube-node-xcg2z","kube-system/kube-apiserver-proxy-ip-10-0-140-141.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rgnx4","openshift-dns/node-resolver-kr9wd","openshift-image-registry/node-ca-bpxxm","openshift-multus/network-metrics-daemon-l9p7b","kube-system/konnectivity-agent-bjm2p","openshift-cluster-node-tuning-operator/tuned-cm7p8"] May 28 15:07:31.565979 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.565959 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bpxxm" May 28 15:07:31.567188 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.567169 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jc4zf" May 28 15:07:31.570338 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.568945 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" May 28 15:07:31.570338 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.569094 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" May 28 15:07:31.570338 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.569187 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-7mzdh\"" May 28 15:07:31.570338 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.569311 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" May 28 15:07:31.570338 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.570107 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-j5fnz\"" May 28 15:07:31.570338 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.569394 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" May 28 15:07:31.570338 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.570200 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" May 28 15:07:31.570338 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.569623 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" May 28 15:07:31.570338 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.569561 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" May 28 15:07:31.571235 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.570459 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" May 28 15:07:31.573331 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.572917 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-l7mvw" May 28 15:07:31.574705 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.574615 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.574842 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.574800 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bjm2p" May 28 15:07:31.575130 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.575114 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" May 28 15:07:31.575209 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.575193 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-jm57m\"" May 28 15:07:31.576536 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.576263 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.577246 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.577225 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" May 28 15:07:31.577560 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.577523 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" May 28 15:07:31.577645 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.577571 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jcqlp\"" May 28 15:07:31.577645 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.577593 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" May 28 15:07:31.577756 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.577735 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" May 28 15:07:31.577807 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.577786 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8lv5g\"" May 28 15:07:31.578804 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.578710 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" May 28 15:07:31.578905 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.578783 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" May 28 15:07:31.578905 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.578783 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hl9fb\"" May 28 15:07:31.578905 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.578785 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" May 28 15:07:31.579147 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.578806 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" May 28 15:07:31.579339 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.579323 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9p7b" May 28 15:07:31.579514 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:31.579410 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9p7b" podUID="e3a78917-de69-42d0-8fbe-85f346276bf9" May 28 15:07:31.579602 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.579520 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rgnx4" May 28 15:07:31.579946 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.579898 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" May 28 15:07:31.579946 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.579943 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" May 28 15:07:31.581496 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.581462 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" May 28 15:07:31.581930 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.581912 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" May 28 15:07:31.582111 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.582072 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-kfvqx\"" May 28 15:07:31.582190 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.582144 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/10ca4edd-091a-4f34-912a-0a579c845f52-cnibin\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.582190 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.582081 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" May 28 15:07:31.582190 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.582173 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llph8\" (UniqueName: \"kubernetes.io/projected/10ca4edd-091a-4f34-912a-0a579c845f52-kube-api-access-llph8\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.582342 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.582198 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8c849d95-73cd-461d-bd6b-845cb70cbd23-sys\") pod \"tuned-cm7p8\" (UID: \"8c849d95-73cd-461d-bd6b-845cb70cbd23\") " pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.582342 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.582220 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a2246888-2a0d-48b8-8fe6-12e283e47eb0-host\") pod \"node-ca-bpxxm\" (UID: \"a2246888-2a0d-48b8-8fe6-12e283e47eb0\") " pod="openshift-image-registry/node-ca-bpxxm" May 28 15:07:31.582342 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.582252 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtxqw\" (UniqueName: \"kubernetes.io/projected/a2246888-2a0d-48b8-8fe6-12e283e47eb0-kube-api-access-mtxqw\") pod \"node-ca-bpxxm\" (UID: \"a2246888-2a0d-48b8-8fe6-12e283e47eb0\") " pod="openshift-image-registry/node-ca-bpxxm" May 28 15:07:31.582342 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.582279 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/615d5292-c021-4a20-be77-56f9230db424-cnibin\") pod \"multus-additional-cni-plugins-jc4zf\" (UID: \"615d5292-c021-4a20-be77-56f9230db424\") " pod="openshift-multus/multus-additional-cni-plugins-jc4zf" May 28 15:07:31.582342 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.582304 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/615d5292-c021-4a20-be77-56f9230db424-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jc4zf\" (UID: \"615d5292-c021-4a20-be77-56f9230db424\") " pod="openshift-multus/multus-additional-cni-plugins-jc4zf" May 28 15:07:31.582342 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.582328 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/10ca4edd-091a-4f34-912a-0a579c845f52-cni-binary-copy\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.582645 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.582364 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8c849d95-73cd-461d-bd6b-845cb70cbd23-etc-systemd\") pod \"tuned-cm7p8\" (UID: \"8c849d95-73cd-461d-bd6b-845cb70cbd23\") " pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.582645 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.582392 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8c849d95-73cd-461d-bd6b-845cb70cbd23-lib-modules\") pod \"tuned-cm7p8\" (UID: \"8c849d95-73cd-461d-bd6b-845cb70cbd23\") " pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.582645 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.582418 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c849d95-73cd-461d-bd6b-845cb70cbd23-var-lib-kubelet\") pod \"tuned-cm7p8\" (UID: \"8c849d95-73cd-461d-bd6b-845cb70cbd23\") " pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.582645 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.582440 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8c849d95-73cd-461d-bd6b-845cb70cbd23-etc-tuned\") pod \"tuned-cm7p8\" (UID: \"8c849d95-73cd-461d-bd6b-845cb70cbd23\") " pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.582645 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.582464 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c849d95-73cd-461d-bd6b-845cb70cbd23-tmp\") pod \"tuned-cm7p8\" (UID: \"8c849d95-73cd-461d-bd6b-845cb70cbd23\") " pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.582645 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.582509 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sxf7\" (UniqueName: \"kubernetes.io/projected/8c849d95-73cd-461d-bd6b-845cb70cbd23-kube-api-access-7sxf7\") pod \"tuned-cm7p8\" (UID: \"8c849d95-73cd-461d-bd6b-845cb70cbd23\") " pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.582645 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.582536 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/615d5292-c021-4a20-be77-56f9230db424-cni-binary-copy\") pod \"multus-additional-cni-plugins-jc4zf\" (UID: \"615d5292-c021-4a20-be77-56f9230db424\") " pod="openshift-multus/multus-additional-cni-plugins-jc4zf" May 28 15:07:31.582645 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.582553 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8c849d95-73cd-461d-bd6b-845cb70cbd23-etc-sysctl-conf\") pod \"tuned-cm7p8\" (UID: \"8c849d95-73cd-461d-bd6b-845cb70cbd23\") " pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.582645 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.582566 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8c849d95-73cd-461d-bd6b-845cb70cbd23-run\") pod \"tuned-cm7p8\" (UID: \"8c849d95-73cd-461d-bd6b-845cb70cbd23\") " pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.582645 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.582589 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/10ca4edd-091a-4f34-912a-0a579c845f52-host-run-netns\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.582645 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.582621 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/10ca4edd-091a-4f34-912a-0a579c845f52-multus-conf-dir\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.582645 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.582644 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/10ca4edd-091a-4f34-912a-0a579c845f52-multus-daemon-config\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.583213 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.582661 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/615d5292-c021-4a20-be77-56f9230db424-system-cni-dir\") pod \"multus-additional-cni-plugins-jc4zf\" (UID: \"615d5292-c021-4a20-be77-56f9230db424\") " pod="openshift-multus/multus-additional-cni-plugins-jc4zf" May 28 15:07:31.583213 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.582688 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/615d5292-c021-4a20-be77-56f9230db424-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jc4zf\" (UID: \"615d5292-c021-4a20-be77-56f9230db424\") " pod="openshift-multus/multus-additional-cni-plugins-jc4zf" May 28 15:07:31.583213 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.582711 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/10ca4edd-091a-4f34-912a-0a579c845f52-os-release\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.583213 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.582733 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/10ca4edd-091a-4f34-912a-0a579c845f52-host-run-k8s-cni-cncf-io\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.583213 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.582768 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/10ca4edd-091a-4f34-912a-0a579c845f52-host-var-lib-cni-bin\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.583213 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.582791 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/10ca4edd-091a-4f34-912a-0a579c845f52-host-var-lib-cni-multus\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.583213 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.582814 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/10ca4edd-091a-4f34-912a-0a579c845f52-host-run-multus-certs\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.583213 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.582831 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rpkl2" May 28 15:07:31.583213 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.582850 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/615d5292-c021-4a20-be77-56f9230db424-os-release\") pod \"multus-additional-cni-plugins-jc4zf\" (UID: \"615d5292-c021-4a20-be77-56f9230db424\") " pod="openshift-multus/multus-additional-cni-plugins-jc4zf" May 28 15:07:31.583213 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.582884 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8c849d95-73cd-461d-bd6b-845cb70cbd23-etc-sysconfig\") pod \"tuned-cm7p8\" (UID: \"8c849d95-73cd-461d-bd6b-845cb70cbd23\") " pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.583213 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:31.582889 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rpkl2" podUID="8306acd6-5d60-42dd-a8e4-d21a859b6525" May 28 15:07:31.583213 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.582905 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a2246888-2a0d-48b8-8fe6-12e283e47eb0-serviceca\") pod \"node-ca-bpxxm\" (UID: \"a2246888-2a0d-48b8-8fe6-12e283e47eb0\") " pod="openshift-image-registry/node-ca-bpxxm" May 28 15:07:31.583213 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.582941 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/615d5292-c021-4a20-be77-56f9230db424-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jc4zf\" (UID: \"615d5292-c021-4a20-be77-56f9230db424\") " pod="openshift-multus/multus-additional-cni-plugins-jc4zf" May 28 15:07:31.583213 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.582976 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/10ca4edd-091a-4f34-912a-0a579c845f52-multus-socket-dir-parent\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.583213 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.583000 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8c849d95-73cd-461d-bd6b-845cb70cbd23-etc-sysctl-d\") pod \"tuned-cm7p8\" (UID: \"8c849d95-73cd-461d-bd6b-845cb70cbd23\") " pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.583213 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.583024 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10ca4edd-091a-4f34-912a-0a579c845f52-system-cni-dir\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.583213 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.583066 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10ca4edd-091a-4f34-912a-0a579c845f52-multus-cni-dir\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.583925 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.583102 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10ca4edd-091a-4f34-912a-0a579c845f52-etc-kubernetes\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.583925 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.583230 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8c849d95-73cd-461d-bd6b-845cb70cbd23-etc-modprobe-d\") pod \"tuned-cm7p8\" (UID: \"8c849d95-73cd-461d-bd6b-845cb70cbd23\") " pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.583925 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.583260 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c849d95-73cd-461d-bd6b-845cb70cbd23-etc-kubernetes\") pod \"tuned-cm7p8\" (UID: \"8c849d95-73cd-461d-bd6b-845cb70cbd23\") " pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.583925 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.583295 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8c849d95-73cd-461d-bd6b-845cb70cbd23-host\") pod \"tuned-cm7p8\" (UID: \"8c849d95-73cd-461d-bd6b-845cb70cbd23\") " pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.583925 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.583322 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84btw\" (UniqueName: \"kubernetes.io/projected/615d5292-c021-4a20-be77-56f9230db424-kube-api-access-84btw\") pod \"multus-additional-cni-plugins-jc4zf\" (UID: \"615d5292-c021-4a20-be77-56f9230db424\") " pod="openshift-multus/multus-additional-cni-plugins-jc4zf" May 28 15:07:31.583925 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.583345 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/10ca4edd-091a-4f34-912a-0a579c845f52-host-var-lib-kubelet\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.583925 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.583368 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/10ca4edd-091a-4f34-912a-0a579c845f52-hostroot\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.585078 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.584726 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-l6f5d" May 28 15:07:31.585078 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.584881 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kr9wd" May 28 15:07:31.587693 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.587613 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" May 28 15:07:31.588011 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.587989 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-kx5qm\"" May 28 15:07:31.588100 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.588064 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" May 28 15:07:31.588734 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.588715 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-k29js\"" May 28 15:07:31.588734 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.588729 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" May 28 15:07:31.588876 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.588750 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" May 28 15:07:31.588876 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.588729 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" May 28 15:07:31.611324 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.611287 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-27 15:02:30 +0000 UTC" deadline="2028-01-25 08:38:22.165607361 +0000 UTC" May 28 15:07:31.611324 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.611316 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14561h30m50.554295573s" May 28 15:07:31.671888 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.671868 2559 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" May 28 15:07:31.683776 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.683753 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-host-cni-bin\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.683882 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.683779 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkstc\" (UniqueName: \"kubernetes.io/projected/297e38b8-9a81-4112-811d-a97cfa5341ff-kube-api-access-bkstc\") pod \"node-resolver-kr9wd\" (UID: \"297e38b8-9a81-4112-811d-a97cfa5341ff\") " pod="openshift-dns/node-resolver-kr9wd" May 28 15:07:31.683882 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.683796 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/44b3ce72-966f-42e8-8b3a-b9615228955c-host-slash\") pod \"iptables-alerter-l6f5d\" (UID: \"44b3ce72-966f-42e8-8b3a-b9615228955c\") " pod="openshift-network-operator/iptables-alerter-l6f5d" May 28 15:07:31.683882 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.683816 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-llph8\" (UniqueName: \"kubernetes.io/projected/10ca4edd-091a-4f34-912a-0a579c845f52-kube-api-access-llph8\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.683882 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.683861 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8c849d95-73cd-461d-bd6b-845cb70cbd23-sys\") pod \"tuned-cm7p8\" (UID: \"8c849d95-73cd-461d-bd6b-845cb70cbd23\") " pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.684092 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.683899 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a2246888-2a0d-48b8-8fe6-12e283e47eb0-host\") pod \"node-ca-bpxxm\" (UID: \"a2246888-2a0d-48b8-8fe6-12e283e47eb0\") " pod="openshift-image-registry/node-ca-bpxxm" May 28 15:07:31.684092 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.683927 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mtxqw\" (UniqueName: \"kubernetes.io/projected/a2246888-2a0d-48b8-8fe6-12e283e47eb0-kube-api-access-mtxqw\") pod \"node-ca-bpxxm\" (UID: \"a2246888-2a0d-48b8-8fe6-12e283e47eb0\") " pod="openshift-image-registry/node-ca-bpxxm" May 28 15:07:31.684092 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.683955 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-run-systemd\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.684092 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.683979 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-run-openvswitch\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.684092 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.683982 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a2246888-2a0d-48b8-8fe6-12e283e47eb0-host\") pod \"node-ca-bpxxm\" (UID: \"a2246888-2a0d-48b8-8fe6-12e283e47eb0\") " pod="openshift-image-registry/node-ca-bpxxm" May 28 15:07:31.684092 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.683978 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8c849d95-73cd-461d-bd6b-845cb70cbd23-sys\") pod \"tuned-cm7p8\" (UID: \"8c849d95-73cd-461d-bd6b-845cb70cbd23\") " pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.684092 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.684028 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mhbx\" (UniqueName: \"kubernetes.io/projected/e3a78917-de69-42d0-8fbe-85f346276bf9-kube-api-access-7mhbx\") pod \"network-metrics-daemon-l9p7b\" (UID: \"e3a78917-de69-42d0-8fbe-85f346276bf9\") " pod="openshift-multus/network-metrics-daemon-l9p7b" May 28 15:07:31.684092 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.684049 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/10ca4edd-091a-4f34-912a-0a579c845f52-cni-binary-copy\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.684092 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.684067 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c849d95-73cd-461d-bd6b-845cb70cbd23-var-lib-kubelet\") pod \"tuned-cm7p8\" (UID: \"8c849d95-73cd-461d-bd6b-845cb70cbd23\") " pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.684092 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.684090 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8c849d95-73cd-461d-bd6b-845cb70cbd23-etc-tuned\") pod \"tuned-cm7p8\" (UID: \"8c849d95-73cd-461d-bd6b-845cb70cbd23\") " pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.684573 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.684116 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7sxf7\" (UniqueName: \"kubernetes.io/projected/8c849d95-73cd-461d-bd6b-845cb70cbd23-kube-api-access-7sxf7\") pod \"tuned-cm7p8\" (UID: \"8c849d95-73cd-461d-bd6b-845cb70cbd23\") " pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.684573 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.684143 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c849d95-73cd-461d-bd6b-845cb70cbd23-var-lib-kubelet\") pod \"tuned-cm7p8\" (UID: \"8c849d95-73cd-461d-bd6b-845cb70cbd23\") " pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.684573 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.684181 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-host-run-netns\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.684573 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.684214 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0f511937-01b1-4615-bc4c-12832012e6ce-ovnkube-config\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.684573 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.684239 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8c849d95-73cd-461d-bd6b-845cb70cbd23-run\") pod \"tuned-cm7p8\" (UID: \"8c849d95-73cd-461d-bd6b-845cb70cbd23\") " pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.684573 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.684264 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/10ca4edd-091a-4f34-912a-0a579c845f52-multus-conf-dir\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.684573 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.684290 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/615d5292-c021-4a20-be77-56f9230db424-system-cni-dir\") pod \"multus-additional-cni-plugins-jc4zf\" (UID: \"615d5292-c021-4a20-be77-56f9230db424\") " pod="openshift-multus/multus-additional-cni-plugins-jc4zf" May 28 15:07:31.684573 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.684316 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-host-cni-netd\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.684573 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.684330 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8c849d95-73cd-461d-bd6b-845cb70cbd23-run\") pod \"tuned-cm7p8\" (UID: \"8c849d95-73cd-461d-bd6b-845cb70cbd23\") " pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.684573 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.684341 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/10ca4edd-091a-4f34-912a-0a579c845f52-host-run-k8s-cni-cncf-io\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.684573 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.684352 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/10ca4edd-091a-4f34-912a-0a579c845f52-multus-conf-dir\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.684573 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.684372 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/615d5292-c021-4a20-be77-56f9230db424-system-cni-dir\") pod \"multus-additional-cni-plugins-jc4zf\" (UID: \"615d5292-c021-4a20-be77-56f9230db424\") " pod="openshift-multus/multus-additional-cni-plugins-jc4zf" May 28 15:07:31.684573 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.684389 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/10ca4edd-091a-4f34-912a-0a579c845f52-host-run-k8s-cni-cncf-io\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.684573 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.684397 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/10ca4edd-091a-4f34-912a-0a579c845f52-host-var-lib-cni-multus\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.684573 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.684403 2559 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" May 28 15:07:31.684573 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.684423 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/10ca4edd-091a-4f34-912a-0a579c845f52-host-run-multus-certs\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.684573 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.684451 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-run-ovn\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.684573 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.684476 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8c849d95-73cd-461d-bd6b-845cb70cbd23-etc-sysconfig\") pod \"tuned-cm7p8\" (UID: \"8c849d95-73cd-461d-bd6b-845cb70cbd23\") " pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.685400 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.684511 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/10ca4edd-091a-4f34-912a-0a579c845f52-host-run-multus-certs\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.685400 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.684520 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a2246888-2a0d-48b8-8fe6-12e283e47eb0-serviceca\") pod \"node-ca-bpxxm\" (UID: \"a2246888-2a0d-48b8-8fe6-12e283e47eb0\") " pod="openshift-image-registry/node-ca-bpxxm" May 28 15:07:31.685400 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.684531 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8c849d95-73cd-461d-bd6b-845cb70cbd23-etc-sysconfig\") pod \"tuned-cm7p8\" (UID: \"8c849d95-73cd-461d-bd6b-845cb70cbd23\") " pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.685400 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.684544 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0f511937-01b1-4615-bc4c-12832012e6ce-env-overrides\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.685400 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.684620 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc-device-dir\") pod \"aws-ebs-csi-driver-node-rgnx4\" (UID: \"ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rgnx4" May 28 15:07:31.685400 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.684648 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/10ca4edd-091a-4f34-912a-0a579c845f52-cni-binary-copy\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.685400 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.684669 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/10ca4edd-091a-4f34-912a-0a579c845f52-multus-socket-dir-parent\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.685400 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.684706 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.685400 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.684734 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0f511937-01b1-4615-bc4c-12832012e6ce-ovn-node-metrics-cert\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.685400 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.684752 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/10ca4edd-091a-4f34-912a-0a579c845f52-multus-socket-dir-parent\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.685400 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.684761 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10ca4edd-091a-4f34-912a-0a579c845f52-multus-cni-dir\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.685400 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.684800 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10ca4edd-091a-4f34-912a-0a579c845f52-etc-kubernetes\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.685400 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.684454 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/10ca4edd-091a-4f34-912a-0a579c845f52-host-var-lib-cni-multus\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.685400 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.684848 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8c849d95-73cd-461d-bd6b-845cb70cbd23-etc-modprobe-d\") pod \"tuned-cm7p8\" (UID: \"8c849d95-73cd-461d-bd6b-845cb70cbd23\") " pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.685400 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.684858 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10ca4edd-091a-4f34-912a-0a579c845f52-etc-kubernetes\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.685400 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.684894 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-84btw\" (UniqueName: \"kubernetes.io/projected/615d5292-c021-4a20-be77-56f9230db424-kube-api-access-84btw\") pod \"multus-additional-cni-plugins-jc4zf\" (UID: \"615d5292-c021-4a20-be77-56f9230db424\") " pod="openshift-multus/multus-additional-cni-plugins-jc4zf" May 28 15:07:31.685400 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.684898 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a2246888-2a0d-48b8-8fe6-12e283e47eb0-serviceca\") pod \"node-ca-bpxxm\" (UID: \"a2246888-2a0d-48b8-8fe6-12e283e47eb0\") " pod="openshift-image-registry/node-ca-bpxxm" May 28 15:07:31.686148 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.684933 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10ca4edd-091a-4f34-912a-0a579c845f52-multus-cni-dir\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.686148 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.684942 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c83b6a1e-4582-4530-8993-9b3ab169873d-agent-certs\") pod \"konnectivity-agent-bjm2p\" (UID: \"c83b6a1e-4582-4530-8993-9b3ab169873d\") " pod="kube-system/konnectivity-agent-bjm2p" May 28 15:07:31.686148 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.684969 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8c849d95-73cd-461d-bd6b-845cb70cbd23-etc-modprobe-d\") pod \"tuned-cm7p8\" (UID: \"8c849d95-73cd-461d-bd6b-845cb70cbd23\") " pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.686148 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.685044 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-host-kubelet\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.686148 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.685080 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-var-lib-openvswitch\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.686148 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.685113 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-etc-openvswitch\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.686148 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.685138 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-node-log\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.686148 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.685180 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/297e38b8-9a81-4112-811d-a97cfa5341ff-hosts-file\") pod \"node-resolver-kr9wd\" (UID: \"297e38b8-9a81-4112-811d-a97cfa5341ff\") " pod="openshift-dns/node-resolver-kr9wd" May 28 15:07:31.686148 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.685226 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rgnx4\" (UID: \"ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rgnx4" May 28 15:07:31.686148 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.685256 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc-socket-dir\") pod \"aws-ebs-csi-driver-node-rgnx4\" (UID: \"ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rgnx4" May 28 15:07:31.686148 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.685281 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc-registration-dir\") pod \"aws-ebs-csi-driver-node-rgnx4\" (UID: \"ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rgnx4" May 28 15:07:31.686148 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.685306 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4h9vt\" (UniqueName: \"kubernetes.io/projected/ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc-kube-api-access-4h9vt\") pod \"aws-ebs-csi-driver-node-rgnx4\" (UID: \"ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rgnx4" May 28 15:07:31.686148 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.685331 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v68kz\" (UniqueName: \"kubernetes.io/projected/8306acd6-5d60-42dd-a8e4-d21a859b6525-kube-api-access-v68kz\") pod \"network-check-target-rpkl2\" (UID: \"8306acd6-5d60-42dd-a8e4-d21a859b6525\") " pod="openshift-network-diagnostics/network-check-target-rpkl2" May 28 15:07:31.686148 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.685361 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/10ca4edd-091a-4f34-912a-0a579c845f52-cnibin\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.686148 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.685386 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/615d5292-c021-4a20-be77-56f9230db424-cnibin\") pod \"multus-additional-cni-plugins-jc4zf\" (UID: \"615d5292-c021-4a20-be77-56f9230db424\") " pod="openshift-multus/multus-additional-cni-plugins-jc4zf" May 28 15:07:31.686148 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.685409 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/615d5292-c021-4a20-be77-56f9230db424-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jc4zf\" (UID: \"615d5292-c021-4a20-be77-56f9230db424\") " pod="openshift-multus/multus-additional-cni-plugins-jc4zf" May 28 15:07:31.686148 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.685410 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/10ca4edd-091a-4f34-912a-0a579c845f52-cnibin\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.686926 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.685436 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/297e38b8-9a81-4112-811d-a97cfa5341ff-tmp-dir\") pod \"node-resolver-kr9wd\" (UID: \"297e38b8-9a81-4112-811d-a97cfa5341ff\") " pod="openshift-dns/node-resolver-kr9wd" May 28 15:07:31.686926 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.685451 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/615d5292-c021-4a20-be77-56f9230db424-cnibin\") pod \"multus-additional-cni-plugins-jc4zf\" (UID: \"615d5292-c021-4a20-be77-56f9230db424\") " pod="openshift-multus/multus-additional-cni-plugins-jc4zf" May 28 15:07:31.686926 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.685460 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3a78917-de69-42d0-8fbe-85f346276bf9-metrics-certs\") pod \"network-metrics-daemon-l9p7b\" (UID: \"e3a78917-de69-42d0-8fbe-85f346276bf9\") " pod="openshift-multus/network-metrics-daemon-l9p7b" May 28 15:07:31.686926 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.685514 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8c849d95-73cd-461d-bd6b-845cb70cbd23-etc-systemd\") pod \"tuned-cm7p8\" (UID: \"8c849d95-73cd-461d-bd6b-845cb70cbd23\") " pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.686926 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.685538 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8c849d95-73cd-461d-bd6b-845cb70cbd23-lib-modules\") pod \"tuned-cm7p8\" (UID: \"8c849d95-73cd-461d-bd6b-845cb70cbd23\") " pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.686926 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.685565 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/615d5292-c021-4a20-be77-56f9230db424-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jc4zf\" (UID: \"615d5292-c021-4a20-be77-56f9230db424\") " pod="openshift-multus/multus-additional-cni-plugins-jc4zf" May 28 15:07:31.686926 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.685603 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c849d95-73cd-461d-bd6b-845cb70cbd23-tmp\") pod \"tuned-cm7p8\" (UID: \"8c849d95-73cd-461d-bd6b-845cb70cbd23\") " pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.686926 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.685611 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8c849d95-73cd-461d-bd6b-845cb70cbd23-etc-systemd\") pod \"tuned-cm7p8\" (UID: \"8c849d95-73cd-461d-bd6b-845cb70cbd23\") " pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.686926 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.685642 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8c849d95-73cd-461d-bd6b-845cb70cbd23-lib-modules\") pod \"tuned-cm7p8\" (UID: \"8c849d95-73cd-461d-bd6b-845cb70cbd23\") " pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.686926 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.685657 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/615d5292-c021-4a20-be77-56f9230db424-cni-binary-copy\") pod \"multus-additional-cni-plugins-jc4zf\" (UID: \"615d5292-c021-4a20-be77-56f9230db424\") " pod="openshift-multus/multus-additional-cni-plugins-jc4zf" May 28 15:07:31.686926 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.685685 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8c849d95-73cd-461d-bd6b-845cb70cbd23-etc-sysctl-conf\") pod \"tuned-cm7p8\" (UID: \"8c849d95-73cd-461d-bd6b-845cb70cbd23\") " pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.686926 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.685713 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/10ca4edd-091a-4f34-912a-0a579c845f52-host-run-netns\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.686926 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.685755 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/10ca4edd-091a-4f34-912a-0a579c845f52-host-run-netns\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.686926 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.685785 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/10ca4edd-091a-4f34-912a-0a579c845f52-multus-daemon-config\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.686926 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.685810 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/615d5292-c021-4a20-be77-56f9230db424-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jc4zf\" (UID: \"615d5292-c021-4a20-be77-56f9230db424\") " pod="openshift-multus/multus-additional-cni-plugins-jc4zf" May 28 15:07:31.686926 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.685839 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c83b6a1e-4582-4530-8993-9b3ab169873d-konnectivity-ca\") pod \"konnectivity-agent-bjm2p\" (UID: \"c83b6a1e-4582-4530-8993-9b3ab169873d\") " pod="kube-system/konnectivity-agent-bjm2p" May 28 15:07:31.686926 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.685867 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/10ca4edd-091a-4f34-912a-0a579c845f52-os-release\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.687438 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.685870 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8c849d95-73cd-461d-bd6b-845cb70cbd23-etc-sysctl-conf\") pod \"tuned-cm7p8\" (UID: \"8c849d95-73cd-461d-bd6b-845cb70cbd23\") " pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.687438 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.685891 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/10ca4edd-091a-4f34-912a-0a579c845f52-host-var-lib-cni-bin\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.687438 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.685918 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/615d5292-c021-4a20-be77-56f9230db424-os-release\") pod \"multus-additional-cni-plugins-jc4zf\" (UID: \"615d5292-c021-4a20-be77-56f9230db424\") " pod="openshift-multus/multus-additional-cni-plugins-jc4zf" May 28 15:07:31.687438 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.685945 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-systemd-units\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.687438 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.685969 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-host-run-ovn-kubernetes\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.687438 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.685994 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wckqg\" (UniqueName: \"kubernetes.io/projected/44b3ce72-966f-42e8-8b3a-b9615228955c-kube-api-access-wckqg\") pod \"iptables-alerter-l6f5d\" (UID: \"44b3ce72-966f-42e8-8b3a-b9615228955c\") " pod="openshift-network-operator/iptables-alerter-l6f5d" May 28 15:07:31.687438 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.686027 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/615d5292-c021-4a20-be77-56f9230db424-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jc4zf\" (UID: \"615d5292-c021-4a20-be77-56f9230db424\") " pod="openshift-multus/multus-additional-cni-plugins-jc4zf" May 28 15:07:31.687438 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.686059 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0f511937-01b1-4615-bc4c-12832012e6ce-ovnkube-script-lib\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.687438 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.686088 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc-sys-fs\") pod \"aws-ebs-csi-driver-node-rgnx4\" (UID: \"ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rgnx4" May 28 15:07:31.687438 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.686130 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8c849d95-73cd-461d-bd6b-845cb70cbd23-etc-sysctl-d\") pod \"tuned-cm7p8\" (UID: \"8c849d95-73cd-461d-bd6b-845cb70cbd23\") " pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.687438 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.686156 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc-etc-selinux\") pod \"aws-ebs-csi-driver-node-rgnx4\" (UID: \"ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rgnx4" May 28 15:07:31.687438 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.686182 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10ca4edd-091a-4f34-912a-0a579c845f52-system-cni-dir\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.687438 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.686211 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c849d95-73cd-461d-bd6b-845cb70cbd23-etc-kubernetes\") pod \"tuned-cm7p8\" (UID: \"8c849d95-73cd-461d-bd6b-845cb70cbd23\") " pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.687438 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.686236 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8c849d95-73cd-461d-bd6b-845cb70cbd23-host\") pod \"tuned-cm7p8\" (UID: \"8c849d95-73cd-461d-bd6b-845cb70cbd23\") " pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.687438 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.686256 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/615d5292-c021-4a20-be77-56f9230db424-cni-binary-copy\") pod \"multus-additional-cni-plugins-jc4zf\" (UID: \"615d5292-c021-4a20-be77-56f9230db424\") " pod="openshift-multus/multus-additional-cni-plugins-jc4zf" May 28 15:07:31.687438 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.686261 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-host-slash\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.687438 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.686322 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpx46\" (UniqueName: \"kubernetes.io/projected/0f511937-01b1-4615-bc4c-12832012e6ce-kube-api-access-mpx46\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.688065 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.686336 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/10ca4edd-091a-4f34-912a-0a579c845f52-os-release\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.688065 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.686366 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/44b3ce72-966f-42e8-8b3a-b9615228955c-iptables-alerter-script\") pod \"iptables-alerter-l6f5d\" (UID: \"44b3ce72-966f-42e8-8b3a-b9615228955c\") " pod="openshift-network-operator/iptables-alerter-l6f5d" May 28 15:07:31.688065 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.686373 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/10ca4edd-091a-4f34-912a-0a579c845f52-multus-daemon-config\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.688065 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.686380 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/615d5292-c021-4a20-be77-56f9230db424-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jc4zf\" (UID: \"615d5292-c021-4a20-be77-56f9230db424\") " pod="openshift-multus/multus-additional-cni-plugins-jc4zf" May 28 15:07:31.688065 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.686387 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/10ca4edd-091a-4f34-912a-0a579c845f52-host-var-lib-cni-bin\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.688065 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.686395 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/10ca4edd-091a-4f34-912a-0a579c845f52-host-var-lib-kubelet\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.688065 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.686451 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8c849d95-73cd-461d-bd6b-845cb70cbd23-host\") pod \"tuned-cm7p8\" (UID: \"8c849d95-73cd-461d-bd6b-845cb70cbd23\") " pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.688065 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.686476 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/10ca4edd-091a-4f34-912a-0a579c845f52-host-var-lib-kubelet\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.688065 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.686512 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10ca4edd-091a-4f34-912a-0a579c845f52-system-cni-dir\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.688065 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.686544 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c849d95-73cd-461d-bd6b-845cb70cbd23-etc-kubernetes\") pod \"tuned-cm7p8\" (UID: \"8c849d95-73cd-461d-bd6b-845cb70cbd23\") " pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.688065 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.686557 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8c849d95-73cd-461d-bd6b-845cb70cbd23-etc-sysctl-d\") pod \"tuned-cm7p8\" (UID: \"8c849d95-73cd-461d-bd6b-845cb70cbd23\") " pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.688065 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.686557 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/615d5292-c021-4a20-be77-56f9230db424-os-release\") pod \"multus-additional-cni-plugins-jc4zf\" (UID: \"615d5292-c021-4a20-be77-56f9230db424\") " pod="openshift-multus/multus-additional-cni-plugins-jc4zf" May 28 15:07:31.688065 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.686590 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/10ca4edd-091a-4f34-912a-0a579c845f52-hostroot\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.688065 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.686646 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-log-socket\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.688065 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.686668 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/10ca4edd-091a-4f34-912a-0a579c845f52-hostroot\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.688065 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.686866 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/615d5292-c021-4a20-be77-56f9230db424-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jc4zf\" (UID: \"615d5292-c021-4a20-be77-56f9230db424\") " pod="openshift-multus/multus-additional-cni-plugins-jc4zf" May 28 15:07:31.688065 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.688042 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8c849d95-73cd-461d-bd6b-845cb70cbd23-etc-tuned\") pod \"tuned-cm7p8\" (UID: \"8c849d95-73cd-461d-bd6b-845cb70cbd23\") " pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.688629 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.688281 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c849d95-73cd-461d-bd6b-845cb70cbd23-tmp\") pod \"tuned-cm7p8\" (UID: \"8c849d95-73cd-461d-bd6b-845cb70cbd23\") " pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.691320 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.691285 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtxqw\" (UniqueName: \"kubernetes.io/projected/a2246888-2a0d-48b8-8fe6-12e283e47eb0-kube-api-access-mtxqw\") pod \"node-ca-bpxxm\" (UID: \"a2246888-2a0d-48b8-8fe6-12e283e47eb0\") " pod="openshift-image-registry/node-ca-bpxxm" May 28 15:07:31.691432 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.691328 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sxf7\" (UniqueName: \"kubernetes.io/projected/8c849d95-73cd-461d-bd6b-845cb70cbd23-kube-api-access-7sxf7\") pod \"tuned-cm7p8\" (UID: \"8c849d95-73cd-461d-bd6b-845cb70cbd23\") " pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.691856 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.691835 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-llph8\" (UniqueName: \"kubernetes.io/projected/10ca4edd-091a-4f34-912a-0a579c845f52-kube-api-access-llph8\") pod \"multus-l7mvw\" (UID: \"10ca4edd-091a-4f34-912a-0a579c845f52\") " pod="openshift-multus/multus-l7mvw" May 28 15:07:31.692088 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.692070 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84btw\" (UniqueName: \"kubernetes.io/projected/615d5292-c021-4a20-be77-56f9230db424-kube-api-access-84btw\") pod \"multus-additional-cni-plugins-jc4zf\" (UID: \"615d5292-c021-4a20-be77-56f9230db424\") " pod="openshift-multus/multus-additional-cni-plugins-jc4zf" May 28 15:07:31.787406 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.787371 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-host-slash\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.787406 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.787409 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mpx46\" (UniqueName: \"kubernetes.io/projected/0f511937-01b1-4615-bc4c-12832012e6ce-kube-api-access-mpx46\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.787662 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.787436 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/44b3ce72-966f-42e8-8b3a-b9615228955c-iptables-alerter-script\") pod \"iptables-alerter-l6f5d\" (UID: \"44b3ce72-966f-42e8-8b3a-b9615228955c\") " pod="openshift-network-operator/iptables-alerter-l6f5d" May 28 15:07:31.787662 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.787453 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-host-slash\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.787662 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.787460 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-log-socket\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.787662 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.787513 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-log-socket\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.787662 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.787528 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-host-cni-bin\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.787662 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.787554 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bkstc\" (UniqueName: \"kubernetes.io/projected/297e38b8-9a81-4112-811d-a97cfa5341ff-kube-api-access-bkstc\") pod \"node-resolver-kr9wd\" (UID: \"297e38b8-9a81-4112-811d-a97cfa5341ff\") " pod="openshift-dns/node-resolver-kr9wd" May 28 15:07:31.787662 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.787558 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-host-cni-bin\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.787662 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.787577 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/44b3ce72-966f-42e8-8b3a-b9615228955c-host-slash\") pod \"iptables-alerter-l6f5d\" (UID: \"44b3ce72-966f-42e8-8b3a-b9615228955c\") " pod="openshift-network-operator/iptables-alerter-l6f5d" May 28 15:07:31.787662 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.787602 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-run-systemd\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.787662 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.787625 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-run-openvswitch\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.787662 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.787652 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7mhbx\" (UniqueName: \"kubernetes.io/projected/e3a78917-de69-42d0-8fbe-85f346276bf9-kube-api-access-7mhbx\") pod \"network-metrics-daemon-l9p7b\" (UID: \"e3a78917-de69-42d0-8fbe-85f346276bf9\") " pod="openshift-multus/network-metrics-daemon-l9p7b" May 28 15:07:31.788159 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.787679 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-host-run-netns\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.788159 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.787675 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/44b3ce72-966f-42e8-8b3a-b9615228955c-host-slash\") pod \"iptables-alerter-l6f5d\" (UID: \"44b3ce72-966f-42e8-8b3a-b9615228955c\") " pod="openshift-network-operator/iptables-alerter-l6f5d" May 28 15:07:31.788159 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.787703 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0f511937-01b1-4615-bc4c-12832012e6ce-ovnkube-config\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.788159 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.787730 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-host-cni-netd\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.788159 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.787737 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-run-systemd\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.788159 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.787766 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-host-run-netns\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.788159 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.787778 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-host-cni-netd\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.788159 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.787768 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-run-ovn\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.788159 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.787797 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-run-ovn\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.788159 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.787736 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-run-openvswitch\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.788159 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.787824 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0f511937-01b1-4615-bc4c-12832012e6ce-env-overrides\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.788159 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.787941 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc-device-dir\") pod \"aws-ebs-csi-driver-node-rgnx4\" (UID: \"ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rgnx4" May 28 15:07:31.788159 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.787994 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.788159 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.788021 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0f511937-01b1-4615-bc4c-12832012e6ce-ovn-node-metrics-cert\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.788159 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.788029 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/44b3ce72-966f-42e8-8b3a-b9615228955c-iptables-alerter-script\") pod \"iptables-alerter-l6f5d\" (UID: \"44b3ce72-966f-42e8-8b3a-b9615228955c\") " pod="openshift-network-operator/iptables-alerter-l6f5d" May 28 15:07:31.788159 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.788055 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c83b6a1e-4582-4530-8993-9b3ab169873d-agent-certs\") pod \"konnectivity-agent-bjm2p\" (UID: \"c83b6a1e-4582-4530-8993-9b3ab169873d\") " pod="kube-system/konnectivity-agent-bjm2p" May 28 15:07:31.788159 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.788073 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.788943 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.788081 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-host-kubelet\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.788943 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.788088 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc-device-dir\") pod \"aws-ebs-csi-driver-node-rgnx4\" (UID: \"ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rgnx4" May 28 15:07:31.788943 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.788114 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-var-lib-openvswitch\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.788943 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.788121 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-host-kubelet\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.788943 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.788140 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-etc-openvswitch\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.788943 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.788162 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-var-lib-openvswitch\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.788943 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.788165 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-node-log\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.788943 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.788194 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/297e38b8-9a81-4112-811d-a97cfa5341ff-hosts-file\") pod \"node-resolver-kr9wd\" (UID: \"297e38b8-9a81-4112-811d-a97cfa5341ff\") " pod="openshift-dns/node-resolver-kr9wd" May 28 15:07:31.788943 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.788219 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rgnx4\" (UID: \"ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rgnx4" May 28 15:07:31.788943 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.788220 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-etc-openvswitch\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.788943 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.788243 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc-socket-dir\") pod \"aws-ebs-csi-driver-node-rgnx4\" (UID: \"ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rgnx4" May 28 15:07:31.788943 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.788265 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-node-log\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.788943 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.788271 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc-registration-dir\") pod \"aws-ebs-csi-driver-node-rgnx4\" (UID: \"ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rgnx4" May 28 15:07:31.788943 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.788283 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/297e38b8-9a81-4112-811d-a97cfa5341ff-hosts-file\") pod \"node-resolver-kr9wd\" (UID: \"297e38b8-9a81-4112-811d-a97cfa5341ff\") " pod="openshift-dns/node-resolver-kr9wd" May 28 15:07:31.788943 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.788297 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4h9vt\" (UniqueName: \"kubernetes.io/projected/ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc-kube-api-access-4h9vt\") pod \"aws-ebs-csi-driver-node-rgnx4\" (UID: \"ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rgnx4" May 28 15:07:31.788943 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.788319 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0f511937-01b1-4615-bc4c-12832012e6ce-env-overrides\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.788943 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.788322 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rgnx4\" (UID: \"ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rgnx4" May 28 15:07:31.789598 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.788322 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v68kz\" (UniqueName: \"kubernetes.io/projected/8306acd6-5d60-42dd-a8e4-d21a859b6525-kube-api-access-v68kz\") pod \"network-check-target-rpkl2\" (UID: \"8306acd6-5d60-42dd-a8e4-d21a859b6525\") " pod="openshift-network-diagnostics/network-check-target-rpkl2" May 28 15:07:31.789598 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.788365 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc-registration-dir\") pod \"aws-ebs-csi-driver-node-rgnx4\" (UID: \"ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rgnx4" May 28 15:07:31.789598 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.788373 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/297e38b8-9a81-4112-811d-a97cfa5341ff-tmp-dir\") pod \"node-resolver-kr9wd\" (UID: \"297e38b8-9a81-4112-811d-a97cfa5341ff\") " pod="openshift-dns/node-resolver-kr9wd" May 28 15:07:31.789598 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.788366 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0f511937-01b1-4615-bc4c-12832012e6ce-ovnkube-config\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.789598 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.788394 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc-socket-dir\") pod \"aws-ebs-csi-driver-node-rgnx4\" (UID: \"ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rgnx4" May 28 15:07:31.789598 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.788417 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3a78917-de69-42d0-8fbe-85f346276bf9-metrics-certs\") pod \"network-metrics-daemon-l9p7b\" (UID: \"e3a78917-de69-42d0-8fbe-85f346276bf9\") " pod="openshift-multus/network-metrics-daemon-l9p7b" May 28 15:07:31.789598 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.788451 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c83b6a1e-4582-4530-8993-9b3ab169873d-konnectivity-ca\") pod \"konnectivity-agent-bjm2p\" (UID: \"c83b6a1e-4582-4530-8993-9b3ab169873d\") " pod="kube-system/konnectivity-agent-bjm2p" May 28 15:07:31.789598 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.788520 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-systemd-units\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.789598 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.788547 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-host-run-ovn-kubernetes\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.789598 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.788575 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wckqg\" (UniqueName: \"kubernetes.io/projected/44b3ce72-966f-42e8-8b3a-b9615228955c-kube-api-access-wckqg\") pod \"iptables-alerter-l6f5d\" (UID: \"44b3ce72-966f-42e8-8b3a-b9615228955c\") " pod="openshift-network-operator/iptables-alerter-l6f5d" May 28 15:07:31.789598 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:31.788590 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 28 15:07:31.789598 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.788600 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0f511937-01b1-4615-bc4c-12832012e6ce-ovnkube-script-lib\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.789598 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.788608 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-host-run-ovn-kubernetes\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.789598 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.788624 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc-sys-fs\") pod \"aws-ebs-csi-driver-node-rgnx4\" (UID: \"ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rgnx4" May 28 15:07:31.789598 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.788654 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc-etc-selinux\") pod \"aws-ebs-csi-driver-node-rgnx4\" (UID: \"ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rgnx4" May 28 15:07:31.789598 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.788700 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/297e38b8-9a81-4112-811d-a97cfa5341ff-tmp-dir\") pod \"node-resolver-kr9wd\" (UID: \"297e38b8-9a81-4112-811d-a97cfa5341ff\") " pod="openshift-dns/node-resolver-kr9wd" May 28 15:07:31.789598 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.788592 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0f511937-01b1-4615-bc4c-12832012e6ce-systemd-units\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.790269 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:31.788757 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3a78917-de69-42d0-8fbe-85f346276bf9-metrics-certs podName:e3a78917-de69-42d0-8fbe-85f346276bf9 nodeName:}" failed. No retries permitted until 2026-05-28 15:07:32.288689588 +0000 UTC m=+3.177676651 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3a78917-de69-42d0-8fbe-85f346276bf9-metrics-certs") pod "network-metrics-daemon-l9p7b" (UID: "e3a78917-de69-42d0-8fbe-85f346276bf9") : object "openshift-multus"/"metrics-daemon-secret" not registered May 28 15:07:31.790269 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.788775 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc-sys-fs\") pod \"aws-ebs-csi-driver-node-rgnx4\" (UID: \"ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rgnx4" May 28 15:07:31.790269 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.788841 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc-etc-selinux\") pod \"aws-ebs-csi-driver-node-rgnx4\" (UID: \"ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rgnx4" May 28 15:07:31.790269 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.789082 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0f511937-01b1-4615-bc4c-12832012e6ce-ovnkube-script-lib\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.790269 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.789137 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c83b6a1e-4582-4530-8993-9b3ab169873d-konnectivity-ca\") pod \"konnectivity-agent-bjm2p\" (UID: \"c83b6a1e-4582-4530-8993-9b3ab169873d\") " pod="kube-system/konnectivity-agent-bjm2p" May 28 15:07:31.790829 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.790806 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0f511937-01b1-4615-bc4c-12832012e6ce-ovn-node-metrics-cert\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.791006 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.790984 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c83b6a1e-4582-4530-8993-9b3ab169873d-agent-certs\") pod \"konnectivity-agent-bjm2p\" (UID: \"c83b6a1e-4582-4530-8993-9b3ab169873d\") " pod="kube-system/konnectivity-agent-bjm2p" May 28 15:07:31.797263 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:31.797243 2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 28 15:07:31.797374 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:31.797266 2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 28 15:07:31.797374 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:31.797283 2559 projected.go:194] Error preparing data for projected volume kube-api-access-v68kz for pod openshift-network-diagnostics/network-check-target-rpkl2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 15:07:31.797374 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:31.797350 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8306acd6-5d60-42dd-a8e4-d21a859b6525-kube-api-access-v68kz podName:8306acd6-5d60-42dd-a8e4-d21a859b6525 nodeName:}" failed. No retries permitted until 2026-05-28 15:07:32.29733503 +0000 UTC m=+3.186322119 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-v68kz" (UniqueName: "kubernetes.io/projected/8306acd6-5d60-42dd-a8e4-d21a859b6525-kube-api-access-v68kz") pod "network-check-target-rpkl2" (UID: "8306acd6-5d60-42dd-a8e4-d21a859b6525") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 15:07:31.799766 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.799740 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkstc\" (UniqueName: \"kubernetes.io/projected/297e38b8-9a81-4112-811d-a97cfa5341ff-kube-api-access-bkstc\") pod \"node-resolver-kr9wd\" (UID: \"297e38b8-9a81-4112-811d-a97cfa5341ff\") " pod="openshift-dns/node-resolver-kr9wd" May 28 15:07:31.799766 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.799752 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wckqg\" (UniqueName: \"kubernetes.io/projected/44b3ce72-966f-42e8-8b3a-b9615228955c-kube-api-access-wckqg\") pod \"iptables-alerter-l6f5d\" (UID: \"44b3ce72-966f-42e8-8b3a-b9615228955c\") " pod="openshift-network-operator/iptables-alerter-l6f5d" May 28 15:07:31.799928 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.799851 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mhbx\" (UniqueName: \"kubernetes.io/projected/e3a78917-de69-42d0-8fbe-85f346276bf9-kube-api-access-7mhbx\") pod \"network-metrics-daemon-l9p7b\" (UID: \"e3a78917-de69-42d0-8fbe-85f346276bf9\") " pod="openshift-multus/network-metrics-daemon-l9p7b" May 28 15:07:31.800473 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.800452 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpx46\" (UniqueName: \"kubernetes.io/projected/0f511937-01b1-4615-bc4c-12832012e6ce-kube-api-access-mpx46\") pod \"ovnkube-node-xcg2z\" (UID: \"0f511937-01b1-4615-bc4c-12832012e6ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.800584 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.800556 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4h9vt\" (UniqueName: \"kubernetes.io/projected/ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc-kube-api-access-4h9vt\") pod \"aws-ebs-csi-driver-node-rgnx4\" (UID: \"ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rgnx4" May 28 15:07:31.880623 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.880560 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bpxxm" May 28 15:07:31.888227 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.888200 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jc4zf" May 28 15:07:31.896897 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.896880 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-l7mvw" May 28 15:07:31.902458 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.902440 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" May 28 15:07:31.909940 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.909924 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bjm2p" May 28 15:07:31.919517 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.919497 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:31.925159 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.925140 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rgnx4" May 28 15:07:31.933625 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.933605 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-l6f5d" May 28 15:07:31.940137 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:31.940119 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kr9wd" May 28 15:07:32.033231 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:32.033203 2559 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" May 28 15:07:32.292760 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:32.292678 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3a78917-de69-42d0-8fbe-85f346276bf9-metrics-certs\") pod \"network-metrics-daemon-l9p7b\" (UID: \"e3a78917-de69-42d0-8fbe-85f346276bf9\") " pod="openshift-multus/network-metrics-daemon-l9p7b" May 28 15:07:32.292934 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:32.292808 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 28 15:07:32.292934 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:32.292872 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3a78917-de69-42d0-8fbe-85f346276bf9-metrics-certs podName:e3a78917-de69-42d0-8fbe-85f346276bf9 nodeName:}" failed. No retries permitted until 2026-05-28 15:07:33.292851095 +0000 UTC m=+4.181838157 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3a78917-de69-42d0-8fbe-85f346276bf9-metrics-certs") pod "network-metrics-daemon-l9p7b" (UID: "e3a78917-de69-42d0-8fbe-85f346276bf9") : object "openshift-multus"/"metrics-daemon-secret" not registered May 28 15:07:32.320011 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:32.319964 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod297e38b8_9a81_4112_811d_a97cfa5341ff.slice/crio-8e675384852a003e756110296f135a1984632df77606486f8b1151df348237f8 WatchSource:0}: Error finding container 8e675384852a003e756110296f135a1984632df77606486f8b1151df348237f8: Status 404 returned error can't find the container with id 8e675384852a003e756110296f135a1984632df77606486f8b1151df348237f8 May 28 15:07:32.320980 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:32.320954 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f511937_01b1_4615_bc4c_12832012e6ce.slice/crio-ab5e5ad53a12d0a88001c6d4051674d31796c7c8367cc056b89437dff9029941 WatchSource:0}: Error finding container ab5e5ad53a12d0a88001c6d4051674d31796c7c8367cc056b89437dff9029941: Status 404 returned error can't find the container with id ab5e5ad53a12d0a88001c6d4051674d31796c7c8367cc056b89437dff9029941 May 28 15:07:32.322275 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:32.322098 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc83b6a1e_4582_4530_8993_9b3ab169873d.slice/crio-e850f8612fcb006bae40d540def0f736a94552e160bd13fdccf5b386aff93c5b WatchSource:0}: Error finding container e850f8612fcb006bae40d540def0f736a94552e160bd13fdccf5b386aff93c5b: Status 404 returned error can't find the container with id e850f8612fcb006bae40d540def0f736a94552e160bd13fdccf5b386aff93c5b May 28 15:07:32.325084 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:32.325063 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod615d5292_c021_4a20_be77_56f9230db424.slice/crio-41f75e28d5838428857e22201977b2711a8bb8b50f1d7c61052f984193097ada WatchSource:0}: Error finding container 41f75e28d5838428857e22201977b2711a8bb8b50f1d7c61052f984193097ada: Status 404 returned error can't find the container with id 41f75e28d5838428857e22201977b2711a8bb8b50f1d7c61052f984193097ada May 28 15:07:32.326213 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:32.326196 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10ca4edd_091a_4f34_912a_0a579c845f52.slice/crio-dd6cd555115c84c5db9399ee03f7683e1638b9e213265323d589a5f317f97647 WatchSource:0}: Error finding container dd6cd555115c84c5db9399ee03f7683e1638b9e213265323d589a5f317f97647: Status 404 returned error can't find the container with id dd6cd555115c84c5db9399ee03f7683e1638b9e213265323d589a5f317f97647 May 28 15:07:32.327801 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:32.327692 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2246888_2a0d_48b8_8fe6_12e283e47eb0.slice/crio-12af9b9599e7d2f42dafed9460bff623e7bac6bab36fd7e6d8a2c538911e28c5 WatchSource:0}: Error finding container 12af9b9599e7d2f42dafed9460bff623e7bac6bab36fd7e6d8a2c538911e28c5: Status 404 returned error can't find the container with id 12af9b9599e7d2f42dafed9460bff623e7bac6bab36fd7e6d8a2c538911e28c5 May 28 15:07:32.328474 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:32.328440 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44b3ce72_966f_42e8_8b3a_b9615228955c.slice/crio-7882bd2c6ed2df916023f7f0676a69fa6ce74d3fd5bc9ab0c99f2627d8ca6a26 WatchSource:0}: Error finding container 7882bd2c6ed2df916023f7f0676a69fa6ce74d3fd5bc9ab0c99f2627d8ca6a26: Status 404 returned error can't find the container with id 7882bd2c6ed2df916023f7f0676a69fa6ce74d3fd5bc9ab0c99f2627d8ca6a26 May 28 15:07:32.329537 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:32.329517 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce0aab3a_a9eb_439d_b8ca_83f238a0e1dc.slice/crio-bc4cc248116b95da3377c53f9ed9784d8162bfd87b8fee8ef501bfd3c897c99a WatchSource:0}: Error finding container bc4cc248116b95da3377c53f9ed9784d8162bfd87b8fee8ef501bfd3c897c99a: Status 404 returned error can't find the container with id bc4cc248116b95da3377c53f9ed9784d8162bfd87b8fee8ef501bfd3c897c99a May 28 15:07:32.330692 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:07:32.330435 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c849d95_73cd_461d_bd6b_845cb70cbd23.slice/crio-0c9b0de9fb2bd3995964c5b47360006c7a21ec9e5ab792064f6103c681914f06 WatchSource:0}: Error finding container 0c9b0de9fb2bd3995964c5b47360006c7a21ec9e5ab792064f6103c681914f06: Status 404 returned error can't find the container with id 0c9b0de9fb2bd3995964c5b47360006c7a21ec9e5ab792064f6103c681914f06 May 28 15:07:32.393334 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:32.393199 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v68kz\" (UniqueName: \"kubernetes.io/projected/8306acd6-5d60-42dd-a8e4-d21a859b6525-kube-api-access-v68kz\") pod \"network-check-target-rpkl2\" (UID: \"8306acd6-5d60-42dd-a8e4-d21a859b6525\") " pod="openshift-network-diagnostics/network-check-target-rpkl2" May 28 15:07:32.393406 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:32.393340 2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 28 15:07:32.393406 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:32.393359 2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 28 15:07:32.393406 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:32.393371 2559 projected.go:194] Error preparing data for projected volume kube-api-access-v68kz for pod openshift-network-diagnostics/network-check-target-rpkl2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 15:07:32.393553 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:32.393428 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8306acd6-5d60-42dd-a8e4-d21a859b6525-kube-api-access-v68kz podName:8306acd6-5d60-42dd-a8e4-d21a859b6525 nodeName:}" failed. No retries permitted until 2026-05-28 15:07:33.393412634 +0000 UTC m=+4.282399713 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-v68kz" (UniqueName: "kubernetes.io/projected/8306acd6-5d60-42dd-a8e4-d21a859b6525-kube-api-access-v68kz") pod "network-check-target-rpkl2" (UID: "8306acd6-5d60-42dd-a8e4-d21a859b6525") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 15:07:32.612096 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:32.612068 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-27 15:02:30 +0000 UTC" deadline="2027-11-21 09:39:48.296771099 +0000 UTC" May 28 15:07:32.612096 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:32.612094 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13002h32m15.684680187s" May 28 15:07:32.715722 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:32.715687 2559 generic.go:358] "Generic (PLEG): container finished" podID="764dacf5d913e4d5bcd2ceb0fe8fea04" containerID="71d5b3c6348418aeaac967904dba51bbfcfea8bc5b91f9970733811fdf79394f" exitCode=0 May 28 15:07:32.715843 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:32.715779 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-141.ec2.internal" event={"ID":"764dacf5d913e4d5bcd2ceb0fe8fea04","Type":"ContainerDied","Data":"71d5b3c6348418aeaac967904dba51bbfcfea8bc5b91f9970733811fdf79394f"} May 28 15:07:32.716686 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:32.716665 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" event={"ID":"8c849d95-73cd-461d-bd6b-845cb70cbd23","Type":"ContainerStarted","Data":"0c9b0de9fb2bd3995964c5b47360006c7a21ec9e5ab792064f6103c681914f06"} May 28 15:07:32.720592 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:32.720569 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rgnx4" event={"ID":"ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc","Type":"ContainerStarted","Data":"bc4cc248116b95da3377c53f9ed9784d8162bfd87b8fee8ef501bfd3c897c99a"} May 28 15:07:32.721527 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:32.721504 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bpxxm" event={"ID":"a2246888-2a0d-48b8-8fe6-12e283e47eb0","Type":"ContainerStarted","Data":"12af9b9599e7d2f42dafed9460bff623e7bac6bab36fd7e6d8a2c538911e28c5"} May 28 15:07:32.722327 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:32.722302 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-l6f5d" event={"ID":"44b3ce72-966f-42e8-8b3a-b9615228955c","Type":"ContainerStarted","Data":"7882bd2c6ed2df916023f7f0676a69fa6ce74d3fd5bc9ab0c99f2627d8ca6a26"} May 28 15:07:32.723122 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:32.723104 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-l7mvw" event={"ID":"10ca4edd-091a-4f34-912a-0a579c845f52","Type":"ContainerStarted","Data":"dd6cd555115c84c5db9399ee03f7683e1638b9e213265323d589a5f317f97647"} May 28 15:07:32.723971 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:32.723946 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jc4zf" event={"ID":"615d5292-c021-4a20-be77-56f9230db424","Type":"ContainerStarted","Data":"41f75e28d5838428857e22201977b2711a8bb8b50f1d7c61052f984193097ada"} May 28 15:07:32.724894 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:32.724876 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kr9wd" event={"ID":"297e38b8-9a81-4112-811d-a97cfa5341ff","Type":"ContainerStarted","Data":"8e675384852a003e756110296f135a1984632df77606486f8b1151df348237f8"} May 28 15:07:32.726075 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:32.726058 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-141.ec2.internal" event={"ID":"f46265c427d4cbfcaecf493db63aaa49","Type":"ContainerStarted","Data":"4507d1efed38206a5de6821ff8e785067cfe8b6df7b30d5cd41ab0fd41b44703"} May 28 15:07:32.727003 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:32.726985 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bjm2p" event={"ID":"c83b6a1e-4582-4530-8993-9b3ab169873d","Type":"ContainerStarted","Data":"e850f8612fcb006bae40d540def0f736a94552e160bd13fdccf5b386aff93c5b"} May 28 15:07:32.727863 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:32.727840 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" event={"ID":"0f511937-01b1-4615-bc4c-12832012e6ce","Type":"ContainerStarted","Data":"ab5e5ad53a12d0a88001c6d4051674d31796c7c8367cc056b89437dff9029941"} May 28 15:07:32.742323 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:32.742277 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-141.ec2.internal" podStartSLOduration=2.742263072 podStartE2EDuration="2.742263072s" podCreationTimestamp="2026-05-28 15:07:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 15:07:32.741922135 +0000 UTC m=+3.630909221" watchObservedRunningTime="2026-05-28 15:07:32.742263072 +0000 UTC m=+3.631250157" May 28 15:07:33.300828 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:33.300744 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3a78917-de69-42d0-8fbe-85f346276bf9-metrics-certs\") pod \"network-metrics-daemon-l9p7b\" (UID: \"e3a78917-de69-42d0-8fbe-85f346276bf9\") " pod="openshift-multus/network-metrics-daemon-l9p7b" May 28 15:07:33.300979 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:33.300903 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 28 15:07:33.301041 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:33.301004 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3a78917-de69-42d0-8fbe-85f346276bf9-metrics-certs podName:e3a78917-de69-42d0-8fbe-85f346276bf9 nodeName:}" failed. No retries permitted until 2026-05-28 15:07:35.300985367 +0000 UTC m=+6.189972436 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3a78917-de69-42d0-8fbe-85f346276bf9-metrics-certs") pod "network-metrics-daemon-l9p7b" (UID: "e3a78917-de69-42d0-8fbe-85f346276bf9") : object "openshift-multus"/"metrics-daemon-secret" not registered May 28 15:07:33.401758 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:33.401720 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v68kz\" (UniqueName: \"kubernetes.io/projected/8306acd6-5d60-42dd-a8e4-d21a859b6525-kube-api-access-v68kz\") pod \"network-check-target-rpkl2\" (UID: \"8306acd6-5d60-42dd-a8e4-d21a859b6525\") " pod="openshift-network-diagnostics/network-check-target-rpkl2" May 28 15:07:33.402002 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:33.401910 2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 28 15:07:33.402002 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:33.401927 2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 28 15:07:33.402002 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:33.401940 2559 projected.go:194] Error preparing data for projected volume kube-api-access-v68kz for pod openshift-network-diagnostics/network-check-target-rpkl2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 15:07:33.402002 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:33.401995 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8306acd6-5d60-42dd-a8e4-d21a859b6525-kube-api-access-v68kz podName:8306acd6-5d60-42dd-a8e4-d21a859b6525 nodeName:}" failed. No retries permitted until 2026-05-28 15:07:35.401977399 +0000 UTC m=+6.290964468 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-v68kz" (UniqueName: "kubernetes.io/projected/8306acd6-5d60-42dd-a8e4-d21a859b6525-kube-api-access-v68kz") pod "network-check-target-rpkl2" (UID: "8306acd6-5d60-42dd-a8e4-d21a859b6525") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 15:07:33.709702 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:33.709633 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9p7b" May 28 15:07:33.710130 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:33.709801 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9p7b" podUID="e3a78917-de69-42d0-8fbe-85f346276bf9" May 28 15:07:33.710130 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:33.709823 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rpkl2" May 28 15:07:33.710130 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:33.709952 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rpkl2" podUID="8306acd6-5d60-42dd-a8e4-d21a859b6525" May 28 15:07:33.745750 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:33.745711 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-141.ec2.internal" event={"ID":"764dacf5d913e4d5bcd2ceb0fe8fea04","Type":"ContainerStarted","Data":"f232420f4349668978c21fd5c9ee60aebd852de79f1778a008131c019ffb7d1a"} May 28 15:07:33.760436 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:33.760388 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-141.ec2.internal" podStartSLOduration=3.760371969 podStartE2EDuration="3.760371969s" podCreationTimestamp="2026-05-28 15:07:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 15:07:33.7597101 +0000 UTC m=+4.648697186" watchObservedRunningTime="2026-05-28 15:07:33.760371969 +0000 UTC m=+4.649359059" May 28 15:07:35.318836 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:35.318343 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3a78917-de69-42d0-8fbe-85f346276bf9-metrics-certs\") pod \"network-metrics-daemon-l9p7b\" (UID: \"e3a78917-de69-42d0-8fbe-85f346276bf9\") " pod="openshift-multus/network-metrics-daemon-l9p7b" May 28 15:07:35.318836 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:35.318518 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 28 15:07:35.318836 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:35.318592 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3a78917-de69-42d0-8fbe-85f346276bf9-metrics-certs podName:e3a78917-de69-42d0-8fbe-85f346276bf9 nodeName:}" failed. No retries permitted until 2026-05-28 15:07:39.318571974 +0000 UTC m=+10.207559040 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3a78917-de69-42d0-8fbe-85f346276bf9-metrics-certs") pod "network-metrics-daemon-l9p7b" (UID: "e3a78917-de69-42d0-8fbe-85f346276bf9") : object "openshift-multus"/"metrics-daemon-secret" not registered May 28 15:07:35.419663 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:35.419619 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v68kz\" (UniqueName: \"kubernetes.io/projected/8306acd6-5d60-42dd-a8e4-d21a859b6525-kube-api-access-v68kz\") pod \"network-check-target-rpkl2\" (UID: \"8306acd6-5d60-42dd-a8e4-d21a859b6525\") " pod="openshift-network-diagnostics/network-check-target-rpkl2" May 28 15:07:35.419843 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:35.419800 2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 28 15:07:35.419843 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:35.419821 2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 28 15:07:35.419843 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:35.419834 2559 projected.go:194] Error preparing data for projected volume kube-api-access-v68kz for pod openshift-network-diagnostics/network-check-target-rpkl2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 15:07:35.420013 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:35.419898 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8306acd6-5d60-42dd-a8e4-d21a859b6525-kube-api-access-v68kz podName:8306acd6-5d60-42dd-a8e4-d21a859b6525 nodeName:}" failed. No retries permitted until 2026-05-28 15:07:39.419878889 +0000 UTC m=+10.308865963 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-v68kz" (UniqueName: "kubernetes.io/projected/8306acd6-5d60-42dd-a8e4-d21a859b6525-kube-api-access-v68kz") pod "network-check-target-rpkl2" (UID: "8306acd6-5d60-42dd-a8e4-d21a859b6525") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 15:07:35.709025 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:35.708838 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9p7b" May 28 15:07:35.709025 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:35.708884 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rpkl2" May 28 15:07:35.709025 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:35.708984 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9p7b" podUID="e3a78917-de69-42d0-8fbe-85f346276bf9" May 28 15:07:35.709303 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:35.709107 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rpkl2" podUID="8306acd6-5d60-42dd-a8e4-d21a859b6525" May 28 15:07:37.708224 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:37.708173 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rpkl2" May 28 15:07:37.708671 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:37.708306 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rpkl2" podUID="8306acd6-5d60-42dd-a8e4-d21a859b6525" May 28 15:07:37.708671 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:37.708365 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9p7b" May 28 15:07:37.708671 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:37.708504 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9p7b" podUID="e3a78917-de69-42d0-8fbe-85f346276bf9" May 28 15:07:39.347628 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:39.347563 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3a78917-de69-42d0-8fbe-85f346276bf9-metrics-certs\") pod \"network-metrics-daemon-l9p7b\" (UID: \"e3a78917-de69-42d0-8fbe-85f346276bf9\") " pod="openshift-multus/network-metrics-daemon-l9p7b" May 28 15:07:39.347987 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:39.347725 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 28 15:07:39.347987 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:39.347796 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3a78917-de69-42d0-8fbe-85f346276bf9-metrics-certs podName:e3a78917-de69-42d0-8fbe-85f346276bf9 nodeName:}" failed. No retries permitted until 2026-05-28 15:07:47.347776418 +0000 UTC m=+18.236763488 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3a78917-de69-42d0-8fbe-85f346276bf9-metrics-certs") pod "network-metrics-daemon-l9p7b" (UID: "e3a78917-de69-42d0-8fbe-85f346276bf9") : object "openshift-multus"/"metrics-daemon-secret" not registered May 28 15:07:39.448680 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:39.448649 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v68kz\" (UniqueName: \"kubernetes.io/projected/8306acd6-5d60-42dd-a8e4-d21a859b6525-kube-api-access-v68kz\") pod \"network-check-target-rpkl2\" (UID: \"8306acd6-5d60-42dd-a8e4-d21a859b6525\") " pod="openshift-network-diagnostics/network-check-target-rpkl2" May 28 15:07:39.448848 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:39.448825 2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 28 15:07:39.448914 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:39.448849 2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 28 15:07:39.448914 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:39.448862 2559 projected.go:194] Error preparing data for projected volume kube-api-access-v68kz for pod openshift-network-diagnostics/network-check-target-rpkl2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 15:07:39.449009 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:39.448925 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8306acd6-5d60-42dd-a8e4-d21a859b6525-kube-api-access-v68kz podName:8306acd6-5d60-42dd-a8e4-d21a859b6525 nodeName:}" failed. No retries permitted until 2026-05-28 15:07:47.448905875 +0000 UTC m=+18.337892938 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-v68kz" (UniqueName: "kubernetes.io/projected/8306acd6-5d60-42dd-a8e4-d21a859b6525-kube-api-access-v68kz") pod "network-check-target-rpkl2" (UID: "8306acd6-5d60-42dd-a8e4-d21a859b6525") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 15:07:39.708771 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:39.708681 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9p7b" May 28 15:07:39.708771 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:39.708705 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rpkl2" May 28 15:07:39.709005 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:39.708799 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9p7b" podUID="e3a78917-de69-42d0-8fbe-85f346276bf9" May 28 15:07:39.709005 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:39.708923 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rpkl2" podUID="8306acd6-5d60-42dd-a8e4-d21a859b6525" May 28 15:07:40.882644 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:40.882591 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-4jdp2"] May 28 15:07:40.898136 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:40.898109 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4jdp2" May 28 15:07:40.898287 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:40.898192 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4jdp2" podUID="b64275b4-68e5-42e4-ac36-ac46f6d55524" May 28 15:07:40.958871 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:40.958834 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b64275b4-68e5-42e4-ac36-ac46f6d55524-original-pull-secret\") pod \"global-pull-secret-syncer-4jdp2\" (UID: \"b64275b4-68e5-42e4-ac36-ac46f6d55524\") " pod="kube-system/global-pull-secret-syncer-4jdp2" May 28 15:07:40.959057 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:40.958883 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b64275b4-68e5-42e4-ac36-ac46f6d55524-kubelet-config\") pod \"global-pull-secret-syncer-4jdp2\" (UID: \"b64275b4-68e5-42e4-ac36-ac46f6d55524\") " pod="kube-system/global-pull-secret-syncer-4jdp2" May 28 15:07:40.959057 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:40.958954 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b64275b4-68e5-42e4-ac36-ac46f6d55524-dbus\") pod \"global-pull-secret-syncer-4jdp2\" (UID: \"b64275b4-68e5-42e4-ac36-ac46f6d55524\") " pod="kube-system/global-pull-secret-syncer-4jdp2" May 28 15:07:41.059886 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:41.059564 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b64275b4-68e5-42e4-ac36-ac46f6d55524-original-pull-secret\") pod \"global-pull-secret-syncer-4jdp2\" (UID: \"b64275b4-68e5-42e4-ac36-ac46f6d55524\") " pod="kube-system/global-pull-secret-syncer-4jdp2" May 28 15:07:41.059886 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:41.059619 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b64275b4-68e5-42e4-ac36-ac46f6d55524-kubelet-config\") pod \"global-pull-secret-syncer-4jdp2\" (UID: \"b64275b4-68e5-42e4-ac36-ac46f6d55524\") " pod="kube-system/global-pull-secret-syncer-4jdp2" May 28 15:07:41.059886 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:41.059645 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b64275b4-68e5-42e4-ac36-ac46f6d55524-dbus\") pod \"global-pull-secret-syncer-4jdp2\" (UID: \"b64275b4-68e5-42e4-ac36-ac46f6d55524\") " pod="kube-system/global-pull-secret-syncer-4jdp2" May 28 15:07:41.059886 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:41.059690 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 28 15:07:41.059886 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:41.059750 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b64275b4-68e5-42e4-ac36-ac46f6d55524-original-pull-secret podName:b64275b4-68e5-42e4-ac36-ac46f6d55524 nodeName:}" failed. No retries permitted until 2026-05-28 15:07:41.559733094 +0000 UTC m=+12.448720162 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b64275b4-68e5-42e4-ac36-ac46f6d55524-original-pull-secret") pod "global-pull-secret-syncer-4jdp2" (UID: "b64275b4-68e5-42e4-ac36-ac46f6d55524") : object "kube-system"/"original-pull-secret" not registered May 28 15:07:41.059886 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:41.059807 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b64275b4-68e5-42e4-ac36-ac46f6d55524-dbus\") pod \"global-pull-secret-syncer-4jdp2\" (UID: \"b64275b4-68e5-42e4-ac36-ac46f6d55524\") " pod="kube-system/global-pull-secret-syncer-4jdp2" May 28 15:07:41.059886 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:41.059856 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b64275b4-68e5-42e4-ac36-ac46f6d55524-kubelet-config\") pod \"global-pull-secret-syncer-4jdp2\" (UID: \"b64275b4-68e5-42e4-ac36-ac46f6d55524\") " pod="kube-system/global-pull-secret-syncer-4jdp2" May 28 15:07:41.562919 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:41.562884 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b64275b4-68e5-42e4-ac36-ac46f6d55524-original-pull-secret\") pod \"global-pull-secret-syncer-4jdp2\" (UID: \"b64275b4-68e5-42e4-ac36-ac46f6d55524\") " pod="kube-system/global-pull-secret-syncer-4jdp2" May 28 15:07:41.563099 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:41.563072 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 28 15:07:41.563194 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:41.563135 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b64275b4-68e5-42e4-ac36-ac46f6d55524-original-pull-secret podName:b64275b4-68e5-42e4-ac36-ac46f6d55524 nodeName:}" failed. No retries permitted until 2026-05-28 15:07:42.563116661 +0000 UTC m=+13.452103728 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b64275b4-68e5-42e4-ac36-ac46f6d55524-original-pull-secret") pod "global-pull-secret-syncer-4jdp2" (UID: "b64275b4-68e5-42e4-ac36-ac46f6d55524") : object "kube-system"/"original-pull-secret" not registered May 28 15:07:41.708729 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:41.708704 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9p7b" May 28 15:07:41.708853 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:41.708832 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9p7b" podUID="e3a78917-de69-42d0-8fbe-85f346276bf9" May 28 15:07:41.709055 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:41.709039 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rpkl2" May 28 15:07:41.709156 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:41.709136 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rpkl2" podUID="8306acd6-5d60-42dd-a8e4-d21a859b6525" May 28 15:07:42.575450 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:42.575171 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b64275b4-68e5-42e4-ac36-ac46f6d55524-original-pull-secret\") pod \"global-pull-secret-syncer-4jdp2\" (UID: \"b64275b4-68e5-42e4-ac36-ac46f6d55524\") " pod="kube-system/global-pull-secret-syncer-4jdp2" May 28 15:07:42.575450 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:42.575326 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 28 15:07:42.576042 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:42.575467 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b64275b4-68e5-42e4-ac36-ac46f6d55524-original-pull-secret podName:b64275b4-68e5-42e4-ac36-ac46f6d55524 nodeName:}" failed. No retries permitted until 2026-05-28 15:07:44.575447094 +0000 UTC m=+15.464434156 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b64275b4-68e5-42e4-ac36-ac46f6d55524-original-pull-secret") pod "global-pull-secret-syncer-4jdp2" (UID: "b64275b4-68e5-42e4-ac36-ac46f6d55524") : object "kube-system"/"original-pull-secret" not registered May 28 15:07:42.709045 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:42.709010 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4jdp2" May 28 15:07:42.709201 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:42.709155 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4jdp2" podUID="b64275b4-68e5-42e4-ac36-ac46f6d55524" May 28 15:07:42.767146 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:42.767105 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bjm2p" event={"ID":"c83b6a1e-4582-4530-8993-9b3ab169873d","Type":"ContainerStarted","Data":"b881c1975a00bb91008d489d40569bb46d7c5bf38e5484dacc27c5070cec867c"} May 28 15:07:42.768686 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:42.768647 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" event={"ID":"8c849d95-73cd-461d-bd6b-845cb70cbd23","Type":"ContainerStarted","Data":"21a636ad63cfe87e86857d4caf8fe129bfc40abe2f6394c472337e7e475b820f"} May 28 15:07:42.770084 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:42.770055 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rgnx4" event={"ID":"ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc","Type":"ContainerStarted","Data":"43dfc91f691d18d751f243d3f70acd8ad1ea298a85029066573864c7b02ad1fa"} May 28 15:07:42.771648 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:42.771611 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bpxxm" event={"ID":"a2246888-2a0d-48b8-8fe6-12e283e47eb0","Type":"ContainerStarted","Data":"d9414a386647bbd631caff97e1b97660ef16966e0ef001c6e352c6e5efe138d1"} May 28 15:07:42.773117 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:42.773059 2559 generic.go:358] "Generic (PLEG): container finished" podID="615d5292-c021-4a20-be77-56f9230db424" containerID="1fc6347aec95018dbcf63d3daab8446a3cc0ea7a5810095b8d9b3c03631d7988" exitCode=0 May 28 15:07:42.773409 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:42.773387 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jc4zf" event={"ID":"615d5292-c021-4a20-be77-56f9230db424","Type":"ContainerDied","Data":"1fc6347aec95018dbcf63d3daab8446a3cc0ea7a5810095b8d9b3c03631d7988"} May 28 15:07:42.775076 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:42.775044 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kr9wd" event={"ID":"297e38b8-9a81-4112-811d-a97cfa5341ff","Type":"ContainerStarted","Data":"8e0f0090b81ac2edb2ee1ac0d4a52b7b055e031c163b77a4e54a09eb54256a92"} May 28 15:07:42.780254 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:42.780198 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-bjm2p" podStartSLOduration=4.438929359 podStartE2EDuration="13.780186695s" podCreationTimestamp="2026-05-28 15:07:29 +0000 UTC" firstStartedPulling="2026-05-28 15:07:32.324019551 +0000 UTC m=+3.213006613" lastFinishedPulling="2026-05-28 15:07:41.665276884 +0000 UTC m=+12.554263949" observedRunningTime="2026-05-28 15:07:42.780172912 +0000 UTC m=+13.669159996" watchObservedRunningTime="2026-05-28 15:07:42.780186695 +0000 UTC m=+13.669173779" May 28 15:07:42.795498 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:42.795450 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-bpxxm" podStartSLOduration=4.4712425 podStartE2EDuration="13.795438223s" podCreationTimestamp="2026-05-28 15:07:29 +0000 UTC" firstStartedPulling="2026-05-28 15:07:32.332079663 +0000 UTC m=+3.221066734" lastFinishedPulling="2026-05-28 15:07:41.65627538 +0000 UTC m=+12.545262457" observedRunningTime="2026-05-28 15:07:42.795076254 +0000 UTC m=+13.684063339" watchObservedRunningTime="2026-05-28 15:07:42.795438223 +0000 UTC m=+13.684425306" May 28 15:07:42.824291 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:42.824239 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-cm7p8" podStartSLOduration=4.484752709 podStartE2EDuration="13.824223791s" podCreationTimestamp="2026-05-28 15:07:29 +0000 UTC" firstStartedPulling="2026-05-28 15:07:32.334130759 +0000 UTC m=+3.223117830" lastFinishedPulling="2026-05-28 15:07:41.673601835 +0000 UTC m=+12.562588912" observedRunningTime="2026-05-28 15:07:42.824186024 +0000 UTC m=+13.713173127" watchObservedRunningTime="2026-05-28 15:07:42.824223791 +0000 UTC m=+13.713210876" May 28 15:07:42.835947 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:42.835857 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-kr9wd" podStartSLOduration=3.501638631 podStartE2EDuration="12.835840791s" podCreationTimestamp="2026-05-28 15:07:30 +0000 UTC" firstStartedPulling="2026-05-28 15:07:32.322508385 +0000 UTC m=+3.211495448" lastFinishedPulling="2026-05-28 15:07:41.656710534 +0000 UTC m=+12.545697608" observedRunningTime="2026-05-28 15:07:42.835272137 +0000 UTC m=+13.724259221" watchObservedRunningTime="2026-05-28 15:07:42.835840791 +0000 UTC m=+13.724827876" May 28 15:07:43.394831 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:43.394802 2559 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" May 28 15:07:43.637624 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:43.637410 2559 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-28T15:07:43.394821627Z","UUID":"eb50faa7-3e01-4273-bd8b-d6bdca394cc8","Handler":null,"Name":"","Endpoint":""} May 28 15:07:43.640205 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:43.640186 2559 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 May 28 15:07:43.640322 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:43.640212 2559 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock May 28 15:07:43.708389 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:43.708354 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9p7b" May 28 15:07:43.708595 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:43.708508 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9p7b" podUID="e3a78917-de69-42d0-8fbe-85f346276bf9" May 28 15:07:43.708776 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:43.708753 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rpkl2" May 28 15:07:43.708901 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:43.708878 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rpkl2" podUID="8306acd6-5d60-42dd-a8e4-d21a859b6525" May 28 15:07:43.779253 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:43.779216 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rgnx4" event={"ID":"ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc","Type":"ContainerStarted","Data":"a3e2098d4a5563869232b22a919b4eff6219fbc9d12efd29855f1ed683208a78"} May 28 15:07:43.780860 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:43.780821 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-l6f5d" event={"ID":"44b3ce72-966f-42e8-8b3a-b9615228955c","Type":"ContainerStarted","Data":"41599378488d3b00d384c117d5ab97561196e90a9ee444f20c03e75e424e2bf8"} May 28 15:07:43.793609 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:43.793570 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-l6f5d" podStartSLOduration=4.455964499 podStartE2EDuration="13.79355928s" podCreationTimestamp="2026-05-28 15:07:30 +0000 UTC" firstStartedPulling="2026-05-28 15:07:32.332083626 +0000 UTC m=+3.221070704" lastFinishedPulling="2026-05-28 15:07:41.66967842 +0000 UTC m=+12.558665485" observedRunningTime="2026-05-28 15:07:43.793413631 +0000 UTC m=+14.682400722" watchObservedRunningTime="2026-05-28 15:07:43.79355928 +0000 UTC m=+14.682546363" May 28 15:07:44.590304 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:44.590255 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b64275b4-68e5-42e4-ac36-ac46f6d55524-original-pull-secret\") pod \"global-pull-secret-syncer-4jdp2\" (UID: \"b64275b4-68e5-42e4-ac36-ac46f6d55524\") " pod="kube-system/global-pull-secret-syncer-4jdp2" May 28 15:07:44.590469 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:44.590439 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 28 15:07:44.590560 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:44.590527 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b64275b4-68e5-42e4-ac36-ac46f6d55524-original-pull-secret podName:b64275b4-68e5-42e4-ac36-ac46f6d55524 nodeName:}" failed. No retries permitted until 2026-05-28 15:07:48.590505369 +0000 UTC m=+19.479492436 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b64275b4-68e5-42e4-ac36-ac46f6d55524-original-pull-secret") pod "global-pull-secret-syncer-4jdp2" (UID: "b64275b4-68e5-42e4-ac36-ac46f6d55524") : object "kube-system"/"original-pull-secret" not registered May 28 15:07:44.709062 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:44.708658 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4jdp2" May 28 15:07:44.709062 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:44.708831 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4jdp2" podUID="b64275b4-68e5-42e4-ac36-ac46f6d55524" May 28 15:07:45.708621 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:45.708127 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9p7b" May 28 15:07:45.708621 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:45.708134 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rpkl2" May 28 15:07:45.708621 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:45.708277 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9p7b" podUID="e3a78917-de69-42d0-8fbe-85f346276bf9" May 28 15:07:45.708621 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:45.708310 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rpkl2" podUID="8306acd6-5d60-42dd-a8e4-d21a859b6525" May 28 15:07:46.274798 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:46.274761 2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-bjm2p" May 28 15:07:46.275564 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:46.275545 2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-bjm2p" May 28 15:07:46.708111 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:46.708033 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4jdp2" May 28 15:07:46.708361 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:46.708167 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4jdp2" podUID="b64275b4-68e5-42e4-ac36-ac46f6d55524" May 28 15:07:47.412441 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:47.412407 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3a78917-de69-42d0-8fbe-85f346276bf9-metrics-certs\") pod \"network-metrics-daemon-l9p7b\" (UID: \"e3a78917-de69-42d0-8fbe-85f346276bf9\") " pod="openshift-multus/network-metrics-daemon-l9p7b" May 28 15:07:47.413013 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:47.412560 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 28 15:07:47.413013 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:47.412619 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3a78917-de69-42d0-8fbe-85f346276bf9-metrics-certs podName:e3a78917-de69-42d0-8fbe-85f346276bf9 nodeName:}" failed. No retries permitted until 2026-05-28 15:08:03.412602465 +0000 UTC m=+34.301589551 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3a78917-de69-42d0-8fbe-85f346276bf9-metrics-certs") pod "network-metrics-daemon-l9p7b" (UID: "e3a78917-de69-42d0-8fbe-85f346276bf9") : object "openshift-multus"/"metrics-daemon-secret" not registered May 28 15:07:47.513319 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:47.513282 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v68kz\" (UniqueName: \"kubernetes.io/projected/8306acd6-5d60-42dd-a8e4-d21a859b6525-kube-api-access-v68kz\") pod \"network-check-target-rpkl2\" (UID: \"8306acd6-5d60-42dd-a8e4-d21a859b6525\") " pod="openshift-network-diagnostics/network-check-target-rpkl2" May 28 15:07:47.513513 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:47.513446 2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 28 15:07:47.513513 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:47.513466 2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 28 15:07:47.513513 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:47.513493 2559 projected.go:194] Error preparing data for projected volume kube-api-access-v68kz for pod openshift-network-diagnostics/network-check-target-rpkl2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 15:07:47.513659 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:47.513552 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8306acd6-5d60-42dd-a8e4-d21a859b6525-kube-api-access-v68kz podName:8306acd6-5d60-42dd-a8e4-d21a859b6525 nodeName:}" failed. No retries permitted until 2026-05-28 15:08:03.513535615 +0000 UTC m=+34.402522681 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-v68kz" (UniqueName: "kubernetes.io/projected/8306acd6-5d60-42dd-a8e4-d21a859b6525-kube-api-access-v68kz") pod "network-check-target-rpkl2" (UID: "8306acd6-5d60-42dd-a8e4-d21a859b6525") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 15:07:47.708897 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:47.708818 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9p7b" May 28 15:07:47.708897 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:47.708818 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rpkl2" May 28 15:07:47.709127 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:47.708971 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9p7b" podUID="e3a78917-de69-42d0-8fbe-85f346276bf9" May 28 15:07:47.709127 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:47.709083 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rpkl2" podUID="8306acd6-5d60-42dd-a8e4-d21a859b6525" May 28 15:07:48.623308 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:48.623272 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b64275b4-68e5-42e4-ac36-ac46f6d55524-original-pull-secret\") pod \"global-pull-secret-syncer-4jdp2\" (UID: \"b64275b4-68e5-42e4-ac36-ac46f6d55524\") " pod="kube-system/global-pull-secret-syncer-4jdp2" May 28 15:07:48.623765 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:48.623427 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 28 15:07:48.623765 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:48.623515 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b64275b4-68e5-42e4-ac36-ac46f6d55524-original-pull-secret podName:b64275b4-68e5-42e4-ac36-ac46f6d55524 nodeName:}" failed. No retries permitted until 2026-05-28 15:07:56.623495202 +0000 UTC m=+27.512482279 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b64275b4-68e5-42e4-ac36-ac46f6d55524-original-pull-secret") pod "global-pull-secret-syncer-4jdp2" (UID: "b64275b4-68e5-42e4-ac36-ac46f6d55524") : object "kube-system"/"original-pull-secret" not registered May 28 15:07:48.708347 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:48.708322 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4jdp2" May 28 15:07:48.708529 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:48.708452 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4jdp2" podUID="b64275b4-68e5-42e4-ac36-ac46f6d55524" May 28 15:07:49.709580 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:49.709380 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9p7b" May 28 15:07:49.710008 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:49.709454 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rpkl2" May 28 15:07:49.710008 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:49.709662 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9p7b" podUID="e3a78917-de69-42d0-8fbe-85f346276bf9" May 28 15:07:49.710008 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:49.709745 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rpkl2" podUID="8306acd6-5d60-42dd-a8e4-d21a859b6525" May 28 15:07:50.708689 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:50.708655 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4jdp2" May 28 15:07:50.708995 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:50.708774 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4jdp2" podUID="b64275b4-68e5-42e4-ac36-ac46f6d55524" May 28 15:07:51.134209 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:51.134125 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-bjm2p" May 28 15:07:51.134662 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:51.134272 2559 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" May 28 15:07:51.135005 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:51.134983 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-bjm2p" May 28 15:07:51.708760 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:51.708725 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rpkl2" May 28 15:07:51.708934 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:51.708725 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9p7b" May 28 15:07:51.708934 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:51.708847 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rpkl2" podUID="8306acd6-5d60-42dd-a8e4-d21a859b6525" May 28 15:07:51.709045 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:51.708968 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9p7b" podUID="e3a78917-de69-42d0-8fbe-85f346276bf9" May 28 15:07:52.708856 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:52.708819 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4jdp2" May 28 15:07:52.709309 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:52.708933 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4jdp2" podUID="b64275b4-68e5-42e4-ac36-ac46f6d55524" May 28 15:07:53.708833 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:53.708811 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rpkl2" May 28 15:07:53.708930 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:53.708820 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9p7b" May 28 15:07:53.708930 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:53.708910 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rpkl2" podUID="8306acd6-5d60-42dd-a8e4-d21a859b6525" May 28 15:07:53.709258 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:53.709037 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9p7b" podUID="e3a78917-de69-42d0-8fbe-85f346276bf9" May 28 15:07:53.800003 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:53.799966 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rgnx4" event={"ID":"ce0aab3a-a9eb-439d-b8ca-83f238a0e1dc","Type":"ContainerStarted","Data":"0188e068c7c17b0f7849e9e2fadd08424267e0b0d9107ab0a380a1898ea8d285"} May 28 15:07:53.801783 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:53.801754 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-l7mvw" event={"ID":"10ca4edd-091a-4f34-912a-0a579c845f52","Type":"ContainerStarted","Data":"1504f7fca70d4e95a0262b810d364589583a34280d647a14685e6c2cb4e88217"} May 28 15:07:53.804799 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:53.803740 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jc4zf" event={"ID":"615d5292-c021-4a20-be77-56f9230db424","Type":"ContainerStarted","Data":"1ba5408d0dbc65e3f0b6de3efb61c6046eb1b3f54efef3c22880a8f9eb915776"} May 28 15:07:53.805414 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:53.805387 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" event={"ID":"0f511937-01b1-4615-bc4c-12832012e6ce","Type":"ContainerStarted","Data":"56ee4a3ca2a6e950c8914865efd9e2f5898de4e3e4cbbb87973f3648a6172a21"} May 28 15:07:53.805534 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:53.805423 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" event={"ID":"0f511937-01b1-4615-bc4c-12832012e6ce","Type":"ContainerStarted","Data":"f54fc397bbf66eea150bbaf7974498d2719d427e458bb907be5095b68a195bab"} May 28 15:07:53.820624 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:53.818243 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rgnx4" podStartSLOduration=3.605640737 podStartE2EDuration="24.818226071s" podCreationTimestamp="2026-05-28 15:07:29 +0000 UTC" firstStartedPulling="2026-05-28 15:07:32.332430077 +0000 UTC m=+3.221417154" lastFinishedPulling="2026-05-28 15:07:53.545015426 +0000 UTC m=+24.434002488" observedRunningTime="2026-05-28 15:07:53.816938505 +0000 UTC m=+24.705925586" watchObservedRunningTime="2026-05-28 15:07:53.818226071 +0000 UTC m=+24.707213156" May 28 15:07:53.834675 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:53.834634 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-l7mvw" podStartSLOduration=3.6359367110000003 podStartE2EDuration="24.834621372s" podCreationTimestamp="2026-05-28 15:07:29 +0000 UTC" firstStartedPulling="2026-05-28 15:07:32.328084065 +0000 UTC m=+3.217071141" lastFinishedPulling="2026-05-28 15:07:53.526768737 +0000 UTC m=+24.415755802" observedRunningTime="2026-05-28 15:07:53.834514492 +0000 UTC m=+24.723501576" watchObservedRunningTime="2026-05-28 15:07:53.834621372 +0000 UTC m=+24.723608456" May 28 15:07:54.708353 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:54.708325 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4jdp2" May 28 15:07:54.708528 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:54.708457 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4jdp2" podUID="b64275b4-68e5-42e4-ac36-ac46f6d55524" May 28 15:07:54.808250 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:54.808220 2559 generic.go:358] "Generic (PLEG): container finished" podID="615d5292-c021-4a20-be77-56f9230db424" containerID="1ba5408d0dbc65e3f0b6de3efb61c6046eb1b3f54efef3c22880a8f9eb915776" exitCode=0 May 28 15:07:54.808762 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:54.808309 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jc4zf" event={"ID":"615d5292-c021-4a20-be77-56f9230db424","Type":"ContainerDied","Data":"1ba5408d0dbc65e3f0b6de3efb61c6046eb1b3f54efef3c22880a8f9eb915776"} May 28 15:07:54.810906 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:54.810884 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" event={"ID":"0f511937-01b1-4615-bc4c-12832012e6ce","Type":"ContainerStarted","Data":"822c794fdfb42a2a2f74c9eaf09e8b4906138e90ec9be5ccd7450be55937ec45"} May 28 15:07:54.811009 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:54.810912 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" event={"ID":"0f511937-01b1-4615-bc4c-12832012e6ce","Type":"ContainerStarted","Data":"4aad761a454de666d3e57051af9e2937063dfbed67ddf3a45e7a059b252d0e23"} May 28 15:07:54.811009 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:54.810922 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" event={"ID":"0f511937-01b1-4615-bc4c-12832012e6ce","Type":"ContainerStarted","Data":"cfd850c4feffdeb82bee55e15bda1cae04ba3b86eb9a6673d0840d01a56d613c"} May 28 15:07:54.811009 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:54.810930 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" event={"ID":"0f511937-01b1-4615-bc4c-12832012e6ce","Type":"ContainerStarted","Data":"90ba5a3996ab843110e99f02d92b4a7dc833a128d337a59179924e511eeec123"} May 28 15:07:55.708662 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:55.708633 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9p7b" May 28 15:07:55.708936 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:55.708665 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rpkl2" May 28 15:07:55.708936 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:55.708767 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9p7b" podUID="e3a78917-de69-42d0-8fbe-85f346276bf9" May 28 15:07:55.708936 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:55.708891 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rpkl2" podUID="8306acd6-5d60-42dd-a8e4-d21a859b6525" May 28 15:07:56.688147 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:56.688069 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b64275b4-68e5-42e4-ac36-ac46f6d55524-original-pull-secret\") pod \"global-pull-secret-syncer-4jdp2\" (UID: \"b64275b4-68e5-42e4-ac36-ac46f6d55524\") " pod="kube-system/global-pull-secret-syncer-4jdp2" May 28 15:07:56.688682 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:56.688201 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 28 15:07:56.688682 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:56.688256 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b64275b4-68e5-42e4-ac36-ac46f6d55524-original-pull-secret podName:b64275b4-68e5-42e4-ac36-ac46f6d55524 nodeName:}" failed. No retries permitted until 2026-05-28 15:08:12.688242072 +0000 UTC m=+43.577229134 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b64275b4-68e5-42e4-ac36-ac46f6d55524-original-pull-secret") pod "global-pull-secret-syncer-4jdp2" (UID: "b64275b4-68e5-42e4-ac36-ac46f6d55524") : object "kube-system"/"original-pull-secret" not registered May 28 15:07:56.708340 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:56.708321 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4jdp2" May 28 15:07:56.708424 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:56.708406 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4jdp2" podUID="b64275b4-68e5-42e4-ac36-ac46f6d55524" May 28 15:07:56.816069 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:56.816034 2559 generic.go:358] "Generic (PLEG): container finished" podID="615d5292-c021-4a20-be77-56f9230db424" containerID="2bc44aabe676d2481d1c3b7bd05ffc8c3be85c86e3be1a00ec6e2e0070f0a7ac" exitCode=0 May 28 15:07:56.816228 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:56.816091 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jc4zf" event={"ID":"615d5292-c021-4a20-be77-56f9230db424","Type":"ContainerDied","Data":"2bc44aabe676d2481d1c3b7bd05ffc8c3be85c86e3be1a00ec6e2e0070f0a7ac"} May 28 15:07:56.818925 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:56.818907 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" event={"ID":"0f511937-01b1-4615-bc4c-12832012e6ce","Type":"ContainerStarted","Data":"29a55d3453a752db46950cc4478999d70d2d2256261106009885818e935493e6"} May 28 15:07:57.708111 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:57.708044 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9p7b" May 28 15:07:57.708439 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:57.708045 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rpkl2" May 28 15:07:57.708439 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:57.708138 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9p7b" podUID="e3a78917-de69-42d0-8fbe-85f346276bf9" May 28 15:07:57.708439 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:57.708204 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rpkl2" podUID="8306acd6-5d60-42dd-a8e4-d21a859b6525" May 28 15:07:57.821981 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:57.821955 2559 generic.go:358] "Generic (PLEG): container finished" podID="615d5292-c021-4a20-be77-56f9230db424" containerID="86cb7f65f45dc2df1c7092de511e5b32b67f813ed15825bae5300281857f0f83" exitCode=0 May 28 15:07:57.822094 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:57.821984 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jc4zf" event={"ID":"615d5292-c021-4a20-be77-56f9230db424","Type":"ContainerDied","Data":"86cb7f65f45dc2df1c7092de511e5b32b67f813ed15825bae5300281857f0f83"} May 28 15:07:58.708437 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:58.708201 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4jdp2" May 28 15:07:58.708871 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:58.708442 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4jdp2" podUID="b64275b4-68e5-42e4-ac36-ac46f6d55524" May 28 15:07:58.828018 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:58.827979 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" event={"ID":"0f511937-01b1-4615-bc4c-12832012e6ce","Type":"ContainerStarted","Data":"583ba729222d28495673b473cfacd2a8342b517842303fd98d75655d10c8862d"} May 28 15:07:58.828374 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:58.828352 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:58.845368 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:58.845339 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:58.854540 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:58.854478 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" podStartSLOduration=8.651534084 podStartE2EDuration="29.854466895s" podCreationTimestamp="2026-05-28 15:07:29 +0000 UTC" firstStartedPulling="2026-05-28 15:07:32.323788364 +0000 UTC m=+3.212775429" lastFinishedPulling="2026-05-28 15:07:53.526721174 +0000 UTC m=+24.415708240" observedRunningTime="2026-05-28 15:07:58.853936688 +0000 UTC m=+29.742923772" watchObservedRunningTime="2026-05-28 15:07:58.854466895 +0000 UTC m=+29.743453979" May 28 15:07:59.710038 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:59.710001 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9p7b" May 28 15:07:59.710474 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:59.710090 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rpkl2" May 28 15:07:59.710474 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:59.710121 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9p7b" podUID="e3a78917-de69-42d0-8fbe-85f346276bf9" May 28 15:07:59.710474 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:07:59.710148 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rpkl2" podUID="8306acd6-5d60-42dd-a8e4-d21a859b6525" May 28 15:07:59.831331 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:59.831289 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:59.831516 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:59.831349 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:07:59.848680 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:07:59.848656 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:08:00.404375 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:00.404342 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4jdp2"] May 28 15:08:00.404550 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:00.404474 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4jdp2" May 28 15:08:00.404623 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:00.404600 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4jdp2" podUID="b64275b4-68e5-42e4-ac36-ac46f6d55524" May 28 15:08:00.407030 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:00.406953 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-l9p7b"] May 28 15:08:00.407171 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:00.407080 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9p7b" May 28 15:08:00.407230 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:00.407192 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9p7b" podUID="e3a78917-de69-42d0-8fbe-85f346276bf9" May 28 15:08:00.407798 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:00.407748 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rpkl2"] May 28 15:08:00.407910 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:00.407872 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rpkl2" May 28 15:08:00.407982 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:00.407954 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rpkl2" podUID="8306acd6-5d60-42dd-a8e4-d21a859b6525" May 28 15:08:01.708528 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:01.708495 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rpkl2" May 28 15:08:01.708968 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:01.708623 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rpkl2" podUID="8306acd6-5d60-42dd-a8e4-d21a859b6525" May 28 15:08:02.708387 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:02.708354 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9p7b" May 28 15:08:02.708555 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:02.708354 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4jdp2" May 28 15:08:02.708555 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:02.708499 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9p7b" podUID="e3a78917-de69-42d0-8fbe-85f346276bf9" May 28 15:08:02.708924 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:02.708568 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4jdp2" podUID="b64275b4-68e5-42e4-ac36-ac46f6d55524" May 28 15:08:03.440474 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:03.440444 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3a78917-de69-42d0-8fbe-85f346276bf9-metrics-certs\") pod \"network-metrics-daemon-l9p7b\" (UID: \"e3a78917-de69-42d0-8fbe-85f346276bf9\") " pod="openshift-multus/network-metrics-daemon-l9p7b" May 28 15:08:03.440666 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:03.440602 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 28 15:08:03.440717 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:03.440667 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3a78917-de69-42d0-8fbe-85f346276bf9-metrics-certs podName:e3a78917-de69-42d0-8fbe-85f346276bf9 nodeName:}" failed. No retries permitted until 2026-05-28 15:08:35.440650726 +0000 UTC m=+66.329637788 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3a78917-de69-42d0-8fbe-85f346276bf9-metrics-certs") pod "network-metrics-daemon-l9p7b" (UID: "e3a78917-de69-42d0-8fbe-85f346276bf9") : object "openshift-multus"/"metrics-daemon-secret" not registered May 28 15:08:03.540950 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:03.540915 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v68kz\" (UniqueName: \"kubernetes.io/projected/8306acd6-5d60-42dd-a8e4-d21a859b6525-kube-api-access-v68kz\") pod \"network-check-target-rpkl2\" (UID: \"8306acd6-5d60-42dd-a8e4-d21a859b6525\") " pod="openshift-network-diagnostics/network-check-target-rpkl2" May 28 15:08:03.541111 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:03.541054 2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 28 15:08:03.541111 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:03.541072 2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 28 15:08:03.541111 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:03.541093 2559 projected.go:194] Error preparing data for projected volume kube-api-access-v68kz for pod openshift-network-diagnostics/network-check-target-rpkl2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 15:08:03.541215 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:03.541149 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8306acd6-5d60-42dd-a8e4-d21a859b6525-kube-api-access-v68kz podName:8306acd6-5d60-42dd-a8e4-d21a859b6525 nodeName:}" failed. No retries permitted until 2026-05-28 15:08:35.54113265 +0000 UTC m=+66.430119714 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-v68kz" (UniqueName: "kubernetes.io/projected/8306acd6-5d60-42dd-a8e4-d21a859b6525-kube-api-access-v68kz") pod "network-check-target-rpkl2" (UID: "8306acd6-5d60-42dd-a8e4-d21a859b6525") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 15:08:03.708182 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:03.708097 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rpkl2" May 28 15:08:03.708322 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:03.708204 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rpkl2" podUID="8306acd6-5d60-42dd-a8e4-d21a859b6525" May 28 15:08:04.708100 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:04.708073 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9p7b" May 28 15:08:04.708787 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:04.708140 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4jdp2" May 28 15:08:04.708787 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:04.708263 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4jdp2" podUID="b64275b4-68e5-42e4-ac36-ac46f6d55524" May 28 15:08:04.708787 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:04.708379 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9p7b" podUID="e3a78917-de69-42d0-8fbe-85f346276bf9" May 28 15:08:05.708594 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:05.708403 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rpkl2" May 28 15:08:05.709067 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:05.708658 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rpkl2" podUID="8306acd6-5d60-42dd-a8e4-d21a859b6525" May 28 15:08:05.844718 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:05.844684 2559 generic.go:358] "Generic (PLEG): container finished" podID="615d5292-c021-4a20-be77-56f9230db424" containerID="a0044dcb8184dc0104f80d96184c8f70ac389a8022485c4efe636653919dda1b" exitCode=0 May 28 15:08:05.844885 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:05.844732 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jc4zf" event={"ID":"615d5292-c021-4a20-be77-56f9230db424","Type":"ContainerDied","Data":"a0044dcb8184dc0104f80d96184c8f70ac389a8022485c4efe636653919dda1b"} May 28 15:08:06.413871 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.413842 2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-141.ec2.internal" event="NodeReady" May 28 15:08:06.414112 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.413953 2559 kubelet_node_status.go:550] "Fast updating node status as it just became ready" May 28 15:08:06.446387 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.446356 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5bd878d5c4-fx69p"] May 28 15:08:06.472182 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.472151 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-9h2xl"] May 28 15:08:06.472342 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.472326 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" May 28 15:08:06.474991 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.474969 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" May 28 15:08:06.475135 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.474970 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" May 28 15:08:06.475286 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.475270 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-7pvbk\"" May 28 15:08:06.475359 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.475343 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" May 28 15:08:06.483211 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.483193 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" May 28 15:08:06.492745 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.492722 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5bd878d5c4-fx69p"] May 28 15:08:06.492745 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.492746 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9h2xl"] May 28 15:08:06.492889 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.492857 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9h2xl" May 28 15:08:06.495465 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.495448 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" May 28 15:08:06.496048 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.496027 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" May 28 15:08:06.496139 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.496076 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-n5mwm\"" May 28 15:08:06.496513 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.496497 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" May 28 15:08:06.559651 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.559627 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-nvshr"] May 28 15:08:06.565191 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.565166 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8959c2ac-69f0-4285-885b-1035cf57475e-trusted-ca\") pod \"image-registry-5bd878d5c4-fx69p\" (UID: \"8959c2ac-69f0-4285-885b-1035cf57475e\") " pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" May 28 15:08:06.565287 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.565196 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8959c2ac-69f0-4285-885b-1035cf57475e-image-registry-private-configuration\") pod \"image-registry-5bd878d5c4-fx69p\" (UID: \"8959c2ac-69f0-4285-885b-1035cf57475e\") " pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" May 28 15:08:06.565287 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.565252 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8959c2ac-69f0-4285-885b-1035cf57475e-registry-tls\") pod \"image-registry-5bd878d5c4-fx69p\" (UID: \"8959c2ac-69f0-4285-885b-1035cf57475e\") " pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" May 28 15:08:06.565287 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.565273 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8959c2ac-69f0-4285-885b-1035cf57475e-bound-sa-token\") pod \"image-registry-5bd878d5c4-fx69p\" (UID: \"8959c2ac-69f0-4285-885b-1035cf57475e\") " pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" May 28 15:08:06.565388 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.565288 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-924p2\" (UniqueName: \"kubernetes.io/projected/8959c2ac-69f0-4285-885b-1035cf57475e-kube-api-access-924p2\") pod \"image-registry-5bd878d5c4-fx69p\" (UID: \"8959c2ac-69f0-4285-885b-1035cf57475e\") " pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" May 28 15:08:06.565388 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.565335 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8959c2ac-69f0-4285-885b-1035cf57475e-ca-trust-extracted\") pod \"image-registry-5bd878d5c4-fx69p\" (UID: \"8959c2ac-69f0-4285-885b-1035cf57475e\") " pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" May 28 15:08:06.565388 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.565359 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8959c2ac-69f0-4285-885b-1035cf57475e-registry-certificates\") pod \"image-registry-5bd878d5c4-fx69p\" (UID: \"8959c2ac-69f0-4285-885b-1035cf57475e\") " pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" May 28 15:08:06.565502 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.565388 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8959c2ac-69f0-4285-885b-1035cf57475e-installation-pull-secrets\") pod \"image-registry-5bd878d5c4-fx69p\" (UID: \"8959c2ac-69f0-4285-885b-1035cf57475e\") " pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" May 28 15:08:06.577058 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.577035 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nvshr" May 28 15:08:06.578606 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.578585 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nvshr"] May 28 15:08:06.579893 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.579875 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" May 28 15:08:06.579978 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.579907 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-b5hqh\"" May 28 15:08:06.579978 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.579880 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" May 28 15:08:06.665865 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.665784 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8959c2ac-69f0-4285-885b-1035cf57475e-ca-trust-extracted\") pod \"image-registry-5bd878d5c4-fx69p\" (UID: \"8959c2ac-69f0-4285-885b-1035cf57475e\") " pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" May 28 15:08:06.665865 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.665821 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8959c2ac-69f0-4285-885b-1035cf57475e-registry-certificates\") pod \"image-registry-5bd878d5c4-fx69p\" (UID: \"8959c2ac-69f0-4285-885b-1035cf57475e\") " pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" May 28 15:08:06.665865 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.665847 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8959c2ac-69f0-4285-885b-1035cf57475e-installation-pull-secrets\") pod \"image-registry-5bd878d5c4-fx69p\" (UID: \"8959c2ac-69f0-4285-885b-1035cf57475e\") " pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" May 28 15:08:06.666105 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.665887 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6gj8\" (UniqueName: \"kubernetes.io/projected/8fa7b229-d678-41cd-b578-a31774414667-kube-api-access-b6gj8\") pod \"ingress-canary-9h2xl\" (UID: \"8fa7b229-d678-41cd-b578-a31774414667\") " pod="openshift-ingress-canary/ingress-canary-9h2xl" May 28 15:08:06.666105 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.665909 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8959c2ac-69f0-4285-885b-1035cf57475e-trusted-ca\") pod \"image-registry-5bd878d5c4-fx69p\" (UID: \"8959c2ac-69f0-4285-885b-1035cf57475e\") " pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" May 28 15:08:06.666105 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.665928 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8959c2ac-69f0-4285-885b-1035cf57475e-image-registry-private-configuration\") pod \"image-registry-5bd878d5c4-fx69p\" (UID: \"8959c2ac-69f0-4285-885b-1035cf57475e\") " pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" May 28 15:08:06.666105 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.666009 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8959c2ac-69f0-4285-885b-1035cf57475e-registry-tls\") pod \"image-registry-5bd878d5c4-fx69p\" (UID: \"8959c2ac-69f0-4285-885b-1035cf57475e\") " pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" May 28 15:08:06.666105 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.666047 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8959c2ac-69f0-4285-885b-1035cf57475e-bound-sa-token\") pod \"image-registry-5bd878d5c4-fx69p\" (UID: \"8959c2ac-69f0-4285-885b-1035cf57475e\") " pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" May 28 15:08:06.666344 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:06.666142 2559 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 28 15:08:06.666344 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:06.666161 2559 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bd878d5c4-fx69p: secret "image-registry-tls" not found May 28 15:08:06.666344 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:06.666255 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8959c2ac-69f0-4285-885b-1035cf57475e-registry-tls podName:8959c2ac-69f0-4285-885b-1035cf57475e nodeName:}" failed. No retries permitted until 2026-05-28 15:08:07.166230032 +0000 UTC m=+38.055217108 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8959c2ac-69f0-4285-885b-1035cf57475e-registry-tls") pod "image-registry-5bd878d5c4-fx69p" (UID: "8959c2ac-69f0-4285-885b-1035cf57475e") : secret "image-registry-tls" not found May 28 15:08:06.666344 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.666243 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-924p2\" (UniqueName: \"kubernetes.io/projected/8959c2ac-69f0-4285-885b-1035cf57475e-kube-api-access-924p2\") pod \"image-registry-5bd878d5c4-fx69p\" (UID: \"8959c2ac-69f0-4285-885b-1035cf57475e\") " pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" May 28 15:08:06.666344 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.666275 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8959c2ac-69f0-4285-885b-1035cf57475e-ca-trust-extracted\") pod \"image-registry-5bd878d5c4-fx69p\" (UID: \"8959c2ac-69f0-4285-885b-1035cf57475e\") " pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" May 28 15:08:06.666344 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.666302 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8fa7b229-d678-41cd-b578-a31774414667-cert\") pod \"ingress-canary-9h2xl\" (UID: \"8fa7b229-d678-41cd-b578-a31774414667\") " pod="openshift-ingress-canary/ingress-canary-9h2xl" May 28 15:08:06.666825 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.666806 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8959c2ac-69f0-4285-885b-1035cf57475e-registry-certificates\") pod \"image-registry-5bd878d5c4-fx69p\" (UID: \"8959c2ac-69f0-4285-885b-1035cf57475e\") " pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" May 28 15:08:06.667405 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.667381 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8959c2ac-69f0-4285-885b-1035cf57475e-trusted-ca\") pod \"image-registry-5bd878d5c4-fx69p\" (UID: \"8959c2ac-69f0-4285-885b-1035cf57475e\") " pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" May 28 15:08:06.670148 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.670127 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8959c2ac-69f0-4285-885b-1035cf57475e-image-registry-private-configuration\") pod \"image-registry-5bd878d5c4-fx69p\" (UID: \"8959c2ac-69f0-4285-885b-1035cf57475e\") " pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" May 28 15:08:06.670221 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.670148 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8959c2ac-69f0-4285-885b-1035cf57475e-installation-pull-secrets\") pod \"image-registry-5bd878d5c4-fx69p\" (UID: \"8959c2ac-69f0-4285-885b-1035cf57475e\") " pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" May 28 15:08:06.676898 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.676872 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-924p2\" (UniqueName: \"kubernetes.io/projected/8959c2ac-69f0-4285-885b-1035cf57475e-kube-api-access-924p2\") pod \"image-registry-5bd878d5c4-fx69p\" (UID: \"8959c2ac-69f0-4285-885b-1035cf57475e\") " pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" May 28 15:08:06.676967 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.676926 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8959c2ac-69f0-4285-885b-1035cf57475e-bound-sa-token\") pod \"image-registry-5bd878d5c4-fx69p\" (UID: \"8959c2ac-69f0-4285-885b-1035cf57475e\") " pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" May 28 15:08:06.708235 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.708215 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9p7b" May 28 15:08:06.708317 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.708215 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4jdp2" May 28 15:08:06.710724 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.710706 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8wjkp\"" May 28 15:08:06.711189 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.710738 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" May 28 15:08:06.711189 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.710824 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" May 28 15:08:06.767609 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.767576 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8fa7b229-d678-41cd-b578-a31774414667-cert\") pod \"ingress-canary-9h2xl\" (UID: \"8fa7b229-d678-41cd-b578-a31774414667\") " pod="openshift-ingress-canary/ingress-canary-9h2xl" May 28 15:08:06.767609 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.767612 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/59c5be81-b339-4a6d-9c00-f66c662406a3-config-volume\") pod \"dns-default-nvshr\" (UID: \"59c5be81-b339-4a6d-9c00-f66c662406a3\") " pod="openshift-dns/dns-default-nvshr" May 28 15:08:06.767841 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.767635 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/59c5be81-b339-4a6d-9c00-f66c662406a3-metrics-tls\") pod \"dns-default-nvshr\" (UID: \"59c5be81-b339-4a6d-9c00-f66c662406a3\") " pod="openshift-dns/dns-default-nvshr" May 28 15:08:06.767841 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.767651 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59c5be81-b339-4a6d-9c00-f66c662406a3-tmp-dir\") pod \"dns-default-nvshr\" (UID: \"59c5be81-b339-4a6d-9c00-f66c662406a3\") " pod="openshift-dns/dns-default-nvshr" May 28 15:08:06.767841 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:06.767728 2559 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 15:08:06.767841 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.767759 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b6gj8\" (UniqueName: \"kubernetes.io/projected/8fa7b229-d678-41cd-b578-a31774414667-kube-api-access-b6gj8\") pod \"ingress-canary-9h2xl\" (UID: \"8fa7b229-d678-41cd-b578-a31774414667\") " pod="openshift-ingress-canary/ingress-canary-9h2xl" May 28 15:08:06.767841 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:06.767791 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fa7b229-d678-41cd-b578-a31774414667-cert podName:8fa7b229-d678-41cd-b578-a31774414667 nodeName:}" failed. No retries permitted until 2026-05-28 15:08:07.267772003 +0000 UTC m=+38.156759072 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/8fa7b229-d678-41cd-b578-a31774414667-cert") pod "ingress-canary-9h2xl" (UID: "8fa7b229-d678-41cd-b578-a31774414667") : secret "canary-serving-cert" not found May 28 15:08:06.767841 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.767839 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2h7q\" (UniqueName: \"kubernetes.io/projected/59c5be81-b339-4a6d-9c00-f66c662406a3-kube-api-access-m2h7q\") pod \"dns-default-nvshr\" (UID: \"59c5be81-b339-4a6d-9c00-f66c662406a3\") " pod="openshift-dns/dns-default-nvshr" May 28 15:08:06.776084 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.776055 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6gj8\" (UniqueName: \"kubernetes.io/projected/8fa7b229-d678-41cd-b578-a31774414667-kube-api-access-b6gj8\") pod \"ingress-canary-9h2xl\" (UID: \"8fa7b229-d678-41cd-b578-a31774414667\") " pod="openshift-ingress-canary/ingress-canary-9h2xl" May 28 15:08:06.848966 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.848932 2559 generic.go:358] "Generic (PLEG): container finished" podID="615d5292-c021-4a20-be77-56f9230db424" containerID="e3e470e2975ac81b7e8851e8564650c351cff2902ad0c8810c60b646dbcc3a04" exitCode=0 May 28 15:08:06.849138 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.849002 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jc4zf" event={"ID":"615d5292-c021-4a20-be77-56f9230db424","Type":"ContainerDied","Data":"e3e470e2975ac81b7e8851e8564650c351cff2902ad0c8810c60b646dbcc3a04"} May 28 15:08:06.868587 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.868560 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m2h7q\" (UniqueName: \"kubernetes.io/projected/59c5be81-b339-4a6d-9c00-f66c662406a3-kube-api-access-m2h7q\") pod \"dns-default-nvshr\" (UID: \"59c5be81-b339-4a6d-9c00-f66c662406a3\") " pod="openshift-dns/dns-default-nvshr" May 28 15:08:06.868715 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.868697 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/59c5be81-b339-4a6d-9c00-f66c662406a3-config-volume\") pod \"dns-default-nvshr\" (UID: \"59c5be81-b339-4a6d-9c00-f66c662406a3\") " pod="openshift-dns/dns-default-nvshr" May 28 15:08:06.868774 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.868716 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/59c5be81-b339-4a6d-9c00-f66c662406a3-metrics-tls\") pod \"dns-default-nvshr\" (UID: \"59c5be81-b339-4a6d-9c00-f66c662406a3\") " pod="openshift-dns/dns-default-nvshr" May 28 15:08:06.868774 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.868736 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59c5be81-b339-4a6d-9c00-f66c662406a3-tmp-dir\") pod \"dns-default-nvshr\" (UID: \"59c5be81-b339-4a6d-9c00-f66c662406a3\") " pod="openshift-dns/dns-default-nvshr" May 28 15:08:06.868879 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:06.868843 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 15:08:06.868920 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:06.868909 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59c5be81-b339-4a6d-9c00-f66c662406a3-metrics-tls podName:59c5be81-b339-4a6d-9c00-f66c662406a3 nodeName:}" failed. No retries permitted until 2026-05-28 15:08:07.3688899 +0000 UTC m=+38.257876967 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/59c5be81-b339-4a6d-9c00-f66c662406a3-metrics-tls") pod "dns-default-nvshr" (UID: "59c5be81-b339-4a6d-9c00-f66c662406a3") : secret "dns-default-metrics-tls" not found May 28 15:08:06.869106 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.869087 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59c5be81-b339-4a6d-9c00-f66c662406a3-tmp-dir\") pod \"dns-default-nvshr\" (UID: \"59c5be81-b339-4a6d-9c00-f66c662406a3\") " pod="openshift-dns/dns-default-nvshr" May 28 15:08:06.869204 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.869188 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/59c5be81-b339-4a6d-9c00-f66c662406a3-config-volume\") pod \"dns-default-nvshr\" (UID: \"59c5be81-b339-4a6d-9c00-f66c662406a3\") " pod="openshift-dns/dns-default-nvshr" May 28 15:08:06.877103 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:06.877081 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2h7q\" (UniqueName: \"kubernetes.io/projected/59c5be81-b339-4a6d-9c00-f66c662406a3-kube-api-access-m2h7q\") pod \"dns-default-nvshr\" (UID: \"59c5be81-b339-4a6d-9c00-f66c662406a3\") " pod="openshift-dns/dns-default-nvshr" May 28 15:08:07.170920 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:07.170877 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8959c2ac-69f0-4285-885b-1035cf57475e-registry-tls\") pod \"image-registry-5bd878d5c4-fx69p\" (UID: \"8959c2ac-69f0-4285-885b-1035cf57475e\") " pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" May 28 15:08:07.171070 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:07.171029 2559 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 28 15:08:07.171070 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:07.171048 2559 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bd878d5c4-fx69p: secret "image-registry-tls" not found May 28 15:08:07.171174 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:07.171103 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8959c2ac-69f0-4285-885b-1035cf57475e-registry-tls podName:8959c2ac-69f0-4285-885b-1035cf57475e nodeName:}" failed. No retries permitted until 2026-05-28 15:08:08.171087044 +0000 UTC m=+39.060074106 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8959c2ac-69f0-4285-885b-1035cf57475e-registry-tls") pod "image-registry-5bd878d5c4-fx69p" (UID: "8959c2ac-69f0-4285-885b-1035cf57475e") : secret "image-registry-tls" not found May 28 15:08:07.272198 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:07.272173 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8fa7b229-d678-41cd-b578-a31774414667-cert\") pod \"ingress-canary-9h2xl\" (UID: \"8fa7b229-d678-41cd-b578-a31774414667\") " pod="openshift-ingress-canary/ingress-canary-9h2xl" May 28 15:08:07.272323 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:07.272312 2559 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 15:08:07.272376 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:07.272366 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fa7b229-d678-41cd-b578-a31774414667-cert podName:8fa7b229-d678-41cd-b578-a31774414667 nodeName:}" failed. No retries permitted until 2026-05-28 15:08:08.272351562 +0000 UTC m=+39.161338640 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/8fa7b229-d678-41cd-b578-a31774414667-cert") pod "ingress-canary-9h2xl" (UID: "8fa7b229-d678-41cd-b578-a31774414667") : secret "canary-serving-cert" not found May 28 15:08:07.373050 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:07.373021 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/59c5be81-b339-4a6d-9c00-f66c662406a3-metrics-tls\") pod \"dns-default-nvshr\" (UID: \"59c5be81-b339-4a6d-9c00-f66c662406a3\") " pod="openshift-dns/dns-default-nvshr" May 28 15:08:07.373183 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:07.373132 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 15:08:07.373183 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:07.373176 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59c5be81-b339-4a6d-9c00-f66c662406a3-metrics-tls podName:59c5be81-b339-4a6d-9c00-f66c662406a3 nodeName:}" failed. No retries permitted until 2026-05-28 15:08:08.373164592 +0000 UTC m=+39.262151654 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/59c5be81-b339-4a6d-9c00-f66c662406a3-metrics-tls") pod "dns-default-nvshr" (UID: "59c5be81-b339-4a6d-9c00-f66c662406a3") : secret "dns-default-metrics-tls" not found May 28 15:08:07.708593 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:07.708518 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rpkl2" May 28 15:08:07.711316 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:07.711289 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" May 28 15:08:07.711316 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:07.711291 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6xkj5\"" May 28 15:08:07.711711 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:07.711317 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" May 28 15:08:07.853187 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:07.853161 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jc4zf" event={"ID":"615d5292-c021-4a20-be77-56f9230db424","Type":"ContainerStarted","Data":"d027197fc6ba2b32617b9b083743bfb85b21139548a6b444a897ffb584cde653"} May 28 15:08:07.874590 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:07.874556 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-jc4zf" podStartSLOduration=6.243982299 podStartE2EDuration="38.874546879s" podCreationTimestamp="2026-05-28 15:07:29 +0000 UTC" firstStartedPulling="2026-05-28 15:07:32.326812364 +0000 UTC m=+3.215799428" lastFinishedPulling="2026-05-28 15:08:04.957376946 +0000 UTC m=+35.846364008" observedRunningTime="2026-05-28 15:08:07.873329638 +0000 UTC m=+38.762316722" watchObservedRunningTime="2026-05-28 15:08:07.874546879 +0000 UTC m=+38.763533960" May 28 15:08:08.179956 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:08.179829 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8959c2ac-69f0-4285-885b-1035cf57475e-registry-tls\") pod \"image-registry-5bd878d5c4-fx69p\" (UID: \"8959c2ac-69f0-4285-885b-1035cf57475e\") " pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" May 28 15:08:08.180063 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:08.179968 2559 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 28 15:08:08.180063 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:08.179987 2559 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bd878d5c4-fx69p: secret "image-registry-tls" not found May 28 15:08:08.180063 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:08.180038 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8959c2ac-69f0-4285-885b-1035cf57475e-registry-tls podName:8959c2ac-69f0-4285-885b-1035cf57475e nodeName:}" failed. No retries permitted until 2026-05-28 15:08:10.180024726 +0000 UTC m=+41.069011790 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8959c2ac-69f0-4285-885b-1035cf57475e-registry-tls") pod "image-registry-5bd878d5c4-fx69p" (UID: "8959c2ac-69f0-4285-885b-1035cf57475e") : secret "image-registry-tls" not found May 28 15:08:08.280777 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:08.280753 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8fa7b229-d678-41cd-b578-a31774414667-cert\") pod \"ingress-canary-9h2xl\" (UID: \"8fa7b229-d678-41cd-b578-a31774414667\") " pod="openshift-ingress-canary/ingress-canary-9h2xl" May 28 15:08:08.280898 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:08.280850 2559 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 15:08:08.280980 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:08.280962 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fa7b229-d678-41cd-b578-a31774414667-cert podName:8fa7b229-d678-41cd-b578-a31774414667 nodeName:}" failed. No retries permitted until 2026-05-28 15:08:10.280881569 +0000 UTC m=+41.169868631 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/8fa7b229-d678-41cd-b578-a31774414667-cert") pod "ingress-canary-9h2xl" (UID: "8fa7b229-d678-41cd-b578-a31774414667") : secret "canary-serving-cert" not found May 28 15:08:08.381778 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:08.381708 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/59c5be81-b339-4a6d-9c00-f66c662406a3-metrics-tls\") pod \"dns-default-nvshr\" (UID: \"59c5be81-b339-4a6d-9c00-f66c662406a3\") " pod="openshift-dns/dns-default-nvshr" May 28 15:08:08.381865 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:08.381814 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 15:08:08.381865 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:08.381855 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59c5be81-b339-4a6d-9c00-f66c662406a3-metrics-tls podName:59c5be81-b339-4a6d-9c00-f66c662406a3 nodeName:}" failed. No retries permitted until 2026-05-28 15:08:10.381844375 +0000 UTC m=+41.270831436 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/59c5be81-b339-4a6d-9c00-f66c662406a3-metrics-tls") pod "dns-default-nvshr" (UID: "59c5be81-b339-4a6d-9c00-f66c662406a3") : secret "dns-default-metrics-tls" not found May 28 15:08:10.192649 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:10.192612 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8959c2ac-69f0-4285-885b-1035cf57475e-registry-tls\") pod \"image-registry-5bd878d5c4-fx69p\" (UID: \"8959c2ac-69f0-4285-885b-1035cf57475e\") " pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" May 28 15:08:10.193096 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:10.192766 2559 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 28 15:08:10.193096 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:10.192789 2559 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bd878d5c4-fx69p: secret "image-registry-tls" not found May 28 15:08:10.193096 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:10.192848 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8959c2ac-69f0-4285-885b-1035cf57475e-registry-tls podName:8959c2ac-69f0-4285-885b-1035cf57475e nodeName:}" failed. No retries permitted until 2026-05-28 15:08:14.192831465 +0000 UTC m=+45.081818544 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8959c2ac-69f0-4285-885b-1035cf57475e-registry-tls") pod "image-registry-5bd878d5c4-fx69p" (UID: "8959c2ac-69f0-4285-885b-1035cf57475e") : secret "image-registry-tls" not found May 28 15:08:10.293608 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:10.293575 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8fa7b229-d678-41cd-b578-a31774414667-cert\") pod \"ingress-canary-9h2xl\" (UID: \"8fa7b229-d678-41cd-b578-a31774414667\") " pod="openshift-ingress-canary/ingress-canary-9h2xl" May 28 15:08:10.293744 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:10.293721 2559 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 15:08:10.293796 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:10.293787 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fa7b229-d678-41cd-b578-a31774414667-cert podName:8fa7b229-d678-41cd-b578-a31774414667 nodeName:}" failed. No retries permitted until 2026-05-28 15:08:14.293773177 +0000 UTC m=+45.182760239 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/8fa7b229-d678-41cd-b578-a31774414667-cert") pod "ingress-canary-9h2xl" (UID: "8fa7b229-d678-41cd-b578-a31774414667") : secret "canary-serving-cert" not found May 28 15:08:10.393971 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:10.393940 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/59c5be81-b339-4a6d-9c00-f66c662406a3-metrics-tls\") pod \"dns-default-nvshr\" (UID: \"59c5be81-b339-4a6d-9c00-f66c662406a3\") " pod="openshift-dns/dns-default-nvshr" May 28 15:08:10.394114 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:10.394095 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 15:08:10.394166 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:10.394157 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59c5be81-b339-4a6d-9c00-f66c662406a3-metrics-tls podName:59c5be81-b339-4a6d-9c00-f66c662406a3 nodeName:}" failed. No retries permitted until 2026-05-28 15:08:14.394143223 +0000 UTC m=+45.283130284 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/59c5be81-b339-4a6d-9c00-f66c662406a3-metrics-tls") pod "dns-default-nvshr" (UID: "59c5be81-b339-4a6d-9c00-f66c662406a3") : secret "dns-default-metrics-tls" not found May 28 15:08:12.713429 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:12.713398 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b64275b4-68e5-42e4-ac36-ac46f6d55524-original-pull-secret\") pod \"global-pull-secret-syncer-4jdp2\" (UID: \"b64275b4-68e5-42e4-ac36-ac46f6d55524\") " pod="kube-system/global-pull-secret-syncer-4jdp2" May 28 15:08:12.725918 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:12.725894 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b64275b4-68e5-42e4-ac36-ac46f6d55524-original-pull-secret\") pod \"global-pull-secret-syncer-4jdp2\" (UID: \"b64275b4-68e5-42e4-ac36-ac46f6d55524\") " pod="kube-system/global-pull-secret-syncer-4jdp2" May 28 15:08:13.022519 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:13.022470 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4jdp2" May 28 15:08:13.189763 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:13.189737 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4jdp2"] May 28 15:08:13.193242 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:08:13.193212 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb64275b4_68e5_42e4_ac36_ac46f6d55524.slice/crio-7ed4b2efe04da2d69fbf7e5dd2d928d1c37c89644eef154ada24548b0a14618c WatchSource:0}: Error finding container 7ed4b2efe04da2d69fbf7e5dd2d928d1c37c89644eef154ada24548b0a14618c: Status 404 returned error can't find the container with id 7ed4b2efe04da2d69fbf7e5dd2d928d1c37c89644eef154ada24548b0a14618c May 28 15:08:13.864537 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:13.864337 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4jdp2" event={"ID":"b64275b4-68e5-42e4-ac36-ac46f6d55524","Type":"ContainerStarted","Data":"7ed4b2efe04da2d69fbf7e5dd2d928d1c37c89644eef154ada24548b0a14618c"} May 28 15:08:14.222654 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:14.222579 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8959c2ac-69f0-4285-885b-1035cf57475e-registry-tls\") pod \"image-registry-5bd878d5c4-fx69p\" (UID: \"8959c2ac-69f0-4285-885b-1035cf57475e\") " pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" May 28 15:08:14.222789 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:14.222757 2559 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 28 15:08:14.222789 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:14.222778 2559 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bd878d5c4-fx69p: secret "image-registry-tls" not found May 28 15:08:14.222860 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:14.222845 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8959c2ac-69f0-4285-885b-1035cf57475e-registry-tls podName:8959c2ac-69f0-4285-885b-1035cf57475e nodeName:}" failed. No retries permitted until 2026-05-28 15:08:22.222825484 +0000 UTC m=+53.111812552 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8959c2ac-69f0-4285-885b-1035cf57475e-registry-tls") pod "image-registry-5bd878d5c4-fx69p" (UID: "8959c2ac-69f0-4285-885b-1035cf57475e") : secret "image-registry-tls" not found May 28 15:08:14.323102 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:14.323068 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8fa7b229-d678-41cd-b578-a31774414667-cert\") pod \"ingress-canary-9h2xl\" (UID: \"8fa7b229-d678-41cd-b578-a31774414667\") " pod="openshift-ingress-canary/ingress-canary-9h2xl" May 28 15:08:14.323267 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:14.323207 2559 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 15:08:14.323311 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:14.323271 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fa7b229-d678-41cd-b578-a31774414667-cert podName:8fa7b229-d678-41cd-b578-a31774414667 nodeName:}" failed. No retries permitted until 2026-05-28 15:08:22.323256381 +0000 UTC m=+53.212243450 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/8fa7b229-d678-41cd-b578-a31774414667-cert") pod "ingress-canary-9h2xl" (UID: "8fa7b229-d678-41cd-b578-a31774414667") : secret "canary-serving-cert" not found May 28 15:08:14.424034 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:14.424001 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/59c5be81-b339-4a6d-9c00-f66c662406a3-metrics-tls\") pod \"dns-default-nvshr\" (UID: \"59c5be81-b339-4a6d-9c00-f66c662406a3\") " pod="openshift-dns/dns-default-nvshr" May 28 15:08:14.424187 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:14.424168 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 15:08:14.424253 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:14.424242 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59c5be81-b339-4a6d-9c00-f66c662406a3-metrics-tls podName:59c5be81-b339-4a6d-9c00-f66c662406a3 nodeName:}" failed. No retries permitted until 2026-05-28 15:08:22.424221221 +0000 UTC m=+53.313208288 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/59c5be81-b339-4a6d-9c00-f66c662406a3-metrics-tls") pod "dns-default-nvshr" (UID: "59c5be81-b339-4a6d-9c00-f66c662406a3") : secret "dns-default-metrics-tls" not found May 28 15:08:17.873160 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:17.873122 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4jdp2" event={"ID":"b64275b4-68e5-42e4-ac36-ac46f6d55524","Type":"ContainerStarted","Data":"cae0c0416850a5b8325d3fdfb2e93d04f43cac0bcfdcd518b2fc554c570b0c5f"} May 28 15:08:17.888413 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:17.888355 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-4jdp2" podStartSLOduration=33.382796747 podStartE2EDuration="37.88834041s" podCreationTimestamp="2026-05-28 15:07:40 +0000 UTC" firstStartedPulling="2026-05-28 15:08:13.195169186 +0000 UTC m=+44.084156251" lastFinishedPulling="2026-05-28 15:08:17.70071285 +0000 UTC m=+48.589699914" observedRunningTime="2026-05-28 15:08:17.887803048 +0000 UTC m=+48.776790132" watchObservedRunningTime="2026-05-28 15:08:17.88834041 +0000 UTC m=+48.777327493" May 28 15:08:22.281184 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:22.281145 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8959c2ac-69f0-4285-885b-1035cf57475e-registry-tls\") pod \"image-registry-5bd878d5c4-fx69p\" (UID: \"8959c2ac-69f0-4285-885b-1035cf57475e\") " pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" May 28 15:08:22.281724 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:22.281252 2559 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 28 15:08:22.281724 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:22.281263 2559 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bd878d5c4-fx69p: secret "image-registry-tls" not found May 28 15:08:22.281724 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:22.281309 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8959c2ac-69f0-4285-885b-1035cf57475e-registry-tls podName:8959c2ac-69f0-4285-885b-1035cf57475e nodeName:}" failed. No retries permitted until 2026-05-28 15:08:38.281295582 +0000 UTC m=+69.170282644 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8959c2ac-69f0-4285-885b-1035cf57475e-registry-tls") pod "image-registry-5bd878d5c4-fx69p" (UID: "8959c2ac-69f0-4285-885b-1035cf57475e") : secret "image-registry-tls" not found May 28 15:08:22.381645 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:22.381610 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8fa7b229-d678-41cd-b578-a31774414667-cert\") pod \"ingress-canary-9h2xl\" (UID: \"8fa7b229-d678-41cd-b578-a31774414667\") " pod="openshift-ingress-canary/ingress-canary-9h2xl" May 28 15:08:22.381793 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:22.381744 2559 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 15:08:22.381837 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:22.381810 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fa7b229-d678-41cd-b578-a31774414667-cert podName:8fa7b229-d678-41cd-b578-a31774414667 nodeName:}" failed. No retries permitted until 2026-05-28 15:08:38.381795563 +0000 UTC m=+69.270782630 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/8fa7b229-d678-41cd-b578-a31774414667-cert") pod "ingress-canary-9h2xl" (UID: "8fa7b229-d678-41cd-b578-a31774414667") : secret "canary-serving-cert" not found May 28 15:08:22.482417 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:22.482385 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/59c5be81-b339-4a6d-9c00-f66c662406a3-metrics-tls\") pod \"dns-default-nvshr\" (UID: \"59c5be81-b339-4a6d-9c00-f66c662406a3\") " pod="openshift-dns/dns-default-nvshr" May 28 15:08:22.482575 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:22.482511 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 15:08:22.482575 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:22.482561 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59c5be81-b339-4a6d-9c00-f66c662406a3-metrics-tls podName:59c5be81-b339-4a6d-9c00-f66c662406a3 nodeName:}" failed. No retries permitted until 2026-05-28 15:08:38.482548325 +0000 UTC m=+69.371535387 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/59c5be81-b339-4a6d-9c00-f66c662406a3-metrics-tls") pod "dns-default-nvshr" (UID: "59c5be81-b339-4a6d-9c00-f66c662406a3") : secret "dns-default-metrics-tls" not found May 28 15:08:31.844701 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:31.844672 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xcg2z" May 28 15:08:35.478719 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:35.478682 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3a78917-de69-42d0-8fbe-85f346276bf9-metrics-certs\") pod \"network-metrics-daemon-l9p7b\" (UID: \"e3a78917-de69-42d0-8fbe-85f346276bf9\") " pod="openshift-multus/network-metrics-daemon-l9p7b" May 28 15:08:35.481950 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:35.481931 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" May 28 15:08:35.489402 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:35.489385 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found May 28 15:08:35.489447 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:35.489440 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3a78917-de69-42d0-8fbe-85f346276bf9-metrics-certs podName:e3a78917-de69-42d0-8fbe-85f346276bf9 nodeName:}" failed. No retries permitted until 2026-05-28 15:09:39.489424926 +0000 UTC m=+130.378411987 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3a78917-de69-42d0-8fbe-85f346276bf9-metrics-certs") pod "network-metrics-daemon-l9p7b" (UID: "e3a78917-de69-42d0-8fbe-85f346276bf9") : secret "metrics-daemon-secret" not found May 28 15:08:35.579465 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:35.579436 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v68kz\" (UniqueName: \"kubernetes.io/projected/8306acd6-5d60-42dd-a8e4-d21a859b6525-kube-api-access-v68kz\") pod \"network-check-target-rpkl2\" (UID: \"8306acd6-5d60-42dd-a8e4-d21a859b6525\") " pod="openshift-network-diagnostics/network-check-target-rpkl2" May 28 15:08:35.582426 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:35.582411 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" May 28 15:08:35.595178 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:35.595161 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" May 28 15:08:35.604548 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:35.604531 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v68kz\" (UniqueName: \"kubernetes.io/projected/8306acd6-5d60-42dd-a8e4-d21a859b6525-kube-api-access-v68kz\") pod \"network-check-target-rpkl2\" (UID: \"8306acd6-5d60-42dd-a8e4-d21a859b6525\") " pod="openshift-network-diagnostics/network-check-target-rpkl2" May 28 15:08:35.619696 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:35.619676 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6xkj5\"" May 28 15:08:35.627751 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:35.627739 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rpkl2" May 28 15:08:35.764001 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:35.763973 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rpkl2"] May 28 15:08:35.767119 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:08:35.767093 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8306acd6_5d60_42dd_a8e4_d21a859b6525.slice/crio-4e87f7fa0f5888b8b196785ee530279e42f96b7d5ea1625fb493dab6acc1c6f3 WatchSource:0}: Error finding container 4e87f7fa0f5888b8b196785ee530279e42f96b7d5ea1625fb493dab6acc1c6f3: Status 404 returned error can't find the container with id 4e87f7fa0f5888b8b196785ee530279e42f96b7d5ea1625fb493dab6acc1c6f3 May 28 15:08:35.908422 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:35.908391 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rpkl2" event={"ID":"8306acd6-5d60-42dd-a8e4-d21a859b6525","Type":"ContainerStarted","Data":"4e87f7fa0f5888b8b196785ee530279e42f96b7d5ea1625fb493dab6acc1c6f3"} May 28 15:08:38.308769 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:38.308731 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8959c2ac-69f0-4285-885b-1035cf57475e-registry-tls\") pod \"image-registry-5bd878d5c4-fx69p\" (UID: \"8959c2ac-69f0-4285-885b-1035cf57475e\") " pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" May 28 15:08:38.309160 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:38.308886 2559 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 28 15:08:38.309160 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:38.308907 2559 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bd878d5c4-fx69p: secret "image-registry-tls" not found May 28 15:08:38.309160 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:38.308966 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8959c2ac-69f0-4285-885b-1035cf57475e-registry-tls podName:8959c2ac-69f0-4285-885b-1035cf57475e nodeName:}" failed. No retries permitted until 2026-05-28 15:09:10.308952495 +0000 UTC m=+101.197939557 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8959c2ac-69f0-4285-885b-1035cf57475e-registry-tls") pod "image-registry-5bd878d5c4-fx69p" (UID: "8959c2ac-69f0-4285-885b-1035cf57475e") : secret "image-registry-tls" not found May 28 15:08:38.410022 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:38.409994 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8fa7b229-d678-41cd-b578-a31774414667-cert\") pod \"ingress-canary-9h2xl\" (UID: \"8fa7b229-d678-41cd-b578-a31774414667\") " pod="openshift-ingress-canary/ingress-canary-9h2xl" May 28 15:08:38.410192 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:38.410113 2559 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 15:08:38.410192 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:38.410165 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fa7b229-d678-41cd-b578-a31774414667-cert podName:8fa7b229-d678-41cd-b578-a31774414667 nodeName:}" failed. No retries permitted until 2026-05-28 15:09:10.4101521 +0000 UTC m=+101.299139161 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/8fa7b229-d678-41cd-b578-a31774414667-cert") pod "ingress-canary-9h2xl" (UID: "8fa7b229-d678-41cd-b578-a31774414667") : secret "canary-serving-cert" not found May 28 15:08:38.511299 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:38.511263 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/59c5be81-b339-4a6d-9c00-f66c662406a3-metrics-tls\") pod \"dns-default-nvshr\" (UID: \"59c5be81-b339-4a6d-9c00-f66c662406a3\") " pod="openshift-dns/dns-default-nvshr" May 28 15:08:38.511442 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:38.511415 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 15:08:38.511528 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:08:38.511515 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59c5be81-b339-4a6d-9c00-f66c662406a3-metrics-tls podName:59c5be81-b339-4a6d-9c00-f66c662406a3 nodeName:}" failed. No retries permitted until 2026-05-28 15:09:10.511477343 +0000 UTC m=+101.400464412 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/59c5be81-b339-4a6d-9c00-f66c662406a3-metrics-tls") pod "dns-default-nvshr" (UID: "59c5be81-b339-4a6d-9c00-f66c662406a3") : secret "dns-default-metrics-tls" not found May 28 15:08:39.916498 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:39.916458 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rpkl2" event={"ID":"8306acd6-5d60-42dd-a8e4-d21a859b6525","Type":"ContainerStarted","Data":"aa306cee3480b0191f9516f998e68e5f83e248d40635c034f8ddcb0d657ae9e3"} May 28 15:08:39.916850 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:39.916678 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-rpkl2" May 28 15:08:39.931771 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:08:39.931731 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-rpkl2" podStartSLOduration=65.993586859 podStartE2EDuration="1m9.931719982s" podCreationTimestamp="2026-05-28 15:07:30 +0000 UTC" firstStartedPulling="2026-05-28 15:08:35.768828506 +0000 UTC m=+66.657815568" lastFinishedPulling="2026-05-28 15:08:39.706961629 +0000 UTC m=+70.595948691" observedRunningTime="2026-05-28 15:08:39.931143535 +0000 UTC m=+70.820130620" watchObservedRunningTime="2026-05-28 15:08:39.931719982 +0000 UTC m=+70.820707065" May 28 15:09:10.336889 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:10.336847 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8959c2ac-69f0-4285-885b-1035cf57475e-registry-tls\") pod \"image-registry-5bd878d5c4-fx69p\" (UID: \"8959c2ac-69f0-4285-885b-1035cf57475e\") " pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" May 28 15:09:10.337302 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:10.337011 2559 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 28 15:09:10.337302 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:10.337028 2559 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bd878d5c4-fx69p: secret "image-registry-tls" not found May 28 15:09:10.337302 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:10.337084 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8959c2ac-69f0-4285-885b-1035cf57475e-registry-tls podName:8959c2ac-69f0-4285-885b-1035cf57475e nodeName:}" failed. No retries permitted until 2026-05-28 15:10:14.337066101 +0000 UTC m=+165.226053171 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8959c2ac-69f0-4285-885b-1035cf57475e-registry-tls") pod "image-registry-5bd878d5c4-fx69p" (UID: "8959c2ac-69f0-4285-885b-1035cf57475e") : secret "image-registry-tls" not found May 28 15:09:10.437678 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:10.437648 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8fa7b229-d678-41cd-b578-a31774414667-cert\") pod \"ingress-canary-9h2xl\" (UID: \"8fa7b229-d678-41cd-b578-a31774414667\") " pod="openshift-ingress-canary/ingress-canary-9h2xl" May 28 15:09:10.437816 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:10.437784 2559 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 15:09:10.437855 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:10.437847 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fa7b229-d678-41cd-b578-a31774414667-cert podName:8fa7b229-d678-41cd-b578-a31774414667 nodeName:}" failed. No retries permitted until 2026-05-28 15:10:14.437831155 +0000 UTC m=+165.326818230 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/8fa7b229-d678-41cd-b578-a31774414667-cert") pod "ingress-canary-9h2xl" (UID: "8fa7b229-d678-41cd-b578-a31774414667") : secret "canary-serving-cert" not found May 28 15:09:10.538636 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:10.538608 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/59c5be81-b339-4a6d-9c00-f66c662406a3-metrics-tls\") pod \"dns-default-nvshr\" (UID: \"59c5be81-b339-4a6d-9c00-f66c662406a3\") " pod="openshift-dns/dns-default-nvshr" May 28 15:09:10.538756 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:10.538738 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 15:09:10.538801 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:10.538791 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59c5be81-b339-4a6d-9c00-f66c662406a3-metrics-tls podName:59c5be81-b339-4a6d-9c00-f66c662406a3 nodeName:}" failed. No retries permitted until 2026-05-28 15:10:14.538777899 +0000 UTC m=+165.427764963 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/59c5be81-b339-4a6d-9c00-f66c662406a3-metrics-tls") pod "dns-default-nvshr" (UID: "59c5be81-b339-4a6d-9c00-f66c662406a3") : secret "dns-default-metrics-tls" not found May 28 15:09:10.921087 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:10.921062 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-rpkl2" May 28 15:09:34.202781 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.202749 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6c75959455-7zpd4"] May 28 15:09:34.204520 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.204501 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-7zpd4" May 28 15:09:34.208108 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.208081 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" May 28 15:09:34.209432 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.209413 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" May 28 15:09:34.209548 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.209430 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" May 28 15:09:34.209548 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.209431 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-drthw\"" May 28 15:09:34.209671 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.209509 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" May 28 15:09:34.214516 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.214498 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6c75959455-7zpd4"] May 28 15:09:34.293578 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.293550 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-786wt\" (UniqueName: \"kubernetes.io/projected/b6714928-0dd7-4b82-b82c-9c83f3967f52-kube-api-access-786wt\") pod \"cluster-monitoring-operator-6c75959455-7zpd4\" (UID: \"b6714928-0dd7-4b82-b82c-9c83f3967f52\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-7zpd4" May 28 15:09:34.293693 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.293602 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b6714928-0dd7-4b82-b82c-9c83f3967f52-telemetry-config\") pod \"cluster-monitoring-operator-6c75959455-7zpd4\" (UID: \"b6714928-0dd7-4b82-b82c-9c83f3967f52\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-7zpd4" May 28 15:09:34.293693 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.293680 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b6714928-0dd7-4b82-b82c-9c83f3967f52-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-7zpd4\" (UID: \"b6714928-0dd7-4b82-b82c-9c83f3967f52\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-7zpd4" May 28 15:09:34.300764 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.300741 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-697587599-kmcmq"] May 28 15:09:34.302667 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.302651 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-kmcmq" May 28 15:09:34.304982 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.304959 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" May 28 15:09:34.305099 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.305082 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-jz95k\"" May 28 15:09:34.305163 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.305101 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" May 28 15:09:34.305163 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.305082 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" May 28 15:09:34.312981 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.312921 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-7zwzb"] May 28 15:09:34.316216 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.316198 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7f44b6dd87-rjwvn"] May 28 15:09:34.316360 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.316347 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-7zwzb" May 28 15:09:34.317967 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.317950 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-69c468644b-ff99j"] May 28 15:09:34.318106 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.318089 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-rjwvn" May 28 15:09:34.319054 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.319017 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" May 28 15:09:34.319146 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.319060 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" May 28 15:09:34.319146 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.319064 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-dv8f5\"" May 28 15:09:34.319146 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.319116 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" May 28 15:09:34.319404 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.319387 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" May 28 15:09:34.319726 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.319712 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-697587599-kmcmq"] May 28 15:09:34.319790 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.319784 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-69c468644b-ff99j" May 28 15:09:34.320551 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.320532 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-mgvmw\"" May 28 15:09:34.322412 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.322393 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" May 28 15:09:34.322538 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.322428 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" May 28 15:09:34.322538 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.322400 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" May 28 15:09:34.322742 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.322714 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" May 28 15:09:34.322876 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.322815 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" May 28 15:09:34.322966 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.322950 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-xsmsc\"" May 28 15:09:34.323566 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.323442 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" May 28 15:09:34.323783 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.323763 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-7zwzb"] May 28 15:09:34.327009 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.326991 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7f44b6dd87-rjwvn"] May 28 15:09:34.328855 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.328838 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-69c468644b-ff99j"] May 28 15:09:34.394577 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.394544 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d75362ae-8357-492a-a275-c3ace0009a19-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-7zwzb\" (UID: \"d75362ae-8357-492a-a275-c3ace0009a19\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-7zwzb" May 28 15:09:34.394735 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.394584 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b6714928-0dd7-4b82-b82c-9c83f3967f52-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-7zpd4\" (UID: \"b6714928-0dd7-4b82-b82c-9c83f3967f52\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-7zpd4" May 28 15:09:34.394735 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.394651 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-786wt\" (UniqueName: \"kubernetes.io/projected/b6714928-0dd7-4b82-b82c-9c83f3967f52-kube-api-access-786wt\") pod \"cluster-monitoring-operator-6c75959455-7zpd4\" (UID: \"b6714928-0dd7-4b82-b82c-9c83f3967f52\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-7zpd4" May 28 15:09:34.394735 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:34.394696 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found May 28 15:09:34.394903 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.394695 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kclkp\" (UniqueName: \"kubernetes.io/projected/34c7820d-e5f2-41e9-89b0-f7e8301a0e6f-kube-api-access-kclkp\") pod \"cluster-samples-operator-697587599-kmcmq\" (UID: \"34c7820d-e5f2-41e9-89b0-f7e8301a0e6f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-kmcmq" May 28 15:09:34.394903 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:34.394752 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6714928-0dd7-4b82-b82c-9c83f3967f52-cluster-monitoring-operator-tls podName:b6714928-0dd7-4b82-b82c-9c83f3967f52 nodeName:}" failed. No retries permitted until 2026-05-28 15:09:34.894733752 +0000 UTC m=+125.783720814 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b6714928-0dd7-4b82-b82c-9c83f3967f52-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-7zpd4" (UID: "b6714928-0dd7-4b82-b82c-9c83f3967f52") : secret "cluster-monitoring-operator-tls" not found May 28 15:09:34.394903 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.394804 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d75362ae-8357-492a-a275-c3ace0009a19-config\") pod \"kube-storage-version-migrator-operator-565644f49-7zwzb\" (UID: \"d75362ae-8357-492a-a275-c3ace0009a19\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-7zwzb" May 28 15:09:34.394903 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.394842 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b6714928-0dd7-4b82-b82c-9c83f3967f52-telemetry-config\") pod \"cluster-monitoring-operator-6c75959455-7zpd4\" (UID: \"b6714928-0dd7-4b82-b82c-9c83f3967f52\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-7zpd4" May 28 15:09:34.394903 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.394873 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dx7js\" (UniqueName: \"kubernetes.io/projected/d75362ae-8357-492a-a275-c3ace0009a19-kube-api-access-dx7js\") pod \"kube-storage-version-migrator-operator-565644f49-7zwzb\" (UID: \"d75362ae-8357-492a-a275-c3ace0009a19\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-7zwzb" May 28 15:09:34.394903 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.394901 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/34c7820d-e5f2-41e9-89b0-f7e8301a0e6f-samples-operator-tls\") pod \"cluster-samples-operator-697587599-kmcmq\" (UID: \"34c7820d-e5f2-41e9-89b0-f7e8301a0e6f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-kmcmq" May 28 15:09:34.395441 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.395423 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b6714928-0dd7-4b82-b82c-9c83f3967f52-telemetry-config\") pod \"cluster-monitoring-operator-6c75959455-7zpd4\" (UID: \"b6714928-0dd7-4b82-b82c-9c83f3967f52\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-7zpd4" May 28 15:09:34.410535 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.410496 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-786wt\" (UniqueName: \"kubernetes.io/projected/b6714928-0dd7-4b82-b82c-9c83f3967f52-kube-api-access-786wt\") pod \"cluster-monitoring-operator-6c75959455-7zpd4\" (UID: \"b6714928-0dd7-4b82-b82c-9c83f3967f52\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-7zpd4" May 28 15:09:34.495773 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.495753 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3889dc6c-fe09-49a2-b52a-cb4de8f769de-service-ca-bundle\") pod \"router-default-69c468644b-ff99j\" (UID: \"3889dc6c-fe09-49a2-b52a-cb4de8f769de\") " pod="openshift-ingress/router-default-69c468644b-ff99j" May 28 15:09:34.495877 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.495790 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/34c7820d-e5f2-41e9-89b0-f7e8301a0e6f-samples-operator-tls\") pod \"cluster-samples-operator-697587599-kmcmq\" (UID: \"34c7820d-e5f2-41e9-89b0-f7e8301a0e6f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-kmcmq" May 28 15:09:34.495877 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.495814 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d75362ae-8357-492a-a275-c3ace0009a19-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-7zwzb\" (UID: \"d75362ae-8357-492a-a275-c3ace0009a19\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-7zwzb" May 28 15:09:34.495877 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.495851 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3889dc6c-fe09-49a2-b52a-cb4de8f769de-default-certificate\") pod \"router-default-69c468644b-ff99j\" (UID: \"3889dc6c-fe09-49a2-b52a-cb4de8f769de\") " pod="openshift-ingress/router-default-69c468644b-ff99j" May 28 15:09:34.495997 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.495878 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3889dc6c-fe09-49a2-b52a-cb4de8f769de-stats-auth\") pod \"router-default-69c468644b-ff99j\" (UID: \"3889dc6c-fe09-49a2-b52a-cb4de8f769de\") " pod="openshift-ingress/router-default-69c468644b-ff99j" May 28 15:09:34.495997 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:34.495894 2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found May 28 15:09:34.495997 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.495907 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3889dc6c-fe09-49a2-b52a-cb4de8f769de-metrics-certs\") pod \"router-default-69c468644b-ff99j\" (UID: \"3889dc6c-fe09-49a2-b52a-cb4de8f769de\") " pod="openshift-ingress/router-default-69c468644b-ff99j" May 28 15:09:34.495997 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:34.495959 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34c7820d-e5f2-41e9-89b0-f7e8301a0e6f-samples-operator-tls podName:34c7820d-e5f2-41e9-89b0-f7e8301a0e6f nodeName:}" failed. No retries permitted until 2026-05-28 15:09:34.99594033 +0000 UTC m=+125.884927411 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/34c7820d-e5f2-41e9-89b0-f7e8301a0e6f-samples-operator-tls") pod "cluster-samples-operator-697587599-kmcmq" (UID: "34c7820d-e5f2-41e9-89b0-f7e8301a0e6f") : secret "samples-operator-tls" not found May 28 15:09:34.495997 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.495992 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kclkp\" (UniqueName: \"kubernetes.io/projected/34c7820d-e5f2-41e9-89b0-f7e8301a0e6f-kube-api-access-kclkp\") pod \"cluster-samples-operator-697587599-kmcmq\" (UID: \"34c7820d-e5f2-41e9-89b0-f7e8301a0e6f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-kmcmq" May 28 15:09:34.496230 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.496035 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-849ww\" (UniqueName: \"kubernetes.io/projected/3889dc6c-fe09-49a2-b52a-cb4de8f769de-kube-api-access-849ww\") pod \"router-default-69c468644b-ff99j\" (UID: \"3889dc6c-fe09-49a2-b52a-cb4de8f769de\") " pod="openshift-ingress/router-default-69c468644b-ff99j" May 28 15:09:34.496230 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.496065 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d75362ae-8357-492a-a275-c3ace0009a19-config\") pod \"kube-storage-version-migrator-operator-565644f49-7zwzb\" (UID: \"d75362ae-8357-492a-a275-c3ace0009a19\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-7zwzb" May 28 15:09:34.496230 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.496134 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzqvb\" (UniqueName: \"kubernetes.io/projected/82f2a321-d05a-4b4d-9666-8539c5b0b7a4-kube-api-access-bzqvb\") pod \"network-check-source-7f44b6dd87-rjwvn\" (UID: \"82f2a321-d05a-4b4d-9666-8539c5b0b7a4\") " pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-rjwvn" May 28 15:09:34.496230 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.496174 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dx7js\" (UniqueName: \"kubernetes.io/projected/d75362ae-8357-492a-a275-c3ace0009a19-kube-api-access-dx7js\") pod \"kube-storage-version-migrator-operator-565644f49-7zwzb\" (UID: \"d75362ae-8357-492a-a275-c3ace0009a19\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-7zwzb" May 28 15:09:34.496588 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.496571 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d75362ae-8357-492a-a275-c3ace0009a19-config\") pod \"kube-storage-version-migrator-operator-565644f49-7zwzb\" (UID: \"d75362ae-8357-492a-a275-c3ace0009a19\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-7zwzb" May 28 15:09:34.497874 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.497857 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d75362ae-8357-492a-a275-c3ace0009a19-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-7zwzb\" (UID: \"d75362ae-8357-492a-a275-c3ace0009a19\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-7zwzb" May 28 15:09:34.504190 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.504171 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kclkp\" (UniqueName: \"kubernetes.io/projected/34c7820d-e5f2-41e9-89b0-f7e8301a0e6f-kube-api-access-kclkp\") pod \"cluster-samples-operator-697587599-kmcmq\" (UID: \"34c7820d-e5f2-41e9-89b0-f7e8301a0e6f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-kmcmq" May 28 15:09:34.504190 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.504180 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dx7js\" (UniqueName: \"kubernetes.io/projected/d75362ae-8357-492a-a275-c3ace0009a19-kube-api-access-dx7js\") pod \"kube-storage-version-migrator-operator-565644f49-7zwzb\" (UID: \"d75362ae-8357-492a-a275-c3ace0009a19\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-7zwzb" May 28 15:09:34.597235 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.597212 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3889dc6c-fe09-49a2-b52a-cb4de8f769de-metrics-certs\") pod \"router-default-69c468644b-ff99j\" (UID: \"3889dc6c-fe09-49a2-b52a-cb4de8f769de\") " pod="openshift-ingress/router-default-69c468644b-ff99j" May 28 15:09:34.597322 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.597256 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-849ww\" (UniqueName: \"kubernetes.io/projected/3889dc6c-fe09-49a2-b52a-cb4de8f769de-kube-api-access-849ww\") pod \"router-default-69c468644b-ff99j\" (UID: \"3889dc6c-fe09-49a2-b52a-cb4de8f769de\") " pod="openshift-ingress/router-default-69c468644b-ff99j" May 28 15:09:34.597322 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.597286 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bzqvb\" (UniqueName: \"kubernetes.io/projected/82f2a321-d05a-4b4d-9666-8539c5b0b7a4-kube-api-access-bzqvb\") pod \"network-check-source-7f44b6dd87-rjwvn\" (UID: \"82f2a321-d05a-4b4d-9666-8539c5b0b7a4\") " pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-rjwvn" May 28 15:09:34.597399 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:34.597336 2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 28 15:09:34.597399 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.597351 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3889dc6c-fe09-49a2-b52a-cb4de8f769de-service-ca-bundle\") pod \"router-default-69c468644b-ff99j\" (UID: \"3889dc6c-fe09-49a2-b52a-cb4de8f769de\") " pod="openshift-ingress/router-default-69c468644b-ff99j" May 28 15:09:34.597399 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:34.597391 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3889dc6c-fe09-49a2-b52a-cb4de8f769de-metrics-certs podName:3889dc6c-fe09-49a2-b52a-cb4de8f769de nodeName:}" failed. No retries permitted until 2026-05-28 15:09:35.09737378 +0000 UTC m=+125.986360842 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3889dc6c-fe09-49a2-b52a-cb4de8f769de-metrics-certs") pod "router-default-69c468644b-ff99j" (UID: "3889dc6c-fe09-49a2-b52a-cb4de8f769de") : secret "router-metrics-certs-default" not found May 28 15:09:34.597557 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.597457 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3889dc6c-fe09-49a2-b52a-cb4de8f769de-default-certificate\") pod \"router-default-69c468644b-ff99j\" (UID: \"3889dc6c-fe09-49a2-b52a-cb4de8f769de\") " pod="openshift-ingress/router-default-69c468644b-ff99j" May 28 15:09:34.597557 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:34.597464 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3889dc6c-fe09-49a2-b52a-cb4de8f769de-service-ca-bundle podName:3889dc6c-fe09-49a2-b52a-cb4de8f769de nodeName:}" failed. No retries permitted until 2026-05-28 15:09:35.097449139 +0000 UTC m=+125.986436204 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3889dc6c-fe09-49a2-b52a-cb4de8f769de-service-ca-bundle") pod "router-default-69c468644b-ff99j" (UID: "3889dc6c-fe09-49a2-b52a-cb4de8f769de") : configmap references non-existent config key: service-ca.crt May 28 15:09:34.597557 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.597511 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3889dc6c-fe09-49a2-b52a-cb4de8f769de-stats-auth\") pod \"router-default-69c468644b-ff99j\" (UID: \"3889dc6c-fe09-49a2-b52a-cb4de8f769de\") " pod="openshift-ingress/router-default-69c468644b-ff99j" May 28 15:09:34.599559 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.599535 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3889dc6c-fe09-49a2-b52a-cb4de8f769de-default-certificate\") pod \"router-default-69c468644b-ff99j\" (UID: \"3889dc6c-fe09-49a2-b52a-cb4de8f769de\") " pod="openshift-ingress/router-default-69c468644b-ff99j" May 28 15:09:34.599656 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.599539 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3889dc6c-fe09-49a2-b52a-cb4de8f769de-stats-auth\") pod \"router-default-69c468644b-ff99j\" (UID: \"3889dc6c-fe09-49a2-b52a-cb4de8f769de\") " pod="openshift-ingress/router-default-69c468644b-ff99j" May 28 15:09:34.605400 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.605383 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzqvb\" (UniqueName: \"kubernetes.io/projected/82f2a321-d05a-4b4d-9666-8539c5b0b7a4-kube-api-access-bzqvb\") pod \"network-check-source-7f44b6dd87-rjwvn\" (UID: \"82f2a321-d05a-4b4d-9666-8539c5b0b7a4\") " pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-rjwvn" May 28 15:09:34.605518 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.605410 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-849ww\" (UniqueName: \"kubernetes.io/projected/3889dc6c-fe09-49a2-b52a-cb4de8f769de-kube-api-access-849ww\") pod \"router-default-69c468644b-ff99j\" (UID: \"3889dc6c-fe09-49a2-b52a-cb4de8f769de\") " pod="openshift-ingress/router-default-69c468644b-ff99j" May 28 15:09:34.627467 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.627451 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-7zwzb" May 28 15:09:34.634026 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.634010 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-rjwvn" May 28 15:09:34.746304 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.746241 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-7zwzb"] May 28 15:09:34.749266 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:09:34.749238 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd75362ae_8357_492a_a275_c3ace0009a19.slice/crio-0356def5ac23aed7785b50521d4166c937e545b9b4d5f3f242e55094be762e11 WatchSource:0}: Error finding container 0356def5ac23aed7785b50521d4166c937e545b9b4d5f3f242e55094be762e11: Status 404 returned error can't find the container with id 0356def5ac23aed7785b50521d4166c937e545b9b4d5f3f242e55094be762e11 May 28 15:09:34.761517 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.761473 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7f44b6dd87-rjwvn"] May 28 15:09:34.763815 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:09:34.763789 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82f2a321_d05a_4b4d_9666_8539c5b0b7a4.slice/crio-156ee9d73a3b9298003f2124f390cf7446abef42621ddd631584b89f80bfd710 WatchSource:0}: Error finding container 156ee9d73a3b9298003f2124f390cf7446abef42621ddd631584b89f80bfd710: Status 404 returned error can't find the container with id 156ee9d73a3b9298003f2124f390cf7446abef42621ddd631584b89f80bfd710 May 28 15:09:34.900335 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:34.900306 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b6714928-0dd7-4b82-b82c-9c83f3967f52-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-7zpd4\" (UID: \"b6714928-0dd7-4b82-b82c-9c83f3967f52\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-7zpd4" May 28 15:09:34.900520 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:34.900477 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found May 28 15:09:34.900578 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:34.900567 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6714928-0dd7-4b82-b82c-9c83f3967f52-cluster-monitoring-operator-tls podName:b6714928-0dd7-4b82-b82c-9c83f3967f52 nodeName:}" failed. No retries permitted until 2026-05-28 15:09:35.900545507 +0000 UTC m=+126.789532573 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b6714928-0dd7-4b82-b82c-9c83f3967f52-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-7zpd4" (UID: "b6714928-0dd7-4b82-b82c-9c83f3967f52") : secret "cluster-monitoring-operator-tls" not found May 28 15:09:35.001064 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:35.000986 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/34c7820d-e5f2-41e9-89b0-f7e8301a0e6f-samples-operator-tls\") pod \"cluster-samples-operator-697587599-kmcmq\" (UID: \"34c7820d-e5f2-41e9-89b0-f7e8301a0e6f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-kmcmq" May 28 15:09:35.001203 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:35.001157 2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found May 28 15:09:35.001251 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:35.001240 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34c7820d-e5f2-41e9-89b0-f7e8301a0e6f-samples-operator-tls podName:34c7820d-e5f2-41e9-89b0-f7e8301a0e6f nodeName:}" failed. No retries permitted until 2026-05-28 15:09:36.001219836 +0000 UTC m=+126.890206925 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/34c7820d-e5f2-41e9-89b0-f7e8301a0e6f-samples-operator-tls") pod "cluster-samples-operator-697587599-kmcmq" (UID: "34c7820d-e5f2-41e9-89b0-f7e8301a0e6f") : secret "samples-operator-tls" not found May 28 15:09:35.016129 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:35.016104 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-rjwvn" event={"ID":"82f2a321-d05a-4b4d-9666-8539c5b0b7a4","Type":"ContainerStarted","Data":"d17c0d3bac0f54b41cf81d517e40a923c795bd4d8701c41703a1ef5ba55fc399"} May 28 15:09:35.016129 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:35.016139 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-rjwvn" event={"ID":"82f2a321-d05a-4b4d-9666-8539c5b0b7a4","Type":"ContainerStarted","Data":"156ee9d73a3b9298003f2124f390cf7446abef42621ddd631584b89f80bfd710"} May 28 15:09:35.017038 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:35.017017 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-7zwzb" event={"ID":"d75362ae-8357-492a-a275-c3ace0009a19","Type":"ContainerStarted","Data":"0356def5ac23aed7785b50521d4166c937e545b9b4d5f3f242e55094be762e11"} May 28 15:09:35.029896 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:35.029865 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-rjwvn" podStartSLOduration=1.029854784 podStartE2EDuration="1.029854784s" podCreationTimestamp="2026-05-28 15:09:34 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 15:09:35.029557434 +0000 UTC m=+125.918544520" watchObservedRunningTime="2026-05-28 15:09:35.029854784 +0000 UTC m=+125.918841868" May 28 15:09:35.101875 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:35.101852 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3889dc6c-fe09-49a2-b52a-cb4de8f769de-service-ca-bundle\") pod \"router-default-69c468644b-ff99j\" (UID: \"3889dc6c-fe09-49a2-b52a-cb4de8f769de\") " pod="openshift-ingress/router-default-69c468644b-ff99j" May 28 15:09:35.101978 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:35.101911 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3889dc6c-fe09-49a2-b52a-cb4de8f769de-metrics-certs\") pod \"router-default-69c468644b-ff99j\" (UID: \"3889dc6c-fe09-49a2-b52a-cb4de8f769de\") " pod="openshift-ingress/router-default-69c468644b-ff99j" May 28 15:09:35.102050 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:35.102033 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3889dc6c-fe09-49a2-b52a-cb4de8f769de-service-ca-bundle podName:3889dc6c-fe09-49a2-b52a-cb4de8f769de nodeName:}" failed. No retries permitted until 2026-05-28 15:09:36.102014795 +0000 UTC m=+126.991001867 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3889dc6c-fe09-49a2-b52a-cb4de8f769de-service-ca-bundle") pod "router-default-69c468644b-ff99j" (UID: "3889dc6c-fe09-49a2-b52a-cb4de8f769de") : configmap references non-existent config key: service-ca.crt May 28 15:09:35.102108 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:35.102081 2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 28 15:09:35.102159 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:35.102141 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3889dc6c-fe09-49a2-b52a-cb4de8f769de-metrics-certs podName:3889dc6c-fe09-49a2-b52a-cb4de8f769de nodeName:}" failed. No retries permitted until 2026-05-28 15:09:36.102125162 +0000 UTC m=+126.991112224 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3889dc6c-fe09-49a2-b52a-cb4de8f769de-metrics-certs") pod "router-default-69c468644b-ff99j" (UID: "3889dc6c-fe09-49a2-b52a-cb4de8f769de") : secret "router-metrics-certs-default" not found May 28 15:09:35.906828 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:35.906791 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b6714928-0dd7-4b82-b82c-9c83f3967f52-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-7zpd4\" (UID: \"b6714928-0dd7-4b82-b82c-9c83f3967f52\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-7zpd4" May 28 15:09:35.907212 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:35.906935 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found May 28 15:09:35.907212 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:35.906997 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6714928-0dd7-4b82-b82c-9c83f3967f52-cluster-monitoring-operator-tls podName:b6714928-0dd7-4b82-b82c-9c83f3967f52 nodeName:}" failed. No retries permitted until 2026-05-28 15:09:37.906982363 +0000 UTC m=+128.795969424 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b6714928-0dd7-4b82-b82c-9c83f3967f52-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-7zpd4" (UID: "b6714928-0dd7-4b82-b82c-9c83f3967f52") : secret "cluster-monitoring-operator-tls" not found May 28 15:09:36.008124 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:36.008093 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/34c7820d-e5f2-41e9-89b0-f7e8301a0e6f-samples-operator-tls\") pod \"cluster-samples-operator-697587599-kmcmq\" (UID: \"34c7820d-e5f2-41e9-89b0-f7e8301a0e6f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-kmcmq" May 28 15:09:36.008296 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:36.008261 2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found May 28 15:09:36.008368 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:36.008336 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34c7820d-e5f2-41e9-89b0-f7e8301a0e6f-samples-operator-tls podName:34c7820d-e5f2-41e9-89b0-f7e8301a0e6f nodeName:}" failed. No retries permitted until 2026-05-28 15:09:38.008315778 +0000 UTC m=+128.897302851 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/34c7820d-e5f2-41e9-89b0-f7e8301a0e6f-samples-operator-tls") pod "cluster-samples-operator-697587599-kmcmq" (UID: "34c7820d-e5f2-41e9-89b0-f7e8301a0e6f") : secret "samples-operator-tls" not found May 28 15:09:36.108594 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:36.108561 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3889dc6c-fe09-49a2-b52a-cb4de8f769de-metrics-certs\") pod \"router-default-69c468644b-ff99j\" (UID: \"3889dc6c-fe09-49a2-b52a-cb4de8f769de\") " pod="openshift-ingress/router-default-69c468644b-ff99j" May 28 15:09:36.108765 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:36.108724 2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 28 15:09:36.108833 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:36.108788 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3889dc6c-fe09-49a2-b52a-cb4de8f769de-metrics-certs podName:3889dc6c-fe09-49a2-b52a-cb4de8f769de nodeName:}" failed. No retries permitted until 2026-05-28 15:09:38.108769597 +0000 UTC m=+128.997756673 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3889dc6c-fe09-49a2-b52a-cb4de8f769de-metrics-certs") pod "router-default-69c468644b-ff99j" (UID: "3889dc6c-fe09-49a2-b52a-cb4de8f769de") : secret "router-metrics-certs-default" not found May 28 15:09:36.108833 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:36.108814 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3889dc6c-fe09-49a2-b52a-cb4de8f769de-service-ca-bundle\") pod \"router-default-69c468644b-ff99j\" (UID: \"3889dc6c-fe09-49a2-b52a-cb4de8f769de\") " pod="openshift-ingress/router-default-69c468644b-ff99j" May 28 15:09:36.109010 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:36.108992 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3889dc6c-fe09-49a2-b52a-cb4de8f769de-service-ca-bundle podName:3889dc6c-fe09-49a2-b52a-cb4de8f769de nodeName:}" failed. No retries permitted until 2026-05-28 15:09:38.108975977 +0000 UTC m=+128.997963054 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3889dc6c-fe09-49a2-b52a-cb4de8f769de-service-ca-bundle") pod "router-default-69c468644b-ff99j" (UID: "3889dc6c-fe09-49a2-b52a-cb4de8f769de") : configmap references non-existent config key: service-ca.crt May 28 15:09:37.022501 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:37.022434 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-7zwzb" event={"ID":"d75362ae-8357-492a-a275-c3ace0009a19","Type":"ContainerStarted","Data":"2e37d6719692b69147afc83b2dc999b88ed31e6ee5c6ada66fe536a86b23a431"} May 28 15:09:37.038926 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:37.038865 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-7zwzb" podStartSLOduration=0.947651763 podStartE2EDuration="3.0388462s" podCreationTimestamp="2026-05-28 15:09:34 +0000 UTC" firstStartedPulling="2026-05-28 15:09:34.75101433 +0000 UTC m=+125.640001392" lastFinishedPulling="2026-05-28 15:09:36.842208764 +0000 UTC m=+127.731195829" observedRunningTime="2026-05-28 15:09:37.036650707 +0000 UTC m=+127.925637805" watchObservedRunningTime="2026-05-28 15:09:37.0388462 +0000 UTC m=+127.927833286" May 28 15:09:37.924046 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:37.924004 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b6714928-0dd7-4b82-b82c-9c83f3967f52-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-7zpd4\" (UID: \"b6714928-0dd7-4b82-b82c-9c83f3967f52\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-7zpd4" May 28 15:09:37.924229 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:37.924156 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found May 28 15:09:37.924229 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:37.924217 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6714928-0dd7-4b82-b82c-9c83f3967f52-cluster-monitoring-operator-tls podName:b6714928-0dd7-4b82-b82c-9c83f3967f52 nodeName:}" failed. No retries permitted until 2026-05-28 15:09:41.924199502 +0000 UTC m=+132.813186570 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b6714928-0dd7-4b82-b82c-9c83f3967f52-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-7zpd4" (UID: "b6714928-0dd7-4b82-b82c-9c83f3967f52") : secret "cluster-monitoring-operator-tls" not found May 28 15:09:38.024414 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:38.024383 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/34c7820d-e5f2-41e9-89b0-f7e8301a0e6f-samples-operator-tls\") pod \"cluster-samples-operator-697587599-kmcmq\" (UID: \"34c7820d-e5f2-41e9-89b0-f7e8301a0e6f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-kmcmq" May 28 15:09:38.024796 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:38.024546 2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found May 28 15:09:38.024796 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:38.024627 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34c7820d-e5f2-41e9-89b0-f7e8301a0e6f-samples-operator-tls podName:34c7820d-e5f2-41e9-89b0-f7e8301a0e6f nodeName:}" failed. No retries permitted until 2026-05-28 15:09:42.024606914 +0000 UTC m=+132.913593984 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/34c7820d-e5f2-41e9-89b0-f7e8301a0e6f-samples-operator-tls") pod "cluster-samples-operator-697587599-kmcmq" (UID: "34c7820d-e5f2-41e9-89b0-f7e8301a0e6f") : secret "samples-operator-tls" not found May 28 15:09:38.125194 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:38.125156 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3889dc6c-fe09-49a2-b52a-cb4de8f769de-service-ca-bundle\") pod \"router-default-69c468644b-ff99j\" (UID: \"3889dc6c-fe09-49a2-b52a-cb4de8f769de\") " pod="openshift-ingress/router-default-69c468644b-ff99j" May 28 15:09:38.125347 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:38.125259 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3889dc6c-fe09-49a2-b52a-cb4de8f769de-metrics-certs\") pod \"router-default-69c468644b-ff99j\" (UID: \"3889dc6c-fe09-49a2-b52a-cb4de8f769de\") " pod="openshift-ingress/router-default-69c468644b-ff99j" May 28 15:09:38.125411 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:38.125359 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3889dc6c-fe09-49a2-b52a-cb4de8f769de-service-ca-bundle podName:3889dc6c-fe09-49a2-b52a-cb4de8f769de nodeName:}" failed. No retries permitted until 2026-05-28 15:09:42.125339997 +0000 UTC m=+133.014327062 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3889dc6c-fe09-49a2-b52a-cb4de8f769de-service-ca-bundle") pod "router-default-69c468644b-ff99j" (UID: "3889dc6c-fe09-49a2-b52a-cb4de8f769de") : configmap references non-existent config key: service-ca.crt May 28 15:09:38.125411 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:38.125376 2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 28 15:09:38.125527 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:38.125436 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3889dc6c-fe09-49a2-b52a-cb4de8f769de-metrics-certs podName:3889dc6c-fe09-49a2-b52a-cb4de8f769de nodeName:}" failed. No retries permitted until 2026-05-28 15:09:42.125418523 +0000 UTC m=+133.014405604 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3889dc6c-fe09-49a2-b52a-cb4de8f769de-metrics-certs") pod "router-default-69c468644b-ff99j" (UID: "3889dc6c-fe09-49a2-b52a-cb4de8f769de") : secret "router-metrics-certs-default" not found May 28 15:09:39.535834 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:39.535799 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3a78917-de69-42d0-8fbe-85f346276bf9-metrics-certs\") pod \"network-metrics-daemon-l9p7b\" (UID: \"e3a78917-de69-42d0-8fbe-85f346276bf9\") " pod="openshift-multus/network-metrics-daemon-l9p7b" May 28 15:09:39.536187 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:39.535933 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found May 28 15:09:39.536187 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:39.535989 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3a78917-de69-42d0-8fbe-85f346276bf9-metrics-certs podName:e3a78917-de69-42d0-8fbe-85f346276bf9 nodeName:}" failed. No retries permitted until 2026-05-28 15:11:41.535974754 +0000 UTC m=+252.424961816 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3a78917-de69-42d0-8fbe-85f346276bf9-metrics-certs") pod "network-metrics-daemon-l9p7b" (UID: "e3a78917-de69-42d0-8fbe-85f346276bf9") : secret "metrics-daemon-secret" not found May 28 15:09:41.953802 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:41.953764 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b6714928-0dd7-4b82-b82c-9c83f3967f52-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-7zpd4\" (UID: \"b6714928-0dd7-4b82-b82c-9c83f3967f52\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-7zpd4" May 28 15:09:41.954284 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:41.953906 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found May 28 15:09:41.954284 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:41.953975 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6714928-0dd7-4b82-b82c-9c83f3967f52-cluster-monitoring-operator-tls podName:b6714928-0dd7-4b82-b82c-9c83f3967f52 nodeName:}" failed. No retries permitted until 2026-05-28 15:09:49.953959317 +0000 UTC m=+140.842946380 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b6714928-0dd7-4b82-b82c-9c83f3967f52-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-7zpd4" (UID: "b6714928-0dd7-4b82-b82c-9c83f3967f52") : secret "cluster-monitoring-operator-tls" not found May 28 15:09:42.054620 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:42.054587 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/34c7820d-e5f2-41e9-89b0-f7e8301a0e6f-samples-operator-tls\") pod \"cluster-samples-operator-697587599-kmcmq\" (UID: \"34c7820d-e5f2-41e9-89b0-f7e8301a0e6f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-kmcmq" May 28 15:09:42.054788 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:42.054734 2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found May 28 15:09:42.054831 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:42.054795 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34c7820d-e5f2-41e9-89b0-f7e8301a0e6f-samples-operator-tls podName:34c7820d-e5f2-41e9-89b0-f7e8301a0e6f nodeName:}" failed. No retries permitted until 2026-05-28 15:09:50.054780315 +0000 UTC m=+140.943767378 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/34c7820d-e5f2-41e9-89b0-f7e8301a0e6f-samples-operator-tls") pod "cluster-samples-operator-697587599-kmcmq" (UID: "34c7820d-e5f2-41e9-89b0-f7e8301a0e6f") : secret "samples-operator-tls" not found May 28 15:09:42.155758 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:42.155723 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3889dc6c-fe09-49a2-b52a-cb4de8f769de-metrics-certs\") pod \"router-default-69c468644b-ff99j\" (UID: \"3889dc6c-fe09-49a2-b52a-cb4de8f769de\") " pod="openshift-ingress/router-default-69c468644b-ff99j" May 28 15:09:42.155906 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:42.155799 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3889dc6c-fe09-49a2-b52a-cb4de8f769de-service-ca-bundle\") pod \"router-default-69c468644b-ff99j\" (UID: \"3889dc6c-fe09-49a2-b52a-cb4de8f769de\") " pod="openshift-ingress/router-default-69c468644b-ff99j" May 28 15:09:42.155906 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:42.155891 2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 28 15:09:42.156005 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:42.155907 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3889dc6c-fe09-49a2-b52a-cb4de8f769de-service-ca-bundle podName:3889dc6c-fe09-49a2-b52a-cb4de8f769de nodeName:}" failed. No retries permitted until 2026-05-28 15:09:50.155892845 +0000 UTC m=+141.044879908 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3889dc6c-fe09-49a2-b52a-cb4de8f769de-service-ca-bundle") pod "router-default-69c468644b-ff99j" (UID: "3889dc6c-fe09-49a2-b52a-cb4de8f769de") : configmap references non-existent config key: service-ca.crt May 28 15:09:42.156005 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:42.155967 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3889dc6c-fe09-49a2-b52a-cb4de8f769de-metrics-certs podName:3889dc6c-fe09-49a2-b52a-cb4de8f769de nodeName:}" failed. No retries permitted until 2026-05-28 15:09:50.155947934 +0000 UTC m=+141.044935011 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3889dc6c-fe09-49a2-b52a-cb4de8f769de-metrics-certs") pod "router-default-69c468644b-ff99j" (UID: "3889dc6c-fe09-49a2-b52a-cb4de8f769de") : secret "router-metrics-certs-default" not found May 28 15:09:42.509000 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:42.508974 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kr9wd_297e38b8-9a81-4112-811d-a97cfa5341ff/dns-node-resolver/0.log" May 28 15:09:43.308313 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:43.308286 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bpxxm_a2246888-2a0d-48b8-8fe6-12e283e47eb0/node-ca/0.log" May 28 15:09:45.110662 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:45.110633 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-565644f49-7zwzb_d75362ae-8357-492a-a275-c3ace0009a19/kube-storage-version-migrator-operator/0.log" May 28 15:09:50.016517 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:50.016465 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b6714928-0dd7-4b82-b82c-9c83f3967f52-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-7zpd4\" (UID: \"b6714928-0dd7-4b82-b82c-9c83f3967f52\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-7zpd4" May 28 15:09:50.016874 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:50.016604 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found May 28 15:09:50.016874 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:09:50.016673 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6714928-0dd7-4b82-b82c-9c83f3967f52-cluster-monitoring-operator-tls podName:b6714928-0dd7-4b82-b82c-9c83f3967f52 nodeName:}" failed. No retries permitted until 2026-05-28 15:10:06.016655427 +0000 UTC m=+156.905642493 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b6714928-0dd7-4b82-b82c-9c83f3967f52-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-7zpd4" (UID: "b6714928-0dd7-4b82-b82c-9c83f3967f52") : secret "cluster-monitoring-operator-tls" not found May 28 15:09:50.117371 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:50.117341 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/34c7820d-e5f2-41e9-89b0-f7e8301a0e6f-samples-operator-tls\") pod \"cluster-samples-operator-697587599-kmcmq\" (UID: \"34c7820d-e5f2-41e9-89b0-f7e8301a0e6f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-kmcmq" May 28 15:09:50.119746 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:50.119720 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/34c7820d-e5f2-41e9-89b0-f7e8301a0e6f-samples-operator-tls\") pod \"cluster-samples-operator-697587599-kmcmq\" (UID: \"34c7820d-e5f2-41e9-89b0-f7e8301a0e6f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-kmcmq" May 28 15:09:50.211572 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:50.211531 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-kmcmq" May 28 15:09:50.218568 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:50.218544 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3889dc6c-fe09-49a2-b52a-cb4de8f769de-service-ca-bundle\") pod \"router-default-69c468644b-ff99j\" (UID: \"3889dc6c-fe09-49a2-b52a-cb4de8f769de\") " pod="openshift-ingress/router-default-69c468644b-ff99j" May 28 15:09:50.218683 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:50.218622 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3889dc6c-fe09-49a2-b52a-cb4de8f769de-metrics-certs\") pod \"router-default-69c468644b-ff99j\" (UID: \"3889dc6c-fe09-49a2-b52a-cb4de8f769de\") " pod="openshift-ingress/router-default-69c468644b-ff99j" May 28 15:09:50.219083 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:50.219063 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3889dc6c-fe09-49a2-b52a-cb4de8f769de-service-ca-bundle\") pod \"router-default-69c468644b-ff99j\" (UID: \"3889dc6c-fe09-49a2-b52a-cb4de8f769de\") " pod="openshift-ingress/router-default-69c468644b-ff99j" May 28 15:09:50.221125 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:50.221094 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3889dc6c-fe09-49a2-b52a-cb4de8f769de-metrics-certs\") pod \"router-default-69c468644b-ff99j\" (UID: \"3889dc6c-fe09-49a2-b52a-cb4de8f769de\") " pod="openshift-ingress/router-default-69c468644b-ff99j" May 28 15:09:50.239016 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:50.238979 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-69c468644b-ff99j" May 28 15:09:50.336768 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:50.336729 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-697587599-kmcmq"] May 28 15:09:50.360927 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:50.360905 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-69c468644b-ff99j"] May 28 15:09:50.363355 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:09:50.363330 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3889dc6c_fe09_49a2_b52a_cb4de8f769de.slice/crio-28a51c7123b4d448d2238565e9a636ec35fefe432201ae582053631437e8f60d WatchSource:0}: Error finding container 28a51c7123b4d448d2238565e9a636ec35fefe432201ae582053631437e8f60d: Status 404 returned error can't find the container with id 28a51c7123b4d448d2238565e9a636ec35fefe432201ae582053631437e8f60d May 28 15:09:51.047090 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:51.047049 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-kmcmq" event={"ID":"34c7820d-e5f2-41e9-89b0-f7e8301a0e6f","Type":"ContainerStarted","Data":"efec51ebd185485772078002df229df5c51c0a589b2266f8566094ac452a3486"} May 28 15:09:51.048436 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:51.048408 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-69c468644b-ff99j" event={"ID":"3889dc6c-fe09-49a2-b52a-cb4de8f769de","Type":"ContainerStarted","Data":"60e5f509bdf3839ca0373563397d2d09bd6d4064b47ddc9eb4838c9e3182b589"} May 28 15:09:51.048555 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:51.048443 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-69c468644b-ff99j" event={"ID":"3889dc6c-fe09-49a2-b52a-cb4de8f769de","Type":"ContainerStarted","Data":"28a51c7123b4d448d2238565e9a636ec35fefe432201ae582053631437e8f60d"} May 28 15:09:51.067829 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:51.067763 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/router-default-69c468644b-ff99j" podStartSLOduration=17.067749983 podStartE2EDuration="17.067749983s" podCreationTimestamp="2026-05-28 15:09:34 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 15:09:51.066847418 +0000 UTC m=+141.955834503" watchObservedRunningTime="2026-05-28 15:09:51.067749983 +0000 UTC m=+141.956737106" May 28 15:09:51.239523 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:51.239476 2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-69c468644b-ff99j" May 28 15:09:51.242231 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:51.242201 2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-69c468644b-ff99j" May 28 15:09:52.050424 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:52.050383 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-69c468644b-ff99j" May 28 15:09:52.051856 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:52.051834 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-69c468644b-ff99j" May 28 15:09:53.054319 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:53.054283 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-kmcmq" event={"ID":"34c7820d-e5f2-41e9-89b0-f7e8301a0e6f","Type":"ContainerStarted","Data":"62ede76a14efa043fe068bc837b5e997592a55eb5f419a8eea128ba2f2877e0d"} May 28 15:09:53.054722 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:09:53.054325 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-kmcmq" event={"ID":"34c7820d-e5f2-41e9-89b0-f7e8301a0e6f","Type":"ContainerStarted","Data":"eed090bd71415802a7440fae432c8f38fc4ca25c793a881fbbfdb39bfb4398fc"} May 28 15:10:01.811336 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:01.811283 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-kmcmq" podStartSLOduration=25.888011346 podStartE2EDuration="27.811268601s" podCreationTimestamp="2026-05-28 15:09:34 +0000 UTC" firstStartedPulling="2026-05-28 15:09:50.372831027 +0000 UTC m=+141.261818090" lastFinishedPulling="2026-05-28 15:09:52.296088282 +0000 UTC m=+143.185075345" observedRunningTime="2026-05-28 15:09:53.068912676 +0000 UTC m=+143.957899760" watchObservedRunningTime="2026-05-28 15:10:01.811268601 +0000 UTC m=+152.700255745" May 28 15:10:01.811800 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:01.811770 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-9s6tj"] May 28 15:10:01.814202 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:01.814181 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-9s6tj" May 28 15:10:01.817873 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:01.817840 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" May 28 15:10:01.817989 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:01.817924 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" May 28 15:10:01.818046 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:01.818021 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-kd87r\"" May 28 15:10:01.822017 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:01.821993 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-9s6tj"] May 28 15:10:01.826295 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:01.826276 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-prjnr"] May 28 15:10:01.828760 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:01.828738 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-prjnr" May 28 15:10:01.831548 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:01.831527 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" May 28 15:10:01.831649 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:01.831555 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" May 28 15:10:01.831649 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:01.831566 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" May 28 15:10:01.831747 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:01.831686 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" May 28 15:10:01.832535 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:01.832520 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nrcdd\"" May 28 15:10:01.843811 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:01.843790 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-prjnr"] May 28 15:10:01.906669 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:01.906638 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e20212be-1864-4edd-8ecb-90312a889a89-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-prjnr\" (UID: \"e20212be-1864-4edd-8ecb-90312a889a89\") " pod="openshift-insights/insights-runtime-extractor-prjnr" May 28 15:10:01.906814 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:01.906696 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkh2k\" (UniqueName: \"kubernetes.io/projected/e20212be-1864-4edd-8ecb-90312a889a89-kube-api-access-dkh2k\") pod \"insights-runtime-extractor-prjnr\" (UID: \"e20212be-1864-4edd-8ecb-90312a889a89\") " pod="openshift-insights/insights-runtime-extractor-prjnr" May 28 15:10:01.906814 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:01.906786 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e20212be-1864-4edd-8ecb-90312a889a89-crio-socket\") pod \"insights-runtime-extractor-prjnr\" (UID: \"e20212be-1864-4edd-8ecb-90312a889a89\") " pod="openshift-insights/insights-runtime-extractor-prjnr" May 28 15:10:01.906814 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:01.906812 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e20212be-1864-4edd-8ecb-90312a889a89-data-volume\") pod \"insights-runtime-extractor-prjnr\" (UID: \"e20212be-1864-4edd-8ecb-90312a889a89\") " pod="openshift-insights/insights-runtime-extractor-prjnr" May 28 15:10:01.906967 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:01.906947 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cf3cad76-3b1f-4cce-bc1b-e3c710fd6a58-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-9s6tj\" (UID: \"cf3cad76-3b1f-4cce-bc1b-e3c710fd6a58\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-9s6tj" May 28 15:10:01.907007 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:01.906995 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cf3cad76-3b1f-4cce-bc1b-e3c710fd6a58-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-9s6tj\" (UID: \"cf3cad76-3b1f-4cce-bc1b-e3c710fd6a58\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-9s6tj" May 28 15:10:01.907041 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:01.907022 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e20212be-1864-4edd-8ecb-90312a889a89-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-prjnr\" (UID: \"e20212be-1864-4edd-8ecb-90312a889a89\") " pod="openshift-insights/insights-runtime-extractor-prjnr" May 28 15:10:01.912283 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:01.912257 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-57bd96b888-76mzv"] May 28 15:10:01.914400 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:01.914380 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-57bd96b888-76mzv" May 28 15:10:01.919649 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:01.919628 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-65vb2\"" May 28 15:10:01.919953 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:01.919937 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" May 28 15:10:01.919953 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:01.919944 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" May 28 15:10:01.926195 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:01.926174 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-57bd96b888-76mzv"] May 28 15:10:02.007315 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:02.007287 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cf3cad76-3b1f-4cce-bc1b-e3c710fd6a58-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-9s6tj\" (UID: \"cf3cad76-3b1f-4cce-bc1b-e3c710fd6a58\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-9s6tj" May 28 15:10:02.007524 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:02.007324 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cf3cad76-3b1f-4cce-bc1b-e3c710fd6a58-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-9s6tj\" (UID: \"cf3cad76-3b1f-4cce-bc1b-e3c710fd6a58\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-9s6tj" May 28 15:10:02.007524 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:02.007341 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e20212be-1864-4edd-8ecb-90312a889a89-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-prjnr\" (UID: \"e20212be-1864-4edd-8ecb-90312a889a89\") " pod="openshift-insights/insights-runtime-extractor-prjnr" May 28 15:10:02.007524 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:02.007360 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e20212be-1864-4edd-8ecb-90312a889a89-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-prjnr\" (UID: \"e20212be-1864-4edd-8ecb-90312a889a89\") " pod="openshift-insights/insights-runtime-extractor-prjnr" May 28 15:10:02.007655 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:02.007573 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-294k9\" (UniqueName: \"kubernetes.io/projected/0951dff9-48da-4274-8d7e-b47651895056-kube-api-access-294k9\") pod \"downloads-57bd96b888-76mzv\" (UID: \"0951dff9-48da-4274-8d7e-b47651895056\") " pod="openshift-console/downloads-57bd96b888-76mzv" May 28 15:10:02.007655 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:02.007616 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dkh2k\" (UniqueName: \"kubernetes.io/projected/e20212be-1864-4edd-8ecb-90312a889a89-kube-api-access-dkh2k\") pod \"insights-runtime-extractor-prjnr\" (UID: \"e20212be-1864-4edd-8ecb-90312a889a89\") " pod="openshift-insights/insights-runtime-extractor-prjnr" May 28 15:10:02.007735 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:02.007656 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e20212be-1864-4edd-8ecb-90312a889a89-crio-socket\") pod \"insights-runtime-extractor-prjnr\" (UID: \"e20212be-1864-4edd-8ecb-90312a889a89\") " pod="openshift-insights/insights-runtime-extractor-prjnr" May 28 15:10:02.007735 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:02.007682 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e20212be-1864-4edd-8ecb-90312a889a89-data-volume\") pod \"insights-runtime-extractor-prjnr\" (UID: \"e20212be-1864-4edd-8ecb-90312a889a89\") " pod="openshift-insights/insights-runtime-extractor-prjnr" May 28 15:10:02.007828 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:02.007784 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e20212be-1864-4edd-8ecb-90312a889a89-crio-socket\") pod \"insights-runtime-extractor-prjnr\" (UID: \"e20212be-1864-4edd-8ecb-90312a889a89\") " pod="openshift-insights/insights-runtime-extractor-prjnr" May 28 15:10:02.008013 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:02.007990 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e20212be-1864-4edd-8ecb-90312a889a89-data-volume\") pod \"insights-runtime-extractor-prjnr\" (UID: \"e20212be-1864-4edd-8ecb-90312a889a89\") " pod="openshift-insights/insights-runtime-extractor-prjnr" May 28 15:10:02.008108 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:02.008026 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cf3cad76-3b1f-4cce-bc1b-e3c710fd6a58-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-9s6tj\" (UID: \"cf3cad76-3b1f-4cce-bc1b-e3c710fd6a58\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-9s6tj" May 28 15:10:02.008108 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:02.008053 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e20212be-1864-4edd-8ecb-90312a889a89-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-prjnr\" (UID: \"e20212be-1864-4edd-8ecb-90312a889a89\") " pod="openshift-insights/insights-runtime-extractor-prjnr" May 28 15:10:02.009747 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:02.009727 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e20212be-1864-4edd-8ecb-90312a889a89-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-prjnr\" (UID: \"e20212be-1864-4edd-8ecb-90312a889a89\") " pod="openshift-insights/insights-runtime-extractor-prjnr" May 28 15:10:02.009841 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:02.009733 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cf3cad76-3b1f-4cce-bc1b-e3c710fd6a58-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-9s6tj\" (UID: \"cf3cad76-3b1f-4cce-bc1b-e3c710fd6a58\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-9s6tj" May 28 15:10:02.023806 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:02.023789 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkh2k\" (UniqueName: \"kubernetes.io/projected/e20212be-1864-4edd-8ecb-90312a889a89-kube-api-access-dkh2k\") pod \"insights-runtime-extractor-prjnr\" (UID: \"e20212be-1864-4edd-8ecb-90312a889a89\") " pod="openshift-insights/insights-runtime-extractor-prjnr" May 28 15:10:02.109007 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:02.108920 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-294k9\" (UniqueName: \"kubernetes.io/projected/0951dff9-48da-4274-8d7e-b47651895056-kube-api-access-294k9\") pod \"downloads-57bd96b888-76mzv\" (UID: \"0951dff9-48da-4274-8d7e-b47651895056\") " pod="openshift-console/downloads-57bd96b888-76mzv" May 28 15:10:02.120087 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:02.120062 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-294k9\" (UniqueName: \"kubernetes.io/projected/0951dff9-48da-4274-8d7e-b47651895056-kube-api-access-294k9\") pod \"downloads-57bd96b888-76mzv\" (UID: \"0951dff9-48da-4274-8d7e-b47651895056\") " pod="openshift-console/downloads-57bd96b888-76mzv" May 28 15:10:02.124405 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:02.124387 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-9s6tj" May 28 15:10:02.139065 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:02.139043 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-prjnr" May 28 15:10:02.223808 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:02.223773 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-57bd96b888-76mzv" May 28 15:10:02.248932 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:02.248900 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-9s6tj"] May 28 15:10:02.251985 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:10:02.251957 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf3cad76_3b1f_4cce_bc1b_e3c710fd6a58.slice/crio-6d20be43c50302bc7c1240fe34568fbcdec4a23dd9a93c511e55d5dc0188ebee WatchSource:0}: Error finding container 6d20be43c50302bc7c1240fe34568fbcdec4a23dd9a93c511e55d5dc0188ebee: Status 404 returned error can't find the container with id 6d20be43c50302bc7c1240fe34568fbcdec4a23dd9a93c511e55d5dc0188ebee May 28 15:10:02.274923 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:02.274868 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-prjnr"] May 28 15:10:02.279069 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:10:02.279029 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode20212be_1864_4edd_8ecb_90312a889a89.slice/crio-75b61329c1f20c89c29832d8b1989d671b2d328044f8fdabb07963f7c015270b WatchSource:0}: Error finding container 75b61329c1f20c89c29832d8b1989d671b2d328044f8fdabb07963f7c015270b: Status 404 returned error can't find the container with id 75b61329c1f20c89c29832d8b1989d671b2d328044f8fdabb07963f7c015270b May 28 15:10:02.350354 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:02.350329 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-57bd96b888-76mzv"] May 28 15:10:02.353706 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:10:02.353673 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0951dff9_48da_4274_8d7e_b47651895056.slice/crio-18a683071ca6639885badd93b03a78a403958cb2a6b3e0a6592a00d2cee2af83 WatchSource:0}: Error finding container 18a683071ca6639885badd93b03a78a403958cb2a6b3e0a6592a00d2cee2af83: Status 404 returned error can't find the container with id 18a683071ca6639885badd93b03a78a403958cb2a6b3e0a6592a00d2cee2af83 May 28 15:10:03.082891 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:03.082854 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-prjnr" event={"ID":"e20212be-1864-4edd-8ecb-90312a889a89","Type":"ContainerStarted","Data":"66b3dc3650aeec20b9f276610f799bc9aeb4522de2e73642eadfa49ea754b19a"} May 28 15:10:03.082891 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:03.082894 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-prjnr" event={"ID":"e20212be-1864-4edd-8ecb-90312a889a89","Type":"ContainerStarted","Data":"d3259cc9fb27e703a851bd4cd1ed75628b204429b08c0aa81a0b34ab93bf1823"} May 28 15:10:03.083384 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:03.082909 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-prjnr" event={"ID":"e20212be-1864-4edd-8ecb-90312a889a89","Type":"ContainerStarted","Data":"75b61329c1f20c89c29832d8b1989d671b2d328044f8fdabb07963f7c015270b"} May 28 15:10:03.083970 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:03.083942 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-9s6tj" event={"ID":"cf3cad76-3b1f-4cce-bc1b-e3c710fd6a58","Type":"ContainerStarted","Data":"6d20be43c50302bc7c1240fe34568fbcdec4a23dd9a93c511e55d5dc0188ebee"} May 28 15:10:03.084987 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:03.084961 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-57bd96b888-76mzv" event={"ID":"0951dff9-48da-4274-8d7e-b47651895056","Type":"ContainerStarted","Data":"18a683071ca6639885badd93b03a78a403958cb2a6b3e0a6592a00d2cee2af83"} May 28 15:10:04.090239 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:04.090158 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-9s6tj" event={"ID":"cf3cad76-3b1f-4cce-bc1b-e3c710fd6a58","Type":"ContainerStarted","Data":"a72a00e12539331a9119856cee3a9174f5df6b61cbec8a637bc4868ceca1c983"} May 28 15:10:04.108686 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:04.108641 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-9s6tj" podStartSLOduration=1.581228924 podStartE2EDuration="3.108627324s" podCreationTimestamp="2026-05-28 15:10:01 +0000 UTC" firstStartedPulling="2026-05-28 15:10:02.254170596 +0000 UTC m=+153.143157658" lastFinishedPulling="2026-05-28 15:10:03.781568996 +0000 UTC m=+154.670556058" observedRunningTime="2026-05-28 15:10:04.107147926 +0000 UTC m=+154.996135028" watchObservedRunningTime="2026-05-28 15:10:04.108627324 +0000 UTC m=+154.997614408" May 28 15:10:05.095161 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:05.095116 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-prjnr" event={"ID":"e20212be-1864-4edd-8ecb-90312a889a89","Type":"ContainerStarted","Data":"5514edf58190c9fad78eec4b8be0a6e29268d2c93c4f1af7d1403bcada98a611"} May 28 15:10:05.112231 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:05.112186 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-prjnr" podStartSLOduration=1.8045431459999999 podStartE2EDuration="4.11217281s" podCreationTimestamp="2026-05-28 15:10:01 +0000 UTC" firstStartedPulling="2026-05-28 15:10:02.331380086 +0000 UTC m=+153.220367154" lastFinishedPulling="2026-05-28 15:10:04.639009752 +0000 UTC m=+155.527996818" observedRunningTime="2026-05-28 15:10:05.11174731 +0000 UTC m=+156.000734406" watchObservedRunningTime="2026-05-28 15:10:05.11217281 +0000 UTC m=+156.001159894" May 28 15:10:05.546043 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:05.546009 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-845b45fdb8-ffvfm"] May 28 15:10:05.548572 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:05.548546 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-845b45fdb8-ffvfm" May 28 15:10:05.552089 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:05.552052 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" May 28 15:10:05.552202 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:05.552088 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" May 28 15:10:05.552202 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:05.552111 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-wzvkg\"" May 28 15:10:05.552202 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:05.552123 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" May 28 15:10:05.552202 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:05.552168 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" May 28 15:10:05.552202 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:05.552124 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" May 28 15:10:05.556278 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:05.556234 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-845b45fdb8-ffvfm"] May 28 15:10:05.640215 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:05.640184 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1-oauth-serving-cert\") pod \"console-845b45fdb8-ffvfm\" (UID: \"43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1\") " pod="openshift-console/console-845b45fdb8-ffvfm" May 28 15:10:05.640369 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:05.640233 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1-console-serving-cert\") pod \"console-845b45fdb8-ffvfm\" (UID: \"43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1\") " pod="openshift-console/console-845b45fdb8-ffvfm" May 28 15:10:05.640369 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:05.640304 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1-service-ca\") pod \"console-845b45fdb8-ffvfm\" (UID: \"43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1\") " pod="openshift-console/console-845b45fdb8-ffvfm" May 28 15:10:05.640369 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:05.640343 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcqdw\" (UniqueName: \"kubernetes.io/projected/43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1-kube-api-access-zcqdw\") pod \"console-845b45fdb8-ffvfm\" (UID: \"43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1\") " pod="openshift-console/console-845b45fdb8-ffvfm" May 28 15:10:05.640530 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:05.640365 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1-console-config\") pod \"console-845b45fdb8-ffvfm\" (UID: \"43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1\") " pod="openshift-console/console-845b45fdb8-ffvfm" May 28 15:10:05.640530 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:05.640444 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1-console-oauth-config\") pod \"console-845b45fdb8-ffvfm\" (UID: \"43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1\") " pod="openshift-console/console-845b45fdb8-ffvfm" May 28 15:10:05.741777 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:05.741745 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zcqdw\" (UniqueName: \"kubernetes.io/projected/43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1-kube-api-access-zcqdw\") pod \"console-845b45fdb8-ffvfm\" (UID: \"43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1\") " pod="openshift-console/console-845b45fdb8-ffvfm" May 28 15:10:05.741932 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:05.741784 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1-console-config\") pod \"console-845b45fdb8-ffvfm\" (UID: \"43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1\") " pod="openshift-console/console-845b45fdb8-ffvfm" May 28 15:10:05.741932 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:05.741847 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1-console-oauth-config\") pod \"console-845b45fdb8-ffvfm\" (UID: \"43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1\") " pod="openshift-console/console-845b45fdb8-ffvfm" May 28 15:10:05.741932 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:05.741870 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1-oauth-serving-cert\") pod \"console-845b45fdb8-ffvfm\" (UID: \"43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1\") " pod="openshift-console/console-845b45fdb8-ffvfm" May 28 15:10:05.741932 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:05.741897 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1-console-serving-cert\") pod \"console-845b45fdb8-ffvfm\" (UID: \"43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1\") " pod="openshift-console/console-845b45fdb8-ffvfm" May 28 15:10:05.742116 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:05.741952 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1-service-ca\") pod \"console-845b45fdb8-ffvfm\" (UID: \"43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1\") " pod="openshift-console/console-845b45fdb8-ffvfm" May 28 15:10:05.742750 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:05.742686 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1-console-config\") pod \"console-845b45fdb8-ffvfm\" (UID: \"43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1\") " pod="openshift-console/console-845b45fdb8-ffvfm" May 28 15:10:05.742750 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:05.742714 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1-oauth-serving-cert\") pod \"console-845b45fdb8-ffvfm\" (UID: \"43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1\") " pod="openshift-console/console-845b45fdb8-ffvfm" May 28 15:10:05.742750 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:05.742733 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1-service-ca\") pod \"console-845b45fdb8-ffvfm\" (UID: \"43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1\") " pod="openshift-console/console-845b45fdb8-ffvfm" May 28 15:10:05.744617 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:05.744549 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1-console-oauth-config\") pod \"console-845b45fdb8-ffvfm\" (UID: \"43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1\") " pod="openshift-console/console-845b45fdb8-ffvfm" May 28 15:10:05.744758 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:05.744732 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1-console-serving-cert\") pod \"console-845b45fdb8-ffvfm\" (UID: \"43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1\") " pod="openshift-console/console-845b45fdb8-ffvfm" May 28 15:10:05.749765 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:05.749741 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcqdw\" (UniqueName: \"kubernetes.io/projected/43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1-kube-api-access-zcqdw\") pod \"console-845b45fdb8-ffvfm\" (UID: \"43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1\") " pod="openshift-console/console-845b45fdb8-ffvfm" May 28 15:10:05.859468 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:05.859392 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-845b45fdb8-ffvfm" May 28 15:10:06.007773 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:06.007734 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-845b45fdb8-ffvfm"] May 28 15:10:06.012127 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:10:06.012100 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43bd88c6_c6e6_49f4_8f8b_fbf05fba75c1.slice/crio-0c6485c71c83e012f34f9c2024d0a37965550894c31fc2e802f1253499ece682 WatchSource:0}: Error finding container 0c6485c71c83e012f34f9c2024d0a37965550894c31fc2e802f1253499ece682: Status 404 returned error can't find the container with id 0c6485c71c83e012f34f9c2024d0a37965550894c31fc2e802f1253499ece682 May 28 15:10:06.044410 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:06.044387 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b6714928-0dd7-4b82-b82c-9c83f3967f52-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-7zpd4\" (UID: \"b6714928-0dd7-4b82-b82c-9c83f3967f52\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-7zpd4" May 28 15:10:06.047008 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:06.046988 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b6714928-0dd7-4b82-b82c-9c83f3967f52-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-7zpd4\" (UID: \"b6714928-0dd7-4b82-b82c-9c83f3967f52\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-7zpd4" May 28 15:10:06.099328 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:06.099298 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-845b45fdb8-ffvfm" event={"ID":"43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1","Type":"ContainerStarted","Data":"0c6485c71c83e012f34f9c2024d0a37965550894c31fc2e802f1253499ece682"} May 28 15:10:06.312908 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:06.312875 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-7zpd4" May 28 15:10:06.439527 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:06.439461 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6c75959455-7zpd4"] May 28 15:10:06.443638 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:10:06.443608 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6714928_0dd7_4b82_b82c_9c83f3967f52.slice/crio-a574966552f19521eec507fb1b944298155c8353a56dae1cc62ff4cb04224e17 WatchSource:0}: Error finding container a574966552f19521eec507fb1b944298155c8353a56dae1cc62ff4cb04224e17: Status 404 returned error can't find the container with id a574966552f19521eec507fb1b944298155c8353a56dae1cc62ff4cb04224e17 May 28 15:10:07.103877 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:07.103835 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-7zpd4" event={"ID":"b6714928-0dd7-4b82-b82c-9c83f3967f52","Type":"ContainerStarted","Data":"a574966552f19521eec507fb1b944298155c8353a56dae1cc62ff4cb04224e17"} May 28 15:10:09.482789 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:10:09.482751 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" podUID="8959c2ac-69f0-4285-885b-1035cf57475e" May 28 15:10:09.501602 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:10:09.501567 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-9h2xl" podUID="8fa7b229-d678-41cd-b578-a31774414667" May 28 15:10:09.586704 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:10:09.586653 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-nvshr" podUID="59c5be81-b339-4a6d-9c00-f66c662406a3" May 28 15:10:09.717580 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:10:09.717502 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-l9p7b" podUID="e3a78917-de69-42d0-8fbe-85f346276bf9" May 28 15:10:09.949547 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:09.949521 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-2v4zh"] May 28 15:10:09.951565 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:09.951550 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-2v4zh" May 28 15:10:09.953753 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:09.953731 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" May 28 15:10:09.953861 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:09.953828 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-6fjvj\"" May 28 15:10:09.959419 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:09.959394 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-2v4zh"] May 28 15:10:10.080528 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:10.080463 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b906c71c-5fbb-4b0a-917c-8fc778742437-tls-certificates\") pod \"prometheus-operator-admission-webhook-6c4bbc67b7-2v4zh\" (UID: \"b906c71c-5fbb-4b0a-917c-8fc778742437\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-2v4zh" May 28 15:10:10.113441 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:10.113400 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-845b45fdb8-ffvfm" event={"ID":"43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1","Type":"ContainerStarted","Data":"0ac7feeb40689769d70aa152e9d3d776967a2bcc40dbe8020d06814cab62b9f9"} May 28 15:10:10.114849 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:10.114812 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-7zpd4" event={"ID":"b6714928-0dd7-4b82-b82c-9c83f3967f52","Type":"ContainerStarted","Data":"493eb640ce157b01684ee8e50373e212b5322a192fdec6275cab7f48b05ad2ce"} May 28 15:10:10.114849 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:10.114841 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nvshr" May 28 15:10:10.115063 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:10.115045 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" May 28 15:10:10.128804 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:10.128761 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-845b45fdb8-ffvfm" podStartSLOduration=1.7190623459999999 podStartE2EDuration="5.128750314s" podCreationTimestamp="2026-05-28 15:10:05 +0000 UTC" firstStartedPulling="2026-05-28 15:10:06.014529499 +0000 UTC m=+156.903516561" lastFinishedPulling="2026-05-28 15:10:09.424217464 +0000 UTC m=+160.313204529" observedRunningTime="2026-05-28 15:10:10.128387852 +0000 UTC m=+161.017374937" watchObservedRunningTime="2026-05-28 15:10:10.128750314 +0000 UTC m=+161.017737398" May 28 15:10:10.142786 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:10.142747 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-7zpd4" podStartSLOduration=33.165117419 podStartE2EDuration="36.14273306s" podCreationTimestamp="2026-05-28 15:09:34 +0000 UTC" firstStartedPulling="2026-05-28 15:10:06.445681399 +0000 UTC m=+157.334668481" lastFinishedPulling="2026-05-28 15:10:09.423297045 +0000 UTC m=+160.312284122" observedRunningTime="2026-05-28 15:10:10.141956165 +0000 UTC m=+161.030943250" watchObservedRunningTime="2026-05-28 15:10:10.14273306 +0000 UTC m=+161.031720145" May 28 15:10:10.182561 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:10.181950 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b906c71c-5fbb-4b0a-917c-8fc778742437-tls-certificates\") pod \"prometheus-operator-admission-webhook-6c4bbc67b7-2v4zh\" (UID: \"b906c71c-5fbb-4b0a-917c-8fc778742437\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-2v4zh" May 28 15:10:10.182561 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:10:10.182523 2559 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found May 28 15:10:10.182766 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:10:10.182653 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b906c71c-5fbb-4b0a-917c-8fc778742437-tls-certificates podName:b906c71c-5fbb-4b0a-917c-8fc778742437 nodeName:}" failed. No retries permitted until 2026-05-28 15:10:10.682635378 +0000 UTC m=+161.571622447 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/b906c71c-5fbb-4b0a-917c-8fc778742437-tls-certificates") pod "prometheus-operator-admission-webhook-6c4bbc67b7-2v4zh" (UID: "b906c71c-5fbb-4b0a-917c-8fc778742437") : secret "prometheus-operator-admission-webhook-tls" not found May 28 15:10:10.687547 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:10.687509 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b906c71c-5fbb-4b0a-917c-8fc778742437-tls-certificates\") pod \"prometheus-operator-admission-webhook-6c4bbc67b7-2v4zh\" (UID: \"b906c71c-5fbb-4b0a-917c-8fc778742437\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-2v4zh" May 28 15:10:10.690230 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:10.690204 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b906c71c-5fbb-4b0a-917c-8fc778742437-tls-certificates\") pod \"prometheus-operator-admission-webhook-6c4bbc67b7-2v4zh\" (UID: \"b906c71c-5fbb-4b0a-917c-8fc778742437\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-2v4zh" May 28 15:10:10.861997 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:10.861961 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-2v4zh" May 28 15:10:10.993144 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:10.993109 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-2v4zh"] May 28 15:10:11.118754 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:11.118708 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-2v4zh" event={"ID":"b906c71c-5fbb-4b0a-917c-8fc778742437","Type":"ContainerStarted","Data":"bdac0578c64a5f5660f9168d265c68404288ab628884787a16d9991ea3df80f2"} May 28 15:10:13.125060 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:13.125026 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-2v4zh" event={"ID":"b906c71c-5fbb-4b0a-917c-8fc778742437","Type":"ContainerStarted","Data":"a5b9723a235d02125aff720a93cafa2ef8ebf3ee3707a472381d44d9e53948e4"} May 28 15:10:13.125545 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:13.125151 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-2v4zh" May 28 15:10:13.130813 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:13.130784 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-2v4zh" May 28 15:10:13.140263 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:13.140228 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-2v4zh" podStartSLOduration=3.009900486 podStartE2EDuration="4.140218304s" podCreationTimestamp="2026-05-28 15:10:09 +0000 UTC" firstStartedPulling="2026-05-28 15:10:10.99786434 +0000 UTC m=+161.886851408" lastFinishedPulling="2026-05-28 15:10:12.12818215 +0000 UTC m=+163.017169226" observedRunningTime="2026-05-28 15:10:13.138823263 +0000 UTC m=+164.027810346" watchObservedRunningTime="2026-05-28 15:10:13.140218304 +0000 UTC m=+164.029205387" May 28 15:10:14.020712 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:14.020671 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7779bbdd78-9f8ct"] May 28 15:10:14.025427 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:14.025397 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7779bbdd78-9f8ct" May 28 15:10:14.029494 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:14.029340 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" May 28 15:10:14.029494 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:14.029340 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-7q6wz\"" May 28 15:10:14.029494 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:14.029375 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" May 28 15:10:14.029494 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:14.029358 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" May 28 15:10:14.033698 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:14.033237 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7779bbdd78-9f8ct"] May 28 15:10:14.118778 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:14.118746 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1b3396aa-17d4-49e3-9220-ec3bb17cf2a8-prometheus-operator-tls\") pod \"prometheus-operator-7779bbdd78-9f8ct\" (UID: \"1b3396aa-17d4-49e3-9220-ec3bb17cf2a8\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-9f8ct" May 28 15:10:14.118961 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:14.118785 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b3396aa-17d4-49e3-9220-ec3bb17cf2a8-metrics-client-ca\") pod \"prometheus-operator-7779bbdd78-9f8ct\" (UID: \"1b3396aa-17d4-49e3-9220-ec3bb17cf2a8\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-9f8ct" May 28 15:10:14.118961 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:14.118813 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1b3396aa-17d4-49e3-9220-ec3bb17cf2a8-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7779bbdd78-9f8ct\" (UID: \"1b3396aa-17d4-49e3-9220-ec3bb17cf2a8\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-9f8ct" May 28 15:10:14.118961 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:14.118911 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwppt\" (UniqueName: \"kubernetes.io/projected/1b3396aa-17d4-49e3-9220-ec3bb17cf2a8-kube-api-access-fwppt\") pod \"prometheus-operator-7779bbdd78-9f8ct\" (UID: \"1b3396aa-17d4-49e3-9220-ec3bb17cf2a8\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-9f8ct" May 28 15:10:14.220009 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:14.219964 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1b3396aa-17d4-49e3-9220-ec3bb17cf2a8-prometheus-operator-tls\") pod \"prometheus-operator-7779bbdd78-9f8ct\" (UID: \"1b3396aa-17d4-49e3-9220-ec3bb17cf2a8\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-9f8ct" May 28 15:10:14.220472 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:14.220017 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b3396aa-17d4-49e3-9220-ec3bb17cf2a8-metrics-client-ca\") pod \"prometheus-operator-7779bbdd78-9f8ct\" (UID: \"1b3396aa-17d4-49e3-9220-ec3bb17cf2a8\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-9f8ct" May 28 15:10:14.220472 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:14.220051 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1b3396aa-17d4-49e3-9220-ec3bb17cf2a8-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7779bbdd78-9f8ct\" (UID: \"1b3396aa-17d4-49e3-9220-ec3bb17cf2a8\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-9f8ct" May 28 15:10:14.220472 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:14.220101 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fwppt\" (UniqueName: \"kubernetes.io/projected/1b3396aa-17d4-49e3-9220-ec3bb17cf2a8-kube-api-access-fwppt\") pod \"prometheus-operator-7779bbdd78-9f8ct\" (UID: \"1b3396aa-17d4-49e3-9220-ec3bb17cf2a8\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-9f8ct" May 28 15:10:14.220472 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:10:14.220133 2559 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found May 28 15:10:14.220472 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:10:14.220207 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b3396aa-17d4-49e3-9220-ec3bb17cf2a8-prometheus-operator-tls podName:1b3396aa-17d4-49e3-9220-ec3bb17cf2a8 nodeName:}" failed. No retries permitted until 2026-05-28 15:10:14.720187014 +0000 UTC m=+165.609174098 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/1b3396aa-17d4-49e3-9220-ec3bb17cf2a8-prometheus-operator-tls") pod "prometheus-operator-7779bbdd78-9f8ct" (UID: "1b3396aa-17d4-49e3-9220-ec3bb17cf2a8") : secret "prometheus-operator-tls" not found May 28 15:10:14.220843 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:14.220818 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b3396aa-17d4-49e3-9220-ec3bb17cf2a8-metrics-client-ca\") pod \"prometheus-operator-7779bbdd78-9f8ct\" (UID: \"1b3396aa-17d4-49e3-9220-ec3bb17cf2a8\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-9f8ct" May 28 15:10:14.222750 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:14.222724 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1b3396aa-17d4-49e3-9220-ec3bb17cf2a8-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7779bbdd78-9f8ct\" (UID: \"1b3396aa-17d4-49e3-9220-ec3bb17cf2a8\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-9f8ct" May 28 15:10:14.228590 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:14.228567 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwppt\" (UniqueName: \"kubernetes.io/projected/1b3396aa-17d4-49e3-9220-ec3bb17cf2a8-kube-api-access-fwppt\") pod \"prometheus-operator-7779bbdd78-9f8ct\" (UID: \"1b3396aa-17d4-49e3-9220-ec3bb17cf2a8\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-9f8ct" May 28 15:10:14.421631 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:14.421549 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8959c2ac-69f0-4285-885b-1035cf57475e-registry-tls\") pod \"image-registry-5bd878d5c4-fx69p\" (UID: \"8959c2ac-69f0-4285-885b-1035cf57475e\") " pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" May 28 15:10:14.424235 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:14.424209 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8959c2ac-69f0-4285-885b-1035cf57475e-registry-tls\") pod \"image-registry-5bd878d5c4-fx69p\" (UID: \"8959c2ac-69f0-4285-885b-1035cf57475e\") " pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" May 28 15:10:14.522635 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:14.522602 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8fa7b229-d678-41cd-b578-a31774414667-cert\") pod \"ingress-canary-9h2xl\" (UID: \"8fa7b229-d678-41cd-b578-a31774414667\") " pod="openshift-ingress-canary/ingress-canary-9h2xl" May 28 15:10:14.525027 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:14.525003 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8fa7b229-d678-41cd-b578-a31774414667-cert\") pod \"ingress-canary-9h2xl\" (UID: \"8fa7b229-d678-41cd-b578-a31774414667\") " pod="openshift-ingress-canary/ingress-canary-9h2xl" May 28 15:10:14.618174 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:14.618147 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-7pvbk\"" May 28 15:10:14.623414 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:14.623388 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/59c5be81-b339-4a6d-9c00-f66c662406a3-metrics-tls\") pod \"dns-default-nvshr\" (UID: \"59c5be81-b339-4a6d-9c00-f66c662406a3\") " pod="openshift-dns/dns-default-nvshr" May 28 15:10:14.625753 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:14.625696 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" May 28 15:10:14.626017 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:14.625965 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/59c5be81-b339-4a6d-9c00-f66c662406a3-metrics-tls\") pod \"dns-default-nvshr\" (UID: \"59c5be81-b339-4a6d-9c00-f66c662406a3\") " pod="openshift-dns/dns-default-nvshr" May 28 15:10:14.724472 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:14.724387 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1b3396aa-17d4-49e3-9220-ec3bb17cf2a8-prometheus-operator-tls\") pod \"prometheus-operator-7779bbdd78-9f8ct\" (UID: \"1b3396aa-17d4-49e3-9220-ec3bb17cf2a8\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-9f8ct" May 28 15:10:14.726917 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:14.726894 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1b3396aa-17d4-49e3-9220-ec3bb17cf2a8-prometheus-operator-tls\") pod \"prometheus-operator-7779bbdd78-9f8ct\" (UID: \"1b3396aa-17d4-49e3-9220-ec3bb17cf2a8\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-9f8ct" May 28 15:10:14.918904 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:14.918870 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-b5hqh\"" May 28 15:10:14.926358 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:14.926327 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nvshr" May 28 15:10:14.936027 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:14.936003 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7779bbdd78-9f8ct" May 28 15:10:15.860092 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:15.860054 2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-845b45fdb8-ffvfm" May 28 15:10:15.860563 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:15.860356 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-845b45fdb8-ffvfm" May 28 15:10:15.861696 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:15.861670 2559 patch_prober.go:28] interesting pod/console-845b45fdb8-ffvfm container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.17:8443/health\": dial tcp 10.132.0.17:8443: connect: connection refused" start-of-body= May 28 15:10:15.861818 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:15.861718 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-845b45fdb8-ffvfm" podUID="43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1" containerName="console" probeResult="failure" output="Get \"https://10.132.0.17:8443/health\": dial tcp 10.132.0.17:8443: connect: connection refused" May 28 15:10:20.596901 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:20.596875 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nvshr"] May 28 15:10:20.607566 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:10:20.607538 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59c5be81_b339_4a6d_9c00_f66c662406a3.slice/crio-3ccb00ea4c118b39db581d32cc73fe708218608322d6bdc8f100daa80aad0349 WatchSource:0}: Error finding container 3ccb00ea4c118b39db581d32cc73fe708218608322d6bdc8f100daa80aad0349: Status 404 returned error can't find the container with id 3ccb00ea4c118b39db581d32cc73fe708218608322d6bdc8f100daa80aad0349 May 28 15:10:20.831785 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:20.831758 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5bd878d5c4-fx69p"] May 28 15:10:20.834015 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:20.833992 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7779bbdd78-9f8ct"] May 28 15:10:20.834806 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:10:20.834716 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8959c2ac_69f0_4285_885b_1035cf57475e.slice/crio-4eca11c6335def40d5e8295d8aa8241f795fe7cca109e7a6196e1165b624cb4d WatchSource:0}: Error finding container 4eca11c6335def40d5e8295d8aa8241f795fe7cca109e7a6196e1165b624cb4d: Status 404 returned error can't find the container with id 4eca11c6335def40d5e8295d8aa8241f795fe7cca109e7a6196e1165b624cb4d May 28 15:10:20.838145 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:10:20.837759 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b3396aa_17d4_49e3_9220_ec3bb17cf2a8.slice/crio-6f649e2b571fd2c8eb73d424dfbdd5cc31bf00c751e745ce3650e8f6341c51e6 WatchSource:0}: Error finding container 6f649e2b571fd2c8eb73d424dfbdd5cc31bf00c751e745ce3650e8f6341c51e6: Status 404 returned error can't find the container with id 6f649e2b571fd2c8eb73d424dfbdd5cc31bf00c751e745ce3650e8f6341c51e6 May 28 15:10:21.150895 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:21.149910 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" event={"ID":"8959c2ac-69f0-4285-885b-1035cf57475e","Type":"ContainerStarted","Data":"45c10a936b0bce931735a806f64bb467d650bd1965d0a5b3e1375721adb9e361"} May 28 15:10:21.150895 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:21.149959 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" event={"ID":"8959c2ac-69f0-4285-885b-1035cf57475e","Type":"ContainerStarted","Data":"4eca11c6335def40d5e8295d8aa8241f795fe7cca109e7a6196e1165b624cb4d"} May 28 15:10:21.150895 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:21.150788 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" May 28 15:10:21.152900 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:21.152860 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7779bbdd78-9f8ct" event={"ID":"1b3396aa-17d4-49e3-9220-ec3bb17cf2a8","Type":"ContainerStarted","Data":"6f649e2b571fd2c8eb73d424dfbdd5cc31bf00c751e745ce3650e8f6341c51e6"} May 28 15:10:21.154172 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:21.154118 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nvshr" event={"ID":"59c5be81-b339-4a6d-9c00-f66c662406a3","Type":"ContainerStarted","Data":"3ccb00ea4c118b39db581d32cc73fe708218608322d6bdc8f100daa80aad0349"} May 28 15:10:21.156786 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:21.156281 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-57bd96b888-76mzv" event={"ID":"0951dff9-48da-4274-8d7e-b47651895056","Type":"ContainerStarted","Data":"eb3525a1383f39bb6a12f227a9ef94f9cfc818b5bd8f584c12a141051e20d83e"} May 28 15:10:21.156786 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:21.156646 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-57bd96b888-76mzv" May 28 15:10:21.164463 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:21.164430 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-57bd96b888-76mzv" May 28 15:10:21.170666 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:21.168969 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" podStartSLOduration=171.168953503 podStartE2EDuration="2m51.168953503s" podCreationTimestamp="2026-05-28 15:07:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 15:10:21.168618094 +0000 UTC m=+172.057605181" watchObservedRunningTime="2026-05-28 15:10:21.168953503 +0000 UTC m=+172.057940589" May 28 15:10:22.708388 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:22.708316 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9h2xl" May 28 15:10:22.708956 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:22.708316 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9p7b" May 28 15:10:22.711740 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:22.711550 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-n5mwm\"" May 28 15:10:22.718924 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:22.718900 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9h2xl" May 28 15:10:23.389392 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:23.388899 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/downloads-57bd96b888-76mzv" podStartSLOduration=4.181781934 podStartE2EDuration="22.38888151s" podCreationTimestamp="2026-05-28 15:10:01 +0000 UTC" firstStartedPulling="2026-05-28 15:10:02.355464811 +0000 UTC m=+153.244451873" lastFinishedPulling="2026-05-28 15:10:20.562564381 +0000 UTC m=+171.451551449" observedRunningTime="2026-05-28 15:10:21.184575236 +0000 UTC m=+172.073562323" watchObservedRunningTime="2026-05-28 15:10:23.38888151 +0000 UTC m=+174.277868599" May 28 15:10:23.389721 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:23.389699 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9h2xl"] May 28 15:10:23.393262 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:10:23.393235 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fa7b229_d678_41cd_b578_a31774414667.slice/crio-32a88b29fbf97b2be050c3276965a3bb081aeb28132358d7b910582bfecb2799 WatchSource:0}: Error finding container 32a88b29fbf97b2be050c3276965a3bb081aeb28132358d7b910582bfecb2799: Status 404 returned error can't find the container with id 32a88b29fbf97b2be050c3276965a3bb081aeb28132358d7b910582bfecb2799 May 28 15:10:24.166519 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:24.166456 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9h2xl" event={"ID":"8fa7b229-d678-41cd-b578-a31774414667","Type":"ContainerStarted","Data":"32a88b29fbf97b2be050c3276965a3bb081aeb28132358d7b910582bfecb2799"} May 28 15:10:24.168744 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:24.168684 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7779bbdd78-9f8ct" event={"ID":"1b3396aa-17d4-49e3-9220-ec3bb17cf2a8","Type":"ContainerStarted","Data":"03ad1f233ae8c6c62b3056ca50eec0e3e78aa2cafe543c97b778cdfcc8def7fc"} May 28 15:10:24.168744 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:24.168724 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7779bbdd78-9f8ct" event={"ID":"1b3396aa-17d4-49e3-9220-ec3bb17cf2a8","Type":"ContainerStarted","Data":"b1cba6b3a45ec64fb51ee836e55f2cc1a7f2f8a573d7ddc4f9d69d094c6785f8"} May 28 15:10:24.170661 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:24.170636 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nvshr" event={"ID":"59c5be81-b339-4a6d-9c00-f66c662406a3","Type":"ContainerStarted","Data":"ce99d572c945b27385800da30bc181ee1c6465adbca9532e401e430bf66440a9"} May 28 15:10:24.170885 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:24.170666 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nvshr" event={"ID":"59c5be81-b339-4a6d-9c00-f66c662406a3","Type":"ContainerStarted","Data":"5bc2df370572e4d77cf2ee127c977a9de3e6ed3b7bed51c7584a73948f05d087"} May 28 15:10:24.170885 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:24.170789 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-nvshr" May 28 15:10:24.199865 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:24.199814 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7779bbdd78-9f8ct" podStartSLOduration=7.8082611790000005 podStartE2EDuration="10.199799559s" podCreationTimestamp="2026-05-28 15:10:14 +0000 UTC" firstStartedPulling="2026-05-28 15:10:20.8397039 +0000 UTC m=+171.728690975" lastFinishedPulling="2026-05-28 15:10:23.231242291 +0000 UTC m=+174.120229355" observedRunningTime="2026-05-28 15:10:24.197429542 +0000 UTC m=+175.086416625" watchObservedRunningTime="2026-05-28 15:10:24.199799559 +0000 UTC m=+175.088786652" May 28 15:10:25.860203 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:25.860162 2559 patch_prober.go:28] interesting pod/console-845b45fdb8-ffvfm container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.17:8443/health\": dial tcp 10.132.0.17:8443: connect: connection refused" start-of-body= May 28 15:10:25.860730 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:25.860240 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-845b45fdb8-ffvfm" podUID="43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1" containerName="console" probeResult="failure" output="Get \"https://10.132.0.17:8443/health\": dial tcp 10.132.0.17:8443: connect: connection refused" May 28 15:10:26.190955 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:26.190913 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9h2xl" event={"ID":"8fa7b229-d678-41cd-b578-a31774414667","Type":"ContainerStarted","Data":"e3b3cea9b847e9729502684081d979f9698e13a202afee2755564f3ec0d5e73a"} May 28 15:10:26.213429 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:26.213373 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-9h2xl" podStartSLOduration=137.949570355 podStartE2EDuration="2m20.213356766s" podCreationTimestamp="2026-05-28 15:08:06 +0000 UTC" firstStartedPulling="2026-05-28 15:10:23.395597719 +0000 UTC m=+174.284584793" lastFinishedPulling="2026-05-28 15:10:25.659384128 +0000 UTC m=+176.548371204" observedRunningTime="2026-05-28 15:10:26.212811068 +0000 UTC m=+177.101798155" watchObservedRunningTime="2026-05-28 15:10:26.213356766 +0000 UTC m=+177.102343868" May 28 15:10:26.213957 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:26.213928 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-nvshr" podStartSLOduration=137.586937839 podStartE2EDuration="2m20.213918412s" podCreationTimestamp="2026-05-28 15:08:06 +0000 UTC" firstStartedPulling="2026-05-28 15:10:20.609759789 +0000 UTC m=+171.498746851" lastFinishedPulling="2026-05-28 15:10:23.236740347 +0000 UTC m=+174.125727424" observedRunningTime="2026-05-28 15:10:24.216205879 +0000 UTC m=+175.105192963" watchObservedRunningTime="2026-05-28 15:10:26.213918412 +0000 UTC m=+177.102905496" May 28 15:10:26.414089 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:26.414054 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-455m8"] May 28 15:10:26.439407 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:26.439376 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-455m8" May 28 15:10:26.442170 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:26.442117 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" May 28 15:10:26.442291 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:26.442166 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" May 28 15:10:26.442291 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:26.442169 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-qhxdm\"" May 28 15:10:26.442393 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:26.442121 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" May 28 15:10:26.528578 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:26.528540 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1f665373-fcd1-43e4-a2a8-65c4a204d15b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-455m8\" (UID: \"1f665373-fcd1-43e4-a2a8-65c4a204d15b\") " pod="openshift-monitoring/node-exporter-455m8" May 28 15:10:26.528741 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:26.528589 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1f665373-fcd1-43e4-a2a8-65c4a204d15b-node-exporter-textfile\") pod \"node-exporter-455m8\" (UID: \"1f665373-fcd1-43e4-a2a8-65c4a204d15b\") " pod="openshift-monitoring/node-exporter-455m8" May 28 15:10:26.528741 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:26.528619 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1f665373-fcd1-43e4-a2a8-65c4a204d15b-node-exporter-tls\") pod \"node-exporter-455m8\" (UID: \"1f665373-fcd1-43e4-a2a8-65c4a204d15b\") " pod="openshift-monitoring/node-exporter-455m8" May 28 15:10:26.528741 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:26.528693 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1f665373-fcd1-43e4-a2a8-65c4a204d15b-sys\") pod \"node-exporter-455m8\" (UID: \"1f665373-fcd1-43e4-a2a8-65c4a204d15b\") " pod="openshift-monitoring/node-exporter-455m8" May 28 15:10:26.528741 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:26.528725 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f665373-fcd1-43e4-a2a8-65c4a204d15b-metrics-client-ca\") pod \"node-exporter-455m8\" (UID: \"1f665373-fcd1-43e4-a2a8-65c4a204d15b\") " pod="openshift-monitoring/node-exporter-455m8" May 28 15:10:26.528944 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:26.528764 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1f665373-fcd1-43e4-a2a8-65c4a204d15b-root\") pod \"node-exporter-455m8\" (UID: \"1f665373-fcd1-43e4-a2a8-65c4a204d15b\") " pod="openshift-monitoring/node-exporter-455m8" May 28 15:10:26.528944 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:26.528793 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1f665373-fcd1-43e4-a2a8-65c4a204d15b-node-exporter-wtmp\") pod \"node-exporter-455m8\" (UID: \"1f665373-fcd1-43e4-a2a8-65c4a204d15b\") " pod="openshift-monitoring/node-exporter-455m8" May 28 15:10:26.528944 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:26.528823 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1f665373-fcd1-43e4-a2a8-65c4a204d15b-node-exporter-accelerators-collector-config\") pod \"node-exporter-455m8\" (UID: \"1f665373-fcd1-43e4-a2a8-65c4a204d15b\") " pod="openshift-monitoring/node-exporter-455m8" May 28 15:10:26.528944 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:26.528895 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvf5v\" (UniqueName: \"kubernetes.io/projected/1f665373-fcd1-43e4-a2a8-65c4a204d15b-kube-api-access-cvf5v\") pod \"node-exporter-455m8\" (UID: \"1f665373-fcd1-43e4-a2a8-65c4a204d15b\") " pod="openshift-monitoring/node-exporter-455m8" May 28 15:10:26.630120 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:26.630081 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cvf5v\" (UniqueName: \"kubernetes.io/projected/1f665373-fcd1-43e4-a2a8-65c4a204d15b-kube-api-access-cvf5v\") pod \"node-exporter-455m8\" (UID: \"1f665373-fcd1-43e4-a2a8-65c4a204d15b\") " pod="openshift-monitoring/node-exporter-455m8" May 28 15:10:26.630307 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:26.630181 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1f665373-fcd1-43e4-a2a8-65c4a204d15b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-455m8\" (UID: \"1f665373-fcd1-43e4-a2a8-65c4a204d15b\") " pod="openshift-monitoring/node-exporter-455m8" May 28 15:10:26.630307 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:26.630210 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1f665373-fcd1-43e4-a2a8-65c4a204d15b-node-exporter-textfile\") pod \"node-exporter-455m8\" (UID: \"1f665373-fcd1-43e4-a2a8-65c4a204d15b\") " pod="openshift-monitoring/node-exporter-455m8" May 28 15:10:26.630307 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:26.630232 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1f665373-fcd1-43e4-a2a8-65c4a204d15b-node-exporter-tls\") pod \"node-exporter-455m8\" (UID: \"1f665373-fcd1-43e4-a2a8-65c4a204d15b\") " pod="openshift-monitoring/node-exporter-455m8" May 28 15:10:26.630496 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:26.630463 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1f665373-fcd1-43e4-a2a8-65c4a204d15b-sys\") pod \"node-exporter-455m8\" (UID: \"1f665373-fcd1-43e4-a2a8-65c4a204d15b\") " pod="openshift-monitoring/node-exporter-455m8" May 28 15:10:26.630561 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:26.630532 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f665373-fcd1-43e4-a2a8-65c4a204d15b-metrics-client-ca\") pod \"node-exporter-455m8\" (UID: \"1f665373-fcd1-43e4-a2a8-65c4a204d15b\") " pod="openshift-monitoring/node-exporter-455m8" May 28 15:10:26.630617 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:26.630555 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1f665373-fcd1-43e4-a2a8-65c4a204d15b-sys\") pod \"node-exporter-455m8\" (UID: \"1f665373-fcd1-43e4-a2a8-65c4a204d15b\") " pod="openshift-monitoring/node-exporter-455m8" May 28 15:10:26.630617 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:26.630579 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1f665373-fcd1-43e4-a2a8-65c4a204d15b-root\") pod \"node-exporter-455m8\" (UID: \"1f665373-fcd1-43e4-a2a8-65c4a204d15b\") " pod="openshift-monitoring/node-exporter-455m8" May 28 15:10:26.630617 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:26.630609 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1f665373-fcd1-43e4-a2a8-65c4a204d15b-node-exporter-wtmp\") pod \"node-exporter-455m8\" (UID: \"1f665373-fcd1-43e4-a2a8-65c4a204d15b\") " pod="openshift-monitoring/node-exporter-455m8" May 28 15:10:26.630769 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:26.630640 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1f665373-fcd1-43e4-a2a8-65c4a204d15b-node-exporter-accelerators-collector-config\") pod \"node-exporter-455m8\" (UID: \"1f665373-fcd1-43e4-a2a8-65c4a204d15b\") " pod="openshift-monitoring/node-exporter-455m8" May 28 15:10:26.630769 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:26.630673 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1f665373-fcd1-43e4-a2a8-65c4a204d15b-node-exporter-textfile\") pod \"node-exporter-455m8\" (UID: \"1f665373-fcd1-43e4-a2a8-65c4a204d15b\") " pod="openshift-monitoring/node-exporter-455m8" May 28 15:10:26.630769 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:26.630703 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1f665373-fcd1-43e4-a2a8-65c4a204d15b-root\") pod \"node-exporter-455m8\" (UID: \"1f665373-fcd1-43e4-a2a8-65c4a204d15b\") " pod="openshift-monitoring/node-exporter-455m8" May 28 15:10:26.630916 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:26.630830 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1f665373-fcd1-43e4-a2a8-65c4a204d15b-node-exporter-wtmp\") pod \"node-exporter-455m8\" (UID: \"1f665373-fcd1-43e4-a2a8-65c4a204d15b\") " pod="openshift-monitoring/node-exporter-455m8" May 28 15:10:26.631136 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:26.631114 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f665373-fcd1-43e4-a2a8-65c4a204d15b-metrics-client-ca\") pod \"node-exporter-455m8\" (UID: \"1f665373-fcd1-43e4-a2a8-65c4a204d15b\") " pod="openshift-monitoring/node-exporter-455m8" May 28 15:10:26.631254 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:26.631177 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1f665373-fcd1-43e4-a2a8-65c4a204d15b-node-exporter-accelerators-collector-config\") pod \"node-exporter-455m8\" (UID: \"1f665373-fcd1-43e4-a2a8-65c4a204d15b\") " pod="openshift-monitoring/node-exporter-455m8" May 28 15:10:26.633007 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:26.632986 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1f665373-fcd1-43e4-a2a8-65c4a204d15b-node-exporter-tls\") pod \"node-exporter-455m8\" (UID: \"1f665373-fcd1-43e4-a2a8-65c4a204d15b\") " pod="openshift-monitoring/node-exporter-455m8" May 28 15:10:26.633102 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:26.633044 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1f665373-fcd1-43e4-a2a8-65c4a204d15b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-455m8\" (UID: \"1f665373-fcd1-43e4-a2a8-65c4a204d15b\") " pod="openshift-monitoring/node-exporter-455m8" May 28 15:10:26.638392 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:26.638349 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvf5v\" (UniqueName: \"kubernetes.io/projected/1f665373-fcd1-43e4-a2a8-65c4a204d15b-kube-api-access-cvf5v\") pod \"node-exporter-455m8\" (UID: \"1f665373-fcd1-43e4-a2a8-65c4a204d15b\") " pod="openshift-monitoring/node-exporter-455m8" May 28 15:10:26.750931 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:26.750896 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-455m8" May 28 15:10:26.760885 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:10:26.760851 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f665373_fcd1_43e4_a2a8_65c4a204d15b.slice/crio-2269dfcb2db835c5162ffae4520e8ff66e060944b32eddc43a48fd14503620e7 WatchSource:0}: Error finding container 2269dfcb2db835c5162ffae4520e8ff66e060944b32eddc43a48fd14503620e7: Status 404 returned error can't find the container with id 2269dfcb2db835c5162ffae4520e8ff66e060944b32eddc43a48fd14503620e7 May 28 15:10:27.195624 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:27.195544 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-455m8" event={"ID":"1f665373-fcd1-43e4-a2a8-65c4a204d15b","Type":"ContainerStarted","Data":"2269dfcb2db835c5162ffae4520e8ff66e060944b32eddc43a48fd14503620e7"} May 28 15:10:28.356381 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:28.356347 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5479d49877-hxbf6"] May 28 15:10:28.374068 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:28.374045 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5479d49877-hxbf6"] May 28 15:10:28.374205 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:28.374192 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5479d49877-hxbf6" May 28 15:10:28.377710 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:28.377074 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" May 28 15:10:28.377710 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:28.377105 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-tnq06gsl7ff\"" May 28 15:10:28.377710 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:28.377333 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" May 28 15:10:28.377710 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:28.377357 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-9f6jl\"" May 28 15:10:28.377710 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:28.377591 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" May 28 15:10:28.377710 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:28.377667 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" May 28 15:10:28.378102 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:28.377763 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" May 28 15:10:28.547827 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:28.547786 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0c588793-be04-492c-b10a-391b5bd3686b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5479d49877-hxbf6\" (UID: \"0c588793-be04-492c-b10a-391b5bd3686b\") " pod="openshift-monitoring/thanos-querier-5479d49877-hxbf6" May 28 15:10:28.547985 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:28.547837 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0c588793-be04-492c-b10a-391b5bd3686b-metrics-client-ca\") pod \"thanos-querier-5479d49877-hxbf6\" (UID: \"0c588793-be04-492c-b10a-391b5bd3686b\") " pod="openshift-monitoring/thanos-querier-5479d49877-hxbf6" May 28 15:10:28.547985 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:28.547868 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0c588793-be04-492c-b10a-391b5bd3686b-secret-thanos-querier-tls\") pod \"thanos-querier-5479d49877-hxbf6\" (UID: \"0c588793-be04-492c-b10a-391b5bd3686b\") " pod="openshift-monitoring/thanos-querier-5479d49877-hxbf6" May 28 15:10:28.547985 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:28.547897 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0c588793-be04-492c-b10a-391b5bd3686b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5479d49877-hxbf6\" (UID: \"0c588793-be04-492c-b10a-391b5bd3686b\") " pod="openshift-monitoring/thanos-querier-5479d49877-hxbf6" May 28 15:10:28.547985 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:28.547920 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0c588793-be04-492c-b10a-391b5bd3686b-secret-grpc-tls\") pod \"thanos-querier-5479d49877-hxbf6\" (UID: \"0c588793-be04-492c-b10a-391b5bd3686b\") " pod="openshift-monitoring/thanos-querier-5479d49877-hxbf6" May 28 15:10:28.548190 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:28.548022 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0c588793-be04-492c-b10a-391b5bd3686b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5479d49877-hxbf6\" (UID: \"0c588793-be04-492c-b10a-391b5bd3686b\") " pod="openshift-monitoring/thanos-querier-5479d49877-hxbf6" May 28 15:10:28.548190 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:28.548078 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0c588793-be04-492c-b10a-391b5bd3686b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5479d49877-hxbf6\" (UID: \"0c588793-be04-492c-b10a-391b5bd3686b\") " pod="openshift-monitoring/thanos-querier-5479d49877-hxbf6" May 28 15:10:28.548190 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:28.548131 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hw2xx\" (UniqueName: \"kubernetes.io/projected/0c588793-be04-492c-b10a-391b5bd3686b-kube-api-access-hw2xx\") pod \"thanos-querier-5479d49877-hxbf6\" (UID: \"0c588793-be04-492c-b10a-391b5bd3686b\") " pod="openshift-monitoring/thanos-querier-5479d49877-hxbf6" May 28 15:10:28.649295 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:28.649260 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0c588793-be04-492c-b10a-391b5bd3686b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5479d49877-hxbf6\" (UID: \"0c588793-be04-492c-b10a-391b5bd3686b\") " pod="openshift-monitoring/thanos-querier-5479d49877-hxbf6" May 28 15:10:28.649438 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:28.649318 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0c588793-be04-492c-b10a-391b5bd3686b-metrics-client-ca\") pod \"thanos-querier-5479d49877-hxbf6\" (UID: \"0c588793-be04-492c-b10a-391b5bd3686b\") " pod="openshift-monitoring/thanos-querier-5479d49877-hxbf6" May 28 15:10:28.649438 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:28.649349 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0c588793-be04-492c-b10a-391b5bd3686b-secret-thanos-querier-tls\") pod \"thanos-querier-5479d49877-hxbf6\" (UID: \"0c588793-be04-492c-b10a-391b5bd3686b\") " pod="openshift-monitoring/thanos-querier-5479d49877-hxbf6" May 28 15:10:28.649438 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:28.649372 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0c588793-be04-492c-b10a-391b5bd3686b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5479d49877-hxbf6\" (UID: \"0c588793-be04-492c-b10a-391b5bd3686b\") " pod="openshift-monitoring/thanos-querier-5479d49877-hxbf6" May 28 15:10:28.649438 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:28.649396 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0c588793-be04-492c-b10a-391b5bd3686b-secret-grpc-tls\") pod \"thanos-querier-5479d49877-hxbf6\" (UID: \"0c588793-be04-492c-b10a-391b5bd3686b\") " pod="openshift-monitoring/thanos-querier-5479d49877-hxbf6" May 28 15:10:28.649680 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:28.649449 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0c588793-be04-492c-b10a-391b5bd3686b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5479d49877-hxbf6\" (UID: \"0c588793-be04-492c-b10a-391b5bd3686b\") " pod="openshift-monitoring/thanos-querier-5479d49877-hxbf6" May 28 15:10:28.649680 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:28.649537 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0c588793-be04-492c-b10a-391b5bd3686b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5479d49877-hxbf6\" (UID: \"0c588793-be04-492c-b10a-391b5bd3686b\") " pod="openshift-monitoring/thanos-querier-5479d49877-hxbf6" May 28 15:10:28.649680 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:28.649583 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hw2xx\" (UniqueName: \"kubernetes.io/projected/0c588793-be04-492c-b10a-391b5bd3686b-kube-api-access-hw2xx\") pod \"thanos-querier-5479d49877-hxbf6\" (UID: \"0c588793-be04-492c-b10a-391b5bd3686b\") " pod="openshift-monitoring/thanos-querier-5479d49877-hxbf6" May 28 15:10:28.650147 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:28.650092 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0c588793-be04-492c-b10a-391b5bd3686b-metrics-client-ca\") pod \"thanos-querier-5479d49877-hxbf6\" (UID: \"0c588793-be04-492c-b10a-391b5bd3686b\") " pod="openshift-monitoring/thanos-querier-5479d49877-hxbf6" May 28 15:10:28.652557 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:28.652527 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0c588793-be04-492c-b10a-391b5bd3686b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5479d49877-hxbf6\" (UID: \"0c588793-be04-492c-b10a-391b5bd3686b\") " pod="openshift-monitoring/thanos-querier-5479d49877-hxbf6" May 28 15:10:28.652907 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:28.652883 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0c588793-be04-492c-b10a-391b5bd3686b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5479d49877-hxbf6\" (UID: \"0c588793-be04-492c-b10a-391b5bd3686b\") " pod="openshift-monitoring/thanos-querier-5479d49877-hxbf6" May 28 15:10:28.653001 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:28.652923 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0c588793-be04-492c-b10a-391b5bd3686b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5479d49877-hxbf6\" (UID: \"0c588793-be04-492c-b10a-391b5bd3686b\") " pod="openshift-monitoring/thanos-querier-5479d49877-hxbf6" May 28 15:10:28.653062 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:28.652998 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0c588793-be04-492c-b10a-391b5bd3686b-secret-thanos-querier-tls\") pod \"thanos-querier-5479d49877-hxbf6\" (UID: \"0c588793-be04-492c-b10a-391b5bd3686b\") " pod="openshift-monitoring/thanos-querier-5479d49877-hxbf6" May 28 15:10:28.653229 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:28.653211 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0c588793-be04-492c-b10a-391b5bd3686b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5479d49877-hxbf6\" (UID: \"0c588793-be04-492c-b10a-391b5bd3686b\") " pod="openshift-monitoring/thanos-querier-5479d49877-hxbf6" May 28 15:10:28.653993 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:28.653971 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0c588793-be04-492c-b10a-391b5bd3686b-secret-grpc-tls\") pod \"thanos-querier-5479d49877-hxbf6\" (UID: \"0c588793-be04-492c-b10a-391b5bd3686b\") " pod="openshift-monitoring/thanos-querier-5479d49877-hxbf6" May 28 15:10:28.658470 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:28.658446 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hw2xx\" (UniqueName: \"kubernetes.io/projected/0c588793-be04-492c-b10a-391b5bd3686b-kube-api-access-hw2xx\") pod \"thanos-querier-5479d49877-hxbf6\" (UID: \"0c588793-be04-492c-b10a-391b5bd3686b\") " pod="openshift-monitoring/thanos-querier-5479d49877-hxbf6" May 28 15:10:28.685943 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:28.685911 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5479d49877-hxbf6" May 28 15:10:28.835158 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:28.835126 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5479d49877-hxbf6"] May 28 15:10:28.838983 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:10:28.838941 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c588793_be04_492c_b10a_391b5bd3686b.slice/crio-d0d6a361e227c74c43437bf752b46f36e5c6b23799b0c5a3d23e1081535876e7 WatchSource:0}: Error finding container d0d6a361e227c74c43437bf752b46f36e5c6b23799b0c5a3d23e1081535876e7: Status 404 returned error can't find the container with id d0d6a361e227c74c43437bf752b46f36e5c6b23799b0c5a3d23e1081535876e7 May 28 15:10:29.202457 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:29.202418 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5479d49877-hxbf6" event={"ID":"0c588793-be04-492c-b10a-391b5bd3686b","Type":"ContainerStarted","Data":"d0d6a361e227c74c43437bf752b46f36e5c6b23799b0c5a3d23e1081535876e7"} May 28 15:10:29.204064 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:29.204031 2559 generic.go:358] "Generic (PLEG): container finished" podID="1f665373-fcd1-43e4-a2a8-65c4a204d15b" containerID="788ff49be342c051b3f5c032a90ffc77e345e075c4e702ad34d5c9481803caa0" exitCode=0 May 28 15:10:29.204194 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:29.204106 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-455m8" event={"ID":"1f665373-fcd1-43e4-a2a8-65c4a204d15b","Type":"ContainerDied","Data":"788ff49be342c051b3f5c032a90ffc77e345e075c4e702ad34d5c9481803caa0"} May 28 15:10:30.208296 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:30.208242 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-455m8" event={"ID":"1f665373-fcd1-43e4-a2a8-65c4a204d15b","Type":"ContainerStarted","Data":"74576aa91678715e1b3cdb56aeb95fb9b95ccace881b6ab5ded78c4f082ab46e"} May 28 15:10:30.208296 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:30.208297 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-455m8" event={"ID":"1f665373-fcd1-43e4-a2a8-65c4a204d15b","Type":"ContainerStarted","Data":"17a999b1597406427c7e0b44d802f5f9bcc3747bd55a79e1b7b6e6045f9bbe7c"} May 28 15:10:30.231533 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:30.231470 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-455m8" podStartSLOduration=2.629783712 podStartE2EDuration="4.231458003s" podCreationTimestamp="2026-05-28 15:10:26 +0000 UTC" firstStartedPulling="2026-05-28 15:10:26.762545604 +0000 UTC m=+177.651532680" lastFinishedPulling="2026-05-28 15:10:28.364219893 +0000 UTC m=+179.253206971" observedRunningTime="2026-05-28 15:10:30.22990344 +0000 UTC m=+181.118890523" watchObservedRunningTime="2026-05-28 15:10:30.231458003 +0000 UTC m=+181.120445115" May 28 15:10:30.772159 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:30.772119 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7844b88b7-gjcqf"] May 28 15:10:30.797604 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:30.797577 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7844b88b7-gjcqf"] May 28 15:10:30.797762 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:30.797703 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7844b88b7-gjcqf" May 28 15:10:30.800473 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:30.800445 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" May 28 15:10:30.800624 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:30.800538 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" May 28 15:10:30.800624 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:30.800544 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" May 28 15:10:30.800756 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:30.800671 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" May 28 15:10:30.801757 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:30.801720 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-8zqvk\"" May 28 15:10:30.801861 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:30.801804 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-ejngta1qfr4dn\"" May 28 15:10:30.970208 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:30.970170 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/6db25964-8104-4046-bfe7-a17f56d28a66-audit-log\") pod \"metrics-server-7844b88b7-gjcqf\" (UID: \"6db25964-8104-4046-bfe7-a17f56d28a66\") " pod="openshift-monitoring/metrics-server-7844b88b7-gjcqf" May 28 15:10:30.970396 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:30.970218 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6db25964-8104-4046-bfe7-a17f56d28a66-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7844b88b7-gjcqf\" (UID: \"6db25964-8104-4046-bfe7-a17f56d28a66\") " pod="openshift-monitoring/metrics-server-7844b88b7-gjcqf" May 28 15:10:30.970396 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:30.970334 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/6db25964-8104-4046-bfe7-a17f56d28a66-secret-metrics-server-client-certs\") pod \"metrics-server-7844b88b7-gjcqf\" (UID: \"6db25964-8104-4046-bfe7-a17f56d28a66\") " pod="openshift-monitoring/metrics-server-7844b88b7-gjcqf" May 28 15:10:30.970396 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:30.970373 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/6db25964-8104-4046-bfe7-a17f56d28a66-metrics-server-audit-profiles\") pod \"metrics-server-7844b88b7-gjcqf\" (UID: \"6db25964-8104-4046-bfe7-a17f56d28a66\") " pod="openshift-monitoring/metrics-server-7844b88b7-gjcqf" May 28 15:10:30.970586 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:30.970474 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6db25964-8104-4046-bfe7-a17f56d28a66-client-ca-bundle\") pod \"metrics-server-7844b88b7-gjcqf\" (UID: \"6db25964-8104-4046-bfe7-a17f56d28a66\") " pod="openshift-monitoring/metrics-server-7844b88b7-gjcqf" May 28 15:10:30.970586 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:30.970537 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/6db25964-8104-4046-bfe7-a17f56d28a66-secret-metrics-server-tls\") pod \"metrics-server-7844b88b7-gjcqf\" (UID: \"6db25964-8104-4046-bfe7-a17f56d28a66\") " pod="openshift-monitoring/metrics-server-7844b88b7-gjcqf" May 28 15:10:30.970677 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:30.970583 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24xr7\" (UniqueName: \"kubernetes.io/projected/6db25964-8104-4046-bfe7-a17f56d28a66-kube-api-access-24xr7\") pod \"metrics-server-7844b88b7-gjcqf\" (UID: \"6db25964-8104-4046-bfe7-a17f56d28a66\") " pod="openshift-monitoring/metrics-server-7844b88b7-gjcqf" May 28 15:10:31.071871 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.071804 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/6db25964-8104-4046-bfe7-a17f56d28a66-secret-metrics-server-client-certs\") pod \"metrics-server-7844b88b7-gjcqf\" (UID: \"6db25964-8104-4046-bfe7-a17f56d28a66\") " pod="openshift-monitoring/metrics-server-7844b88b7-gjcqf" May 28 15:10:31.071871 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.071839 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/6db25964-8104-4046-bfe7-a17f56d28a66-metrics-server-audit-profiles\") pod \"metrics-server-7844b88b7-gjcqf\" (UID: \"6db25964-8104-4046-bfe7-a17f56d28a66\") " pod="openshift-monitoring/metrics-server-7844b88b7-gjcqf" May 28 15:10:31.072046 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.071924 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6db25964-8104-4046-bfe7-a17f56d28a66-client-ca-bundle\") pod \"metrics-server-7844b88b7-gjcqf\" (UID: \"6db25964-8104-4046-bfe7-a17f56d28a66\") " pod="openshift-monitoring/metrics-server-7844b88b7-gjcqf" May 28 15:10:31.072046 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.071964 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/6db25964-8104-4046-bfe7-a17f56d28a66-secret-metrics-server-tls\") pod \"metrics-server-7844b88b7-gjcqf\" (UID: \"6db25964-8104-4046-bfe7-a17f56d28a66\") " pod="openshift-monitoring/metrics-server-7844b88b7-gjcqf" May 28 15:10:31.072156 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.072100 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-24xr7\" (UniqueName: \"kubernetes.io/projected/6db25964-8104-4046-bfe7-a17f56d28a66-kube-api-access-24xr7\") pod \"metrics-server-7844b88b7-gjcqf\" (UID: \"6db25964-8104-4046-bfe7-a17f56d28a66\") " pod="openshift-monitoring/metrics-server-7844b88b7-gjcqf" May 28 15:10:31.072209 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.072153 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/6db25964-8104-4046-bfe7-a17f56d28a66-audit-log\") pod \"metrics-server-7844b88b7-gjcqf\" (UID: \"6db25964-8104-4046-bfe7-a17f56d28a66\") " pod="openshift-monitoring/metrics-server-7844b88b7-gjcqf" May 28 15:10:31.072209 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.072183 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6db25964-8104-4046-bfe7-a17f56d28a66-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7844b88b7-gjcqf\" (UID: \"6db25964-8104-4046-bfe7-a17f56d28a66\") " pod="openshift-monitoring/metrics-server-7844b88b7-gjcqf" May 28 15:10:31.072707 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.072682 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/6db25964-8104-4046-bfe7-a17f56d28a66-audit-log\") pod \"metrics-server-7844b88b7-gjcqf\" (UID: \"6db25964-8104-4046-bfe7-a17f56d28a66\") " pod="openshift-monitoring/metrics-server-7844b88b7-gjcqf" May 28 15:10:31.072931 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.072910 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6db25964-8104-4046-bfe7-a17f56d28a66-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7844b88b7-gjcqf\" (UID: \"6db25964-8104-4046-bfe7-a17f56d28a66\") " pod="openshift-monitoring/metrics-server-7844b88b7-gjcqf" May 28 15:10:31.073051 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.073028 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/6db25964-8104-4046-bfe7-a17f56d28a66-metrics-server-audit-profiles\") pod \"metrics-server-7844b88b7-gjcqf\" (UID: \"6db25964-8104-4046-bfe7-a17f56d28a66\") " pod="openshift-monitoring/metrics-server-7844b88b7-gjcqf" May 28 15:10:31.074366 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.074346 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6db25964-8104-4046-bfe7-a17f56d28a66-client-ca-bundle\") pod \"metrics-server-7844b88b7-gjcqf\" (UID: \"6db25964-8104-4046-bfe7-a17f56d28a66\") " pod="openshift-monitoring/metrics-server-7844b88b7-gjcqf" May 28 15:10:31.074504 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.074415 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/6db25964-8104-4046-bfe7-a17f56d28a66-secret-metrics-server-client-certs\") pod \"metrics-server-7844b88b7-gjcqf\" (UID: \"6db25964-8104-4046-bfe7-a17f56d28a66\") " pod="openshift-monitoring/metrics-server-7844b88b7-gjcqf" May 28 15:10:31.074504 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.074472 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/6db25964-8104-4046-bfe7-a17f56d28a66-secret-metrics-server-tls\") pod \"metrics-server-7844b88b7-gjcqf\" (UID: \"6db25964-8104-4046-bfe7-a17f56d28a66\") " pod="openshift-monitoring/metrics-server-7844b88b7-gjcqf" May 28 15:10:31.080872 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.080836 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-24xr7\" (UniqueName: \"kubernetes.io/projected/6db25964-8104-4046-bfe7-a17f56d28a66-kube-api-access-24xr7\") pod \"metrics-server-7844b88b7-gjcqf\" (UID: \"6db25964-8104-4046-bfe7-a17f56d28a66\") " pod="openshift-monitoring/metrics-server-7844b88b7-gjcqf" May 28 15:10:31.109766 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.109728 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7844b88b7-gjcqf" May 28 15:10:31.153763 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.153732 2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-845b45fdb8-ffvfm"] May 28 15:10:31.190252 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.190219 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f45f64f46-k67zx"] May 28 15:10:31.203251 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.203220 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f45f64f46-k67zx"] May 28 15:10:31.203384 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.203259 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f45f64f46-k67zx" May 28 15:10:31.211035 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.210845 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" May 28 15:10:31.247574 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.247524 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7844b88b7-gjcqf"] May 28 15:10:31.375008 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.374934 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e7660a27-79df-4fa5-8785-2d9ceead8c61-oauth-serving-cert\") pod \"console-6f45f64f46-k67zx\" (UID: \"e7660a27-79df-4fa5-8785-2d9ceead8c61\") " pod="openshift-console/console-6f45f64f46-k67zx" May 28 15:10:31.375008 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.374982 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e7660a27-79df-4fa5-8785-2d9ceead8c61-console-oauth-config\") pod \"console-6f45f64f46-k67zx\" (UID: \"e7660a27-79df-4fa5-8785-2d9ceead8c61\") " pod="openshift-console/console-6f45f64f46-k67zx" May 28 15:10:31.375218 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.375026 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e7660a27-79df-4fa5-8785-2d9ceead8c61-service-ca\") pod \"console-6f45f64f46-k67zx\" (UID: \"e7660a27-79df-4fa5-8785-2d9ceead8c61\") " pod="openshift-console/console-6f45f64f46-k67zx" May 28 15:10:31.375218 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.375100 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e7660a27-79df-4fa5-8785-2d9ceead8c61-console-serving-cert\") pod \"console-6f45f64f46-k67zx\" (UID: \"e7660a27-79df-4fa5-8785-2d9ceead8c61\") " pod="openshift-console/console-6f45f64f46-k67zx" May 28 15:10:31.375218 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.375199 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e7660a27-79df-4fa5-8785-2d9ceead8c61-console-config\") pod \"console-6f45f64f46-k67zx\" (UID: \"e7660a27-79df-4fa5-8785-2d9ceead8c61\") " pod="openshift-console/console-6f45f64f46-k67zx" May 28 15:10:31.375431 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.375263 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4t9gl\" (UniqueName: \"kubernetes.io/projected/e7660a27-79df-4fa5-8785-2d9ceead8c61-kube-api-access-4t9gl\") pod \"console-6f45f64f46-k67zx\" (UID: \"e7660a27-79df-4fa5-8785-2d9ceead8c61\") " pod="openshift-console/console-6f45f64f46-k67zx" May 28 15:10:31.375431 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.375370 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7660a27-79df-4fa5-8785-2d9ceead8c61-trusted-ca-bundle\") pod \"console-6f45f64f46-k67zx\" (UID: \"e7660a27-79df-4fa5-8785-2d9ceead8c61\") " pod="openshift-console/console-6f45f64f46-k67zx" May 28 15:10:31.476652 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.476630 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7660a27-79df-4fa5-8785-2d9ceead8c61-trusted-ca-bundle\") pod \"console-6f45f64f46-k67zx\" (UID: \"e7660a27-79df-4fa5-8785-2d9ceead8c61\") " pod="openshift-console/console-6f45f64f46-k67zx" May 28 15:10:31.476739 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.476662 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e7660a27-79df-4fa5-8785-2d9ceead8c61-oauth-serving-cert\") pod \"console-6f45f64f46-k67zx\" (UID: \"e7660a27-79df-4fa5-8785-2d9ceead8c61\") " pod="openshift-console/console-6f45f64f46-k67zx" May 28 15:10:31.476739 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.476715 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e7660a27-79df-4fa5-8785-2d9ceead8c61-console-oauth-config\") pod \"console-6f45f64f46-k67zx\" (UID: \"e7660a27-79df-4fa5-8785-2d9ceead8c61\") " pod="openshift-console/console-6f45f64f46-k67zx" May 28 15:10:31.476903 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.476740 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e7660a27-79df-4fa5-8785-2d9ceead8c61-service-ca\") pod \"console-6f45f64f46-k67zx\" (UID: \"e7660a27-79df-4fa5-8785-2d9ceead8c61\") " pod="openshift-console/console-6f45f64f46-k67zx" May 28 15:10:31.476957 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.476936 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e7660a27-79df-4fa5-8785-2d9ceead8c61-console-serving-cert\") pod \"console-6f45f64f46-k67zx\" (UID: \"e7660a27-79df-4fa5-8785-2d9ceead8c61\") " pod="openshift-console/console-6f45f64f46-k67zx" May 28 15:10:31.477013 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.476976 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e7660a27-79df-4fa5-8785-2d9ceead8c61-console-config\") pod \"console-6f45f64f46-k67zx\" (UID: \"e7660a27-79df-4fa5-8785-2d9ceead8c61\") " pod="openshift-console/console-6f45f64f46-k67zx" May 28 15:10:31.477070 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.477013 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4t9gl\" (UniqueName: \"kubernetes.io/projected/e7660a27-79df-4fa5-8785-2d9ceead8c61-kube-api-access-4t9gl\") pod \"console-6f45f64f46-k67zx\" (UID: \"e7660a27-79df-4fa5-8785-2d9ceead8c61\") " pod="openshift-console/console-6f45f64f46-k67zx" May 28 15:10:31.477675 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.477649 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e7660a27-79df-4fa5-8785-2d9ceead8c61-service-ca\") pod \"console-6f45f64f46-k67zx\" (UID: \"e7660a27-79df-4fa5-8785-2d9ceead8c61\") " pod="openshift-console/console-6f45f64f46-k67zx" May 28 15:10:31.477774 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.477759 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7660a27-79df-4fa5-8785-2d9ceead8c61-trusted-ca-bundle\") pod \"console-6f45f64f46-k67zx\" (UID: \"e7660a27-79df-4fa5-8785-2d9ceead8c61\") " pod="openshift-console/console-6f45f64f46-k67zx" May 28 15:10:31.477878 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.477862 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e7660a27-79df-4fa5-8785-2d9ceead8c61-console-config\") pod \"console-6f45f64f46-k67zx\" (UID: \"e7660a27-79df-4fa5-8785-2d9ceead8c61\") " pod="openshift-console/console-6f45f64f46-k67zx" May 28 15:10:31.477939 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.477919 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e7660a27-79df-4fa5-8785-2d9ceead8c61-oauth-serving-cert\") pod \"console-6f45f64f46-k67zx\" (UID: \"e7660a27-79df-4fa5-8785-2d9ceead8c61\") " pod="openshift-console/console-6f45f64f46-k67zx" May 28 15:10:31.479155 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.479136 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e7660a27-79df-4fa5-8785-2d9ceead8c61-console-oauth-config\") pod \"console-6f45f64f46-k67zx\" (UID: \"e7660a27-79df-4fa5-8785-2d9ceead8c61\") " pod="openshift-console/console-6f45f64f46-k67zx" May 28 15:10:31.479535 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.479511 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e7660a27-79df-4fa5-8785-2d9ceead8c61-console-serving-cert\") pod \"console-6f45f64f46-k67zx\" (UID: \"e7660a27-79df-4fa5-8785-2d9ceead8c61\") " pod="openshift-console/console-6f45f64f46-k67zx" May 28 15:10:31.484173 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.484152 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t9gl\" (UniqueName: \"kubernetes.io/projected/e7660a27-79df-4fa5-8785-2d9ceead8c61-kube-api-access-4t9gl\") pod \"console-6f45f64f46-k67zx\" (UID: \"e7660a27-79df-4fa5-8785-2d9ceead8c61\") " pod="openshift-console/console-6f45f64f46-k67zx" May 28 15:10:31.513960 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.513939 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f45f64f46-k67zx" May 28 15:10:31.586941 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.586788 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-9f5b64bf-mnlzs"] May 28 15:10:31.599832 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.599804 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-9f5b64bf-mnlzs" May 28 15:10:31.603050 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.603019 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" May 28 15:10:31.603167 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.603094 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" May 28 15:10:31.603736 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.603713 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" May 28 15:10:31.603846 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.603708 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" May 28 15:10:31.604257 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.604238 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" May 28 15:10:31.604459 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.604422 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-9rww8\"" May 28 15:10:31.609872 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.609842 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" May 28 15:10:31.612645 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.612619 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-9f5b64bf-mnlzs"] May 28 15:10:31.654860 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.654836 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f45f64f46-k67zx"] May 28 15:10:31.657213 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:10:31.657170 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7660a27_79df_4fa5_8785_2d9ceead8c61.slice/crio-25973ad6e340e8bffe03206bee103086bc7e19e4b95db4994577fffb5a7c1807 WatchSource:0}: Error finding container 25973ad6e340e8bffe03206bee103086bc7e19e4b95db4994577fffb5a7c1807: Status 404 returned error can't find the container with id 25973ad6e340e8bffe03206bee103086bc7e19e4b95db4994577fffb5a7c1807 May 28 15:10:31.779561 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.779508 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59716931-45d5-4ad2-8fea-fc240f8155ac-serving-certs-ca-bundle\") pod \"telemeter-client-9f5b64bf-mnlzs\" (UID: \"59716931-45d5-4ad2-8fea-fc240f8155ac\") " pod="openshift-monitoring/telemeter-client-9f5b64bf-mnlzs" May 28 15:10:31.779561 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.779551 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/59716931-45d5-4ad2-8fea-fc240f8155ac-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-9f5b64bf-mnlzs\" (UID: \"59716931-45d5-4ad2-8fea-fc240f8155ac\") " pod="openshift-monitoring/telemeter-client-9f5b64bf-mnlzs" May 28 15:10:31.779758 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.779594 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/59716931-45d5-4ad2-8fea-fc240f8155ac-secret-telemeter-client\") pod \"telemeter-client-9f5b64bf-mnlzs\" (UID: \"59716931-45d5-4ad2-8fea-fc240f8155ac\") " pod="openshift-monitoring/telemeter-client-9f5b64bf-mnlzs" May 28 15:10:31.779758 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.779623 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/59716931-45d5-4ad2-8fea-fc240f8155ac-metrics-client-ca\") pod \"telemeter-client-9f5b64bf-mnlzs\" (UID: \"59716931-45d5-4ad2-8fea-fc240f8155ac\") " pod="openshift-monitoring/telemeter-client-9f5b64bf-mnlzs" May 28 15:10:31.779758 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.779638 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59716931-45d5-4ad2-8fea-fc240f8155ac-telemeter-trusted-ca-bundle\") pod \"telemeter-client-9f5b64bf-mnlzs\" (UID: \"59716931-45d5-4ad2-8fea-fc240f8155ac\") " pod="openshift-monitoring/telemeter-client-9f5b64bf-mnlzs" May 28 15:10:31.779758 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.779655 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/59716931-45d5-4ad2-8fea-fc240f8155ac-federate-client-tls\") pod \"telemeter-client-9f5b64bf-mnlzs\" (UID: \"59716931-45d5-4ad2-8fea-fc240f8155ac\") " pod="openshift-monitoring/telemeter-client-9f5b64bf-mnlzs" May 28 15:10:31.779758 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.779671 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfvnh\" (UniqueName: \"kubernetes.io/projected/59716931-45d5-4ad2-8fea-fc240f8155ac-kube-api-access-wfvnh\") pod \"telemeter-client-9f5b64bf-mnlzs\" (UID: \"59716931-45d5-4ad2-8fea-fc240f8155ac\") " pod="openshift-monitoring/telemeter-client-9f5b64bf-mnlzs" May 28 15:10:31.779758 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.779698 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/59716931-45d5-4ad2-8fea-fc240f8155ac-telemeter-client-tls\") pod \"telemeter-client-9f5b64bf-mnlzs\" (UID: \"59716931-45d5-4ad2-8fea-fc240f8155ac\") " pod="openshift-monitoring/telemeter-client-9f5b64bf-mnlzs" May 28 15:10:31.880842 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.880809 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59716931-45d5-4ad2-8fea-fc240f8155ac-serving-certs-ca-bundle\") pod \"telemeter-client-9f5b64bf-mnlzs\" (UID: \"59716931-45d5-4ad2-8fea-fc240f8155ac\") " pod="openshift-monitoring/telemeter-client-9f5b64bf-mnlzs" May 28 15:10:31.881018 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.880849 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/59716931-45d5-4ad2-8fea-fc240f8155ac-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-9f5b64bf-mnlzs\" (UID: \"59716931-45d5-4ad2-8fea-fc240f8155ac\") " pod="openshift-monitoring/telemeter-client-9f5b64bf-mnlzs" May 28 15:10:31.881018 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.880888 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/59716931-45d5-4ad2-8fea-fc240f8155ac-secret-telemeter-client\") pod \"telemeter-client-9f5b64bf-mnlzs\" (UID: \"59716931-45d5-4ad2-8fea-fc240f8155ac\") " pod="openshift-monitoring/telemeter-client-9f5b64bf-mnlzs" May 28 15:10:31.881018 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.880920 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/59716931-45d5-4ad2-8fea-fc240f8155ac-metrics-client-ca\") pod \"telemeter-client-9f5b64bf-mnlzs\" (UID: \"59716931-45d5-4ad2-8fea-fc240f8155ac\") " pod="openshift-monitoring/telemeter-client-9f5b64bf-mnlzs" May 28 15:10:31.881018 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.880937 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59716931-45d5-4ad2-8fea-fc240f8155ac-telemeter-trusted-ca-bundle\") pod \"telemeter-client-9f5b64bf-mnlzs\" (UID: \"59716931-45d5-4ad2-8fea-fc240f8155ac\") " pod="openshift-monitoring/telemeter-client-9f5b64bf-mnlzs" May 28 15:10:31.881018 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.880955 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/59716931-45d5-4ad2-8fea-fc240f8155ac-federate-client-tls\") pod \"telemeter-client-9f5b64bf-mnlzs\" (UID: \"59716931-45d5-4ad2-8fea-fc240f8155ac\") " pod="openshift-monitoring/telemeter-client-9f5b64bf-mnlzs" May 28 15:10:31.881018 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.880971 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wfvnh\" (UniqueName: \"kubernetes.io/projected/59716931-45d5-4ad2-8fea-fc240f8155ac-kube-api-access-wfvnh\") pod \"telemeter-client-9f5b64bf-mnlzs\" (UID: \"59716931-45d5-4ad2-8fea-fc240f8155ac\") " pod="openshift-monitoring/telemeter-client-9f5b64bf-mnlzs" May 28 15:10:31.881018 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.881001 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/59716931-45d5-4ad2-8fea-fc240f8155ac-telemeter-client-tls\") pod \"telemeter-client-9f5b64bf-mnlzs\" (UID: \"59716931-45d5-4ad2-8fea-fc240f8155ac\") " pod="openshift-monitoring/telemeter-client-9f5b64bf-mnlzs" May 28 15:10:31.881784 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.881750 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59716931-45d5-4ad2-8fea-fc240f8155ac-serving-certs-ca-bundle\") pod \"telemeter-client-9f5b64bf-mnlzs\" (UID: \"59716931-45d5-4ad2-8fea-fc240f8155ac\") " pod="openshift-monitoring/telemeter-client-9f5b64bf-mnlzs" May 28 15:10:31.881784 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.881773 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/59716931-45d5-4ad2-8fea-fc240f8155ac-metrics-client-ca\") pod \"telemeter-client-9f5b64bf-mnlzs\" (UID: \"59716931-45d5-4ad2-8fea-fc240f8155ac\") " pod="openshift-monitoring/telemeter-client-9f5b64bf-mnlzs" May 28 15:10:31.881957 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.881928 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59716931-45d5-4ad2-8fea-fc240f8155ac-telemeter-trusted-ca-bundle\") pod \"telemeter-client-9f5b64bf-mnlzs\" (UID: \"59716931-45d5-4ad2-8fea-fc240f8155ac\") " pod="openshift-monitoring/telemeter-client-9f5b64bf-mnlzs" May 28 15:10:31.883892 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.883867 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/59716931-45d5-4ad2-8fea-fc240f8155ac-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-9f5b64bf-mnlzs\" (UID: \"59716931-45d5-4ad2-8fea-fc240f8155ac\") " pod="openshift-monitoring/telemeter-client-9f5b64bf-mnlzs" May 28 15:10:31.884015 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.883998 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/59716931-45d5-4ad2-8fea-fc240f8155ac-secret-telemeter-client\") pod \"telemeter-client-9f5b64bf-mnlzs\" (UID: \"59716931-45d5-4ad2-8fea-fc240f8155ac\") " pod="openshift-monitoring/telemeter-client-9f5b64bf-mnlzs" May 28 15:10:31.884086 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.884060 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/59716931-45d5-4ad2-8fea-fc240f8155ac-federate-client-tls\") pod \"telemeter-client-9f5b64bf-mnlzs\" (UID: \"59716931-45d5-4ad2-8fea-fc240f8155ac\") " pod="openshift-monitoring/telemeter-client-9f5b64bf-mnlzs" May 28 15:10:31.884170 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.884151 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/59716931-45d5-4ad2-8fea-fc240f8155ac-telemeter-client-tls\") pod \"telemeter-client-9f5b64bf-mnlzs\" (UID: \"59716931-45d5-4ad2-8fea-fc240f8155ac\") " pod="openshift-monitoring/telemeter-client-9f5b64bf-mnlzs" May 28 15:10:31.889648 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.889607 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfvnh\" (UniqueName: \"kubernetes.io/projected/59716931-45d5-4ad2-8fea-fc240f8155ac-kube-api-access-wfvnh\") pod \"telemeter-client-9f5b64bf-mnlzs\" (UID: \"59716931-45d5-4ad2-8fea-fc240f8155ac\") " pod="openshift-monitoring/telemeter-client-9f5b64bf-mnlzs" May 28 15:10:31.926141 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:31.926113 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-9f5b64bf-mnlzs" May 28 15:10:32.056293 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:32.056266 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-9f5b64bf-mnlzs"] May 28 15:10:32.059172 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:10:32.059148 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59716931_45d5_4ad2_8fea_fc240f8155ac.slice/crio-dc4da3db0178551ef78f2a4c7d7061281bbed665afed1a492d2d14690b13702c WatchSource:0}: Error finding container dc4da3db0178551ef78f2a4c7d7061281bbed665afed1a492d2d14690b13702c: Status 404 returned error can't find the container with id dc4da3db0178551ef78f2a4c7d7061281bbed665afed1a492d2d14690b13702c May 28 15:10:32.216130 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:32.216030 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9f5b64bf-mnlzs" event={"ID":"59716931-45d5-4ad2-8fea-fc240f8155ac","Type":"ContainerStarted","Data":"dc4da3db0178551ef78f2a4c7d7061281bbed665afed1a492d2d14690b13702c"} May 28 15:10:32.217250 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:32.217223 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7844b88b7-gjcqf" event={"ID":"6db25964-8104-4046-bfe7-a17f56d28a66","Type":"ContainerStarted","Data":"7369d000c3e9fef212cef4851a474258bb2ee8874d02c40e31bd0450c18f3efe"} May 28 15:10:32.219360 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:32.219326 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5479d49877-hxbf6" event={"ID":"0c588793-be04-492c-b10a-391b5bd3686b","Type":"ContainerStarted","Data":"b66c3b51bc6ed1613fd453ce4202a2f7c76458a3275c43b5b8a0ac85a5211791"} May 28 15:10:32.219472 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:32.219360 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5479d49877-hxbf6" event={"ID":"0c588793-be04-492c-b10a-391b5bd3686b","Type":"ContainerStarted","Data":"2bdf84e7c4e5ba71a7dc1a624837db0c5a4479c0f1fbfc7b70e01df69d3b3b9e"} May 28 15:10:32.219472 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:32.219374 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5479d49877-hxbf6" event={"ID":"0c588793-be04-492c-b10a-391b5bd3686b","Type":"ContainerStarted","Data":"b6f16947bd5e04ed4b5fad823548a178802eb45cf77b64b720bbeb321c2a0951"} May 28 15:10:32.220780 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:32.220757 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f45f64f46-k67zx" event={"ID":"e7660a27-79df-4fa5-8785-2d9ceead8c61","Type":"ContainerStarted","Data":"20c6691b9ebb4469b1f5bd639d9cea2f926690794df46f4c4221cf723d954acb"} May 28 15:10:32.220780 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:32.220782 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f45f64f46-k67zx" event={"ID":"e7660a27-79df-4fa5-8785-2d9ceead8c61","Type":"ContainerStarted","Data":"25973ad6e340e8bffe03206bee103086bc7e19e4b95db4994577fffb5a7c1807"} May 28 15:10:32.239789 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:32.239744 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-6f45f64f46-k67zx" podStartSLOduration=1.23972786 podStartE2EDuration="1.23972786s" podCreationTimestamp="2026-05-28 15:10:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 15:10:32.237757154 +0000 UTC m=+183.126744239" watchObservedRunningTime="2026-05-28 15:10:32.23972786 +0000 UTC m=+183.128714945" May 28 15:10:34.177043 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:34.177013 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-nvshr" May 28 15:10:34.229902 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:34.229299 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7844b88b7-gjcqf" event={"ID":"6db25964-8104-4046-bfe7-a17f56d28a66","Type":"ContainerStarted","Data":"ad4498f90a1ed5158e9300d93711751432ea6033b61c91833d815adcafdcdacd"} May 28 15:10:34.232343 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:34.232303 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5479d49877-hxbf6" event={"ID":"0c588793-be04-492c-b10a-391b5bd3686b","Type":"ContainerStarted","Data":"195f72a71973a6a7e726809062b78d5f74a3e2fa08af9140115037dd532228f5"} May 28 15:10:34.246476 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:34.246428 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7844b88b7-gjcqf" podStartSLOduration=1.819419355 podStartE2EDuration="4.246414217s" podCreationTimestamp="2026-05-28 15:10:30 +0000 UTC" firstStartedPulling="2026-05-28 15:10:31.256007893 +0000 UTC m=+182.144994956" lastFinishedPulling="2026-05-28 15:10:33.683002754 +0000 UTC m=+184.571989818" observedRunningTime="2026-05-28 15:10:34.24586311 +0000 UTC m=+185.134850195" watchObservedRunningTime="2026-05-28 15:10:34.246414217 +0000 UTC m=+185.135401302" May 28 15:10:34.631128 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:34.631091 2559 patch_prober.go:28] interesting pod/image-registry-5bd878d5c4-fx69p container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} May 28 15:10:34.631255 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:34.631133 2559 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" podUID="8959c2ac-69f0-4285-885b-1035cf57475e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" May 28 15:10:35.236582 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:35.236459 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9f5b64bf-mnlzs" event={"ID":"59716931-45d5-4ad2-8fea-fc240f8155ac","Type":"ContainerStarted","Data":"cfc77c227f22c3dc03565f69153d036b290fa02fef31eaa43f2eb9a2cfaabb06"} May 28 15:10:35.238892 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:35.238868 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5479d49877-hxbf6" event={"ID":"0c588793-be04-492c-b10a-391b5bd3686b","Type":"ContainerStarted","Data":"8d758a18bf67f2c91d11ff88e8bcc87076c1c0185bd3464c9660e5abce6af400"} May 28 15:10:35.238993 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:35.238899 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5479d49877-hxbf6" event={"ID":"0c588793-be04-492c-b10a-391b5bd3686b","Type":"ContainerStarted","Data":"760bf4469d0429d1341cde9677b6e1dc14776223e034a9a72cdfa51311ddbaf2"} May 28 15:10:35.239059 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:35.239041 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5479d49877-hxbf6" May 28 15:10:35.258553 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:35.258512 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5479d49877-hxbf6" podStartSLOduration=2.41966739 podStartE2EDuration="7.258499545s" podCreationTimestamp="2026-05-28 15:10:28 +0000 UTC" firstStartedPulling="2026-05-28 15:10:28.841462225 +0000 UTC m=+179.730449306" lastFinishedPulling="2026-05-28 15:10:33.680294394 +0000 UTC m=+184.569281461" observedRunningTime="2026-05-28 15:10:35.257998762 +0000 UTC m=+186.146985843" watchObservedRunningTime="2026-05-28 15:10:35.258499545 +0000 UTC m=+186.147486626" May 28 15:10:36.248043 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:36.248017 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5479d49877-hxbf6" May 28 15:10:37.246852 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:37.246812 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9f5b64bf-mnlzs" event={"ID":"59716931-45d5-4ad2-8fea-fc240f8155ac","Type":"ContainerStarted","Data":"3b649d368eb4da33b2d3d149732b7e5b4a5a4a9fe61a4003a1b201aedf0e723a"} May 28 15:10:37.247080 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:37.246858 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9f5b64bf-mnlzs" event={"ID":"59716931-45d5-4ad2-8fea-fc240f8155ac","Type":"ContainerStarted","Data":"91c6e1a891f2ccebe6e92f29bf79e0ef49c64922f213b8a6dd1a7b0c317a75fb"} May 28 15:10:37.269613 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:37.269561 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-9f5b64bf-mnlzs" podStartSLOduration=1.97640032 podStartE2EDuration="6.26954857s" podCreationTimestamp="2026-05-28 15:10:31 +0000 UTC" firstStartedPulling="2026-05-28 15:10:32.060978878 +0000 UTC m=+182.949965940" lastFinishedPulling="2026-05-28 15:10:36.35412711 +0000 UTC m=+187.243114190" observedRunningTime="2026-05-28 15:10:37.267357851 +0000 UTC m=+188.156344936" watchObservedRunningTime="2026-05-28 15:10:37.26954857 +0000 UTC m=+188.158535654" May 28 15:10:38.144082 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:38.144048 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-bb7c5ff46-c77cf"] May 28 15:10:38.161740 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:38.161711 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bb7c5ff46-c77cf"] May 28 15:10:38.161860 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:38.161756 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bb7c5ff46-c77cf" May 28 15:10:38.341878 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:38.341841 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/66a1691c-0ec4-45c0-a465-e481b07a1f39-console-serving-cert\") pod \"console-bb7c5ff46-c77cf\" (UID: \"66a1691c-0ec4-45c0-a465-e481b07a1f39\") " pod="openshift-console/console-bb7c5ff46-c77cf" May 28 15:10:38.341878 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:38.341881 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/66a1691c-0ec4-45c0-a465-e481b07a1f39-console-oauth-config\") pod \"console-bb7c5ff46-c77cf\" (UID: \"66a1691c-0ec4-45c0-a465-e481b07a1f39\") " pod="openshift-console/console-bb7c5ff46-c77cf" May 28 15:10:38.342298 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:38.341920 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/66a1691c-0ec4-45c0-a465-e481b07a1f39-console-config\") pod \"console-bb7c5ff46-c77cf\" (UID: \"66a1691c-0ec4-45c0-a465-e481b07a1f39\") " pod="openshift-console/console-bb7c5ff46-c77cf" May 28 15:10:38.342298 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:38.341976 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/66a1691c-0ec4-45c0-a465-e481b07a1f39-oauth-serving-cert\") pod \"console-bb7c5ff46-c77cf\" (UID: \"66a1691c-0ec4-45c0-a465-e481b07a1f39\") " pod="openshift-console/console-bb7c5ff46-c77cf" May 28 15:10:38.342298 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:38.342016 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66a1691c-0ec4-45c0-a465-e481b07a1f39-trusted-ca-bundle\") pod \"console-bb7c5ff46-c77cf\" (UID: \"66a1691c-0ec4-45c0-a465-e481b07a1f39\") " pod="openshift-console/console-bb7c5ff46-c77cf" May 28 15:10:38.342298 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:38.342088 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/66a1691c-0ec4-45c0-a465-e481b07a1f39-service-ca\") pod \"console-bb7c5ff46-c77cf\" (UID: \"66a1691c-0ec4-45c0-a465-e481b07a1f39\") " pod="openshift-console/console-bb7c5ff46-c77cf" May 28 15:10:38.342298 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:38.342122 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzxwm\" (UniqueName: \"kubernetes.io/projected/66a1691c-0ec4-45c0-a465-e481b07a1f39-kube-api-access-dzxwm\") pod \"console-bb7c5ff46-c77cf\" (UID: \"66a1691c-0ec4-45c0-a465-e481b07a1f39\") " pod="openshift-console/console-bb7c5ff46-c77cf" May 28 15:10:38.443000 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:38.442927 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/66a1691c-0ec4-45c0-a465-e481b07a1f39-console-serving-cert\") pod \"console-bb7c5ff46-c77cf\" (UID: \"66a1691c-0ec4-45c0-a465-e481b07a1f39\") " pod="openshift-console/console-bb7c5ff46-c77cf" May 28 15:10:38.443000 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:38.442969 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/66a1691c-0ec4-45c0-a465-e481b07a1f39-console-oauth-config\") pod \"console-bb7c5ff46-c77cf\" (UID: \"66a1691c-0ec4-45c0-a465-e481b07a1f39\") " pod="openshift-console/console-bb7c5ff46-c77cf" May 28 15:10:38.443176 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:38.443115 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/66a1691c-0ec4-45c0-a465-e481b07a1f39-console-config\") pod \"console-bb7c5ff46-c77cf\" (UID: \"66a1691c-0ec4-45c0-a465-e481b07a1f39\") " pod="openshift-console/console-bb7c5ff46-c77cf" May 28 15:10:38.443296 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:38.443274 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/66a1691c-0ec4-45c0-a465-e481b07a1f39-oauth-serving-cert\") pod \"console-bb7c5ff46-c77cf\" (UID: \"66a1691c-0ec4-45c0-a465-e481b07a1f39\") " pod="openshift-console/console-bb7c5ff46-c77cf" May 28 15:10:38.443351 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:38.443341 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66a1691c-0ec4-45c0-a465-e481b07a1f39-trusted-ca-bundle\") pod \"console-bb7c5ff46-c77cf\" (UID: \"66a1691c-0ec4-45c0-a465-e481b07a1f39\") " pod="openshift-console/console-bb7c5ff46-c77cf" May 28 15:10:38.443545 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:38.443520 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/66a1691c-0ec4-45c0-a465-e481b07a1f39-service-ca\") pod \"console-bb7c5ff46-c77cf\" (UID: \"66a1691c-0ec4-45c0-a465-e481b07a1f39\") " pod="openshift-console/console-bb7c5ff46-c77cf" May 28 15:10:38.443848 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:38.443817 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/66a1691c-0ec4-45c0-a465-e481b07a1f39-console-config\") pod \"console-bb7c5ff46-c77cf\" (UID: \"66a1691c-0ec4-45c0-a465-e481b07a1f39\") " pod="openshift-console/console-bb7c5ff46-c77cf" May 28 15:10:38.443945 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:38.443844 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dzxwm\" (UniqueName: \"kubernetes.io/projected/66a1691c-0ec4-45c0-a465-e481b07a1f39-kube-api-access-dzxwm\") pod \"console-bb7c5ff46-c77cf\" (UID: \"66a1691c-0ec4-45c0-a465-e481b07a1f39\") " pod="openshift-console/console-bb7c5ff46-c77cf" May 28 15:10:38.444195 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:38.444172 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/66a1691c-0ec4-45c0-a465-e481b07a1f39-oauth-serving-cert\") pod \"console-bb7c5ff46-c77cf\" (UID: \"66a1691c-0ec4-45c0-a465-e481b07a1f39\") " pod="openshift-console/console-bb7c5ff46-c77cf" May 28 15:10:38.444635 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:38.444609 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/66a1691c-0ec4-45c0-a465-e481b07a1f39-service-ca\") pod \"console-bb7c5ff46-c77cf\" (UID: \"66a1691c-0ec4-45c0-a465-e481b07a1f39\") " pod="openshift-console/console-bb7c5ff46-c77cf" May 28 15:10:38.445097 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:38.445076 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66a1691c-0ec4-45c0-a465-e481b07a1f39-trusted-ca-bundle\") pod \"console-bb7c5ff46-c77cf\" (UID: \"66a1691c-0ec4-45c0-a465-e481b07a1f39\") " pod="openshift-console/console-bb7c5ff46-c77cf" May 28 15:10:38.445644 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:38.445626 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/66a1691c-0ec4-45c0-a465-e481b07a1f39-console-serving-cert\") pod \"console-bb7c5ff46-c77cf\" (UID: \"66a1691c-0ec4-45c0-a465-e481b07a1f39\") " pod="openshift-console/console-bb7c5ff46-c77cf" May 28 15:10:38.445731 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:38.445715 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/66a1691c-0ec4-45c0-a465-e481b07a1f39-console-oauth-config\") pod \"console-bb7c5ff46-c77cf\" (UID: \"66a1691c-0ec4-45c0-a465-e481b07a1f39\") " pod="openshift-console/console-bb7c5ff46-c77cf" May 28 15:10:38.451387 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:38.451365 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzxwm\" (UniqueName: \"kubernetes.io/projected/66a1691c-0ec4-45c0-a465-e481b07a1f39-kube-api-access-dzxwm\") pod \"console-bb7c5ff46-c77cf\" (UID: \"66a1691c-0ec4-45c0-a465-e481b07a1f39\") " pod="openshift-console/console-bb7c5ff46-c77cf" May 28 15:10:38.471251 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:38.471230 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bb7c5ff46-c77cf" May 28 15:10:38.591344 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:38.591313 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bb7c5ff46-c77cf"] May 28 15:10:38.594166 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:10:38.594139 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66a1691c_0ec4_45c0_a465_e481b07a1f39.slice/crio-cbf34b18258c4b082c53344ef5fd3bacc822bb14d3963dac716ab2ab21db3819 WatchSource:0}: Error finding container cbf34b18258c4b082c53344ef5fd3bacc822bb14d3963dac716ab2ab21db3819: Status 404 returned error can't find the container with id cbf34b18258c4b082c53344ef5fd3bacc822bb14d3963dac716ab2ab21db3819 May 28 15:10:39.255239 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:39.255199 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bb7c5ff46-c77cf" event={"ID":"66a1691c-0ec4-45c0-a465-e481b07a1f39","Type":"ContainerStarted","Data":"4c943cb14b3ec260a81ac0cc313bc8484c82bfd8279a619d51de26b33daa747b"} May 28 15:10:39.255394 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:39.255245 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bb7c5ff46-c77cf" event={"ID":"66a1691c-0ec4-45c0-a465-e481b07a1f39","Type":"ContainerStarted","Data":"cbf34b18258c4b082c53344ef5fd3bacc822bb14d3963dac716ab2ab21db3819"} May 28 15:10:39.271862 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:39.271816 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-bb7c5ff46-c77cf" podStartSLOduration=1.271803755 podStartE2EDuration="1.271803755s" podCreationTimestamp="2026-05-28 15:10:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 15:10:39.270628077 +0000 UTC m=+190.159615164" watchObservedRunningTime="2026-05-28 15:10:39.271803755 +0000 UTC m=+190.160790838" May 28 15:10:41.515149 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:41.515117 2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6f45f64f46-k67zx" May 28 15:10:41.515149 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:41.515161 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f45f64f46-k67zx" May 28 15:10:41.519438 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:41.519414 2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6f45f64f46-k67zx" May 28 15:10:42.270718 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:42.270686 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6f45f64f46-k67zx" May 28 15:10:43.166029 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:43.165999 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" May 28 15:10:48.471809 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:48.471769 2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-bb7c5ff46-c77cf" May 28 15:10:48.472375 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:48.471918 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-bb7c5ff46-c77cf" May 28 15:10:48.476730 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:48.476711 2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-bb7c5ff46-c77cf" May 28 15:10:49.286750 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:49.286721 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-bb7c5ff46-c77cf" May 28 15:10:49.331356 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:49.331325 2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f45f64f46-k67zx"] May 28 15:10:49.938554 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:49.938506 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nvshr_59c5be81-b339-4a6d-9c00-f66c662406a3/dns/0.log" May 28 15:10:49.945440 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:49.945414 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nvshr_59c5be81-b339-4a6d-9c00-f66c662406a3/kube-rbac-proxy/0.log" May 28 15:10:50.112826 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:50.112805 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kr9wd_297e38b8-9a81-4112-811d-a97cfa5341ff/dns-node-resolver/0.log" May 28 15:10:51.110136 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:51.110102 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7844b88b7-gjcqf" May 28 15:10:51.110136 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:51.110142 2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7844b88b7-gjcqf" May 28 15:10:52.304412 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:52.304378 2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5bd878d5c4-fx69p"] May 28 15:10:56.177440 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:56.177367 2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-845b45fdb8-ffvfm" podUID="43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1" containerName="console" containerID="cri-o://0ac7feeb40689769d70aa152e9d3d776967a2bcc40dbe8020d06814cab62b9f9" gracePeriod=15 May 28 15:10:56.302526 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:56.302503 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-845b45fdb8-ffvfm_43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1/console/0.log" May 28 15:10:56.302672 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:56.302539 2559 generic.go:358] "Generic (PLEG): container finished" podID="43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1" containerID="0ac7feeb40689769d70aa152e9d3d776967a2bcc40dbe8020d06814cab62b9f9" exitCode=2 May 28 15:10:56.302672 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:56.302612 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-845b45fdb8-ffvfm" event={"ID":"43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1","Type":"ContainerDied","Data":"0ac7feeb40689769d70aa152e9d3d776967a2bcc40dbe8020d06814cab62b9f9"} May 28 15:10:56.443171 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:56.443149 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-845b45fdb8-ffvfm_43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1/console/0.log" May 28 15:10:56.443272 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:56.443225 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-845b45fdb8-ffvfm" May 28 15:10:56.497825 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:56.497799 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1-console-config\") pod \"43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1\" (UID: \"43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1\") " May 28 15:10:56.497955 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:56.497841 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1-console-serving-cert\") pod \"43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1\" (UID: \"43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1\") " May 28 15:10:56.497955 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:56.497877 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zcqdw\" (UniqueName: \"kubernetes.io/projected/43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1-kube-api-access-zcqdw\") pod \"43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1\" (UID: \"43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1\") " May 28 15:10:56.497955 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:56.497911 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1-service-ca\") pod \"43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1\" (UID: \"43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1\") " May 28 15:10:56.498061 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:56.497987 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1-oauth-serving-cert\") pod \"43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1\" (UID: \"43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1\") " May 28 15:10:56.498061 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:56.498012 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1-console-oauth-config\") pod \"43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1\" (UID: \"43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1\") " May 28 15:10:56.498250 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:56.498172 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1-console-config" (OuterVolumeSpecName: "console-config") pod "43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1" (UID: "43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 28 15:10:56.498354 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:56.498287 2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1-console-config\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:10:56.498402 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:56.498354 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1-service-ca" (OuterVolumeSpecName: "service-ca") pod "43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1" (UID: "43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 28 15:10:56.498446 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:56.498392 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1" (UID: "43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 28 15:10:56.500055 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:56.500033 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1" (UID: "43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 28 15:10:56.500145 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:56.500052 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1" (UID: "43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 28 15:10:56.500145 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:56.500082 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1-kube-api-access-zcqdw" (OuterVolumeSpecName: "kube-api-access-zcqdw") pod "43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1" (UID: "43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1"). InnerVolumeSpecName "kube-api-access-zcqdw". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 15:10:56.599738 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:56.599707 2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1-console-serving-cert\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:10:56.599738 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:56.599733 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zcqdw\" (UniqueName: \"kubernetes.io/projected/43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1-kube-api-access-zcqdw\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:10:56.599738 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:56.599744 2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1-service-ca\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:10:56.599950 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:56.599754 2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1-oauth-serving-cert\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:10:56.599950 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:56.599763 2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1-console-oauth-config\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:10:57.306547 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:57.306521 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-845b45fdb8-ffvfm_43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1/console/0.log" May 28 15:10:57.306909 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:57.306638 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-845b45fdb8-ffvfm" May 28 15:10:57.306909 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:57.306647 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-845b45fdb8-ffvfm" event={"ID":"43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1","Type":"ContainerDied","Data":"0c6485c71c83e012f34f9c2024d0a37965550894c31fc2e802f1253499ece682"} May 28 15:10:57.306909 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:57.306690 2559 scope.go:117] "RemoveContainer" containerID="0ac7feeb40689769d70aa152e9d3d776967a2bcc40dbe8020d06814cab62b9f9" May 28 15:10:57.327856 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:57.327831 2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-845b45fdb8-ffvfm"] May 28 15:10:57.330853 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:57.330832 2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-845b45fdb8-ffvfm"] May 28 15:10:57.712621 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:10:57.712531 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1" path="/var/lib/kubelet/pods/43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1/volumes" May 28 15:11:08.342690 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:08.342657 2559 generic.go:358] "Generic (PLEG): container finished" podID="d75362ae-8357-492a-a275-c3ace0009a19" containerID="2e37d6719692b69147afc83b2dc999b88ed31e6ee5c6ada66fe536a86b23a431" exitCode=0 May 28 15:11:08.343128 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:08.342713 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-7zwzb" event={"ID":"d75362ae-8357-492a-a275-c3ace0009a19","Type":"ContainerDied","Data":"2e37d6719692b69147afc83b2dc999b88ed31e6ee5c6ada66fe536a86b23a431"} May 28 15:11:08.343128 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:08.342991 2559 scope.go:117] "RemoveContainer" containerID="2e37d6719692b69147afc83b2dc999b88ed31e6ee5c6ada66fe536a86b23a431" May 28 15:11:09.346559 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:09.346523 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-7zwzb" event={"ID":"d75362ae-8357-492a-a275-c3ace0009a19","Type":"ContainerStarted","Data":"b1d0539b022d67556ddcd5c47427904683f902d88defd5423e7b4add7ce2728b"} May 28 15:11:11.115134 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:11.115105 2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7844b88b7-gjcqf" May 28 15:11:11.119047 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:11.119023 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7844b88b7-gjcqf" May 28 15:11:14.350150 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:14.350093 2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-6f45f64f46-k67zx" podUID="e7660a27-79df-4fa5-8785-2d9ceead8c61" containerName="console" containerID="cri-o://20c6691b9ebb4469b1f5bd639d9cea2f926690794df46f4c4221cf723d954acb" gracePeriod=15 May 28 15:11:14.588974 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:14.588951 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6f45f64f46-k67zx_e7660a27-79df-4fa5-8785-2d9ceead8c61/console/0.log" May 28 15:11:14.589075 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:14.589013 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f45f64f46-k67zx" May 28 15:11:14.759533 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:14.759476 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4t9gl\" (UniqueName: \"kubernetes.io/projected/e7660a27-79df-4fa5-8785-2d9ceead8c61-kube-api-access-4t9gl\") pod \"e7660a27-79df-4fa5-8785-2d9ceead8c61\" (UID: \"e7660a27-79df-4fa5-8785-2d9ceead8c61\") " May 28 15:11:14.759675 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:14.759560 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e7660a27-79df-4fa5-8785-2d9ceead8c61-service-ca\") pod \"e7660a27-79df-4fa5-8785-2d9ceead8c61\" (UID: \"e7660a27-79df-4fa5-8785-2d9ceead8c61\") " May 28 15:11:14.759730 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:14.759712 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e7660a27-79df-4fa5-8785-2d9ceead8c61-oauth-serving-cert\") pod \"e7660a27-79df-4fa5-8785-2d9ceead8c61\" (UID: \"e7660a27-79df-4fa5-8785-2d9ceead8c61\") " May 28 15:11:14.759767 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:14.759743 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e7660a27-79df-4fa5-8785-2d9ceead8c61-console-oauth-config\") pod \"e7660a27-79df-4fa5-8785-2d9ceead8c61\" (UID: \"e7660a27-79df-4fa5-8785-2d9ceead8c61\") " May 28 15:11:14.759822 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:14.759769 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e7660a27-79df-4fa5-8785-2d9ceead8c61-console-serving-cert\") pod \"e7660a27-79df-4fa5-8785-2d9ceead8c61\" (UID: \"e7660a27-79df-4fa5-8785-2d9ceead8c61\") " May 28 15:11:14.759822 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:14.759806 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e7660a27-79df-4fa5-8785-2d9ceead8c61-console-config\") pod \"e7660a27-79df-4fa5-8785-2d9ceead8c61\" (UID: \"e7660a27-79df-4fa5-8785-2d9ceead8c61\") " May 28 15:11:14.759923 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:14.759840 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7660a27-79df-4fa5-8785-2d9ceead8c61-trusted-ca-bundle\") pod \"e7660a27-79df-4fa5-8785-2d9ceead8c61\" (UID: \"e7660a27-79df-4fa5-8785-2d9ceead8c61\") " May 28 15:11:14.759923 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:14.759860 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7660a27-79df-4fa5-8785-2d9ceead8c61-service-ca" (OuterVolumeSpecName: "service-ca") pod "e7660a27-79df-4fa5-8785-2d9ceead8c61" (UID: "e7660a27-79df-4fa5-8785-2d9ceead8c61"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 28 15:11:14.760093 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:14.760075 2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e7660a27-79df-4fa5-8785-2d9ceead8c61-service-ca\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:11:14.760180 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:14.760076 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7660a27-79df-4fa5-8785-2d9ceead8c61-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e7660a27-79df-4fa5-8785-2d9ceead8c61" (UID: "e7660a27-79df-4fa5-8785-2d9ceead8c61"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 28 15:11:14.760229 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:14.760187 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7660a27-79df-4fa5-8785-2d9ceead8c61-console-config" (OuterVolumeSpecName: "console-config") pod "e7660a27-79df-4fa5-8785-2d9ceead8c61" (UID: "e7660a27-79df-4fa5-8785-2d9ceead8c61"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 28 15:11:14.760280 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:14.760256 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7660a27-79df-4fa5-8785-2d9ceead8c61-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e7660a27-79df-4fa5-8785-2d9ceead8c61" (UID: "e7660a27-79df-4fa5-8785-2d9ceead8c61"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 28 15:11:14.761849 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:14.761821 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7660a27-79df-4fa5-8785-2d9ceead8c61-kube-api-access-4t9gl" (OuterVolumeSpecName: "kube-api-access-4t9gl") pod "e7660a27-79df-4fa5-8785-2d9ceead8c61" (UID: "e7660a27-79df-4fa5-8785-2d9ceead8c61"). InnerVolumeSpecName "kube-api-access-4t9gl". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 15:11:14.761944 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:14.761849 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7660a27-79df-4fa5-8785-2d9ceead8c61-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e7660a27-79df-4fa5-8785-2d9ceead8c61" (UID: "e7660a27-79df-4fa5-8785-2d9ceead8c61"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 28 15:11:14.761944 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:14.761901 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7660a27-79df-4fa5-8785-2d9ceead8c61-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e7660a27-79df-4fa5-8785-2d9ceead8c61" (UID: "e7660a27-79df-4fa5-8785-2d9ceead8c61"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 28 15:11:14.860556 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:14.860532 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4t9gl\" (UniqueName: \"kubernetes.io/projected/e7660a27-79df-4fa5-8785-2d9ceead8c61-kube-api-access-4t9gl\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:11:14.860556 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:14.860555 2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e7660a27-79df-4fa5-8785-2d9ceead8c61-oauth-serving-cert\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:11:14.860696 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:14.860566 2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e7660a27-79df-4fa5-8785-2d9ceead8c61-console-oauth-config\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:11:14.860696 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:14.860593 2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e7660a27-79df-4fa5-8785-2d9ceead8c61-console-serving-cert\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:11:14.860696 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:14.860608 2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e7660a27-79df-4fa5-8785-2d9ceead8c61-console-config\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:11:14.860696 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:14.860622 2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7660a27-79df-4fa5-8785-2d9ceead8c61-trusted-ca-bundle\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:11:15.366141 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:15.366115 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6f45f64f46-k67zx_e7660a27-79df-4fa5-8785-2d9ceead8c61/console/0.log" May 28 15:11:15.366501 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:15.366152 2559 generic.go:358] "Generic (PLEG): container finished" podID="e7660a27-79df-4fa5-8785-2d9ceead8c61" containerID="20c6691b9ebb4469b1f5bd639d9cea2f926690794df46f4c4221cf723d954acb" exitCode=2 May 28 15:11:15.366501 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:15.366217 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f45f64f46-k67zx" May 28 15:11:15.366501 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:15.366252 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f45f64f46-k67zx" event={"ID":"e7660a27-79df-4fa5-8785-2d9ceead8c61","Type":"ContainerDied","Data":"20c6691b9ebb4469b1f5bd639d9cea2f926690794df46f4c4221cf723d954acb"} May 28 15:11:15.366501 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:15.366288 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f45f64f46-k67zx" event={"ID":"e7660a27-79df-4fa5-8785-2d9ceead8c61","Type":"ContainerDied","Data":"25973ad6e340e8bffe03206bee103086bc7e19e4b95db4994577fffb5a7c1807"} May 28 15:11:15.366501 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:15.366306 2559 scope.go:117] "RemoveContainer" containerID="20c6691b9ebb4469b1f5bd639d9cea2f926690794df46f4c4221cf723d954acb" May 28 15:11:15.381826 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:15.381800 2559 scope.go:117] "RemoveContainer" containerID="20c6691b9ebb4469b1f5bd639d9cea2f926690794df46f4c4221cf723d954acb" May 28 15:11:15.382091 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:11:15.382061 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20c6691b9ebb4469b1f5bd639d9cea2f926690794df46f4c4221cf723d954acb\": container with ID starting with 20c6691b9ebb4469b1f5bd639d9cea2f926690794df46f4c4221cf723d954acb not found: ID does not exist" containerID="20c6691b9ebb4469b1f5bd639d9cea2f926690794df46f4c4221cf723d954acb" May 28 15:11:15.382139 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:15.382098 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20c6691b9ebb4469b1f5bd639d9cea2f926690794df46f4c4221cf723d954acb"} err="failed to get container status \"20c6691b9ebb4469b1f5bd639d9cea2f926690794df46f4c4221cf723d954acb\": rpc error: code = NotFound desc = could not find container \"20c6691b9ebb4469b1f5bd639d9cea2f926690794df46f4c4221cf723d954acb\": container with ID starting with 20c6691b9ebb4469b1f5bd639d9cea2f926690794df46f4c4221cf723d954acb not found: ID does not exist" May 28 15:11:15.389789 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:15.389765 2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f45f64f46-k67zx"] May 28 15:11:15.392818 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:15.392799 2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6f45f64f46-k67zx"] May 28 15:11:15.712280 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:15.712201 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7660a27-79df-4fa5-8785-2d9ceead8c61" path="/var/lib/kubelet/pods/e7660a27-79df-4fa5-8785-2d9ceead8c61/volumes" May 28 15:11:17.325849 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:17.325805 2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" podUID="8959c2ac-69f0-4285-885b-1035cf57475e" containerName="registry" containerID="cri-o://45c10a936b0bce931735a806f64bb467d650bd1965d0a5b3e1375721adb9e361" gracePeriod=30 May 28 15:11:17.565054 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:17.565034 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" May 28 15:11:17.684702 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:17.684624 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8959c2ac-69f0-4285-885b-1035cf57475e-image-registry-private-configuration\") pod \"8959c2ac-69f0-4285-885b-1035cf57475e\" (UID: \"8959c2ac-69f0-4285-885b-1035cf57475e\") " May 28 15:11:17.684702 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:17.684666 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8959c2ac-69f0-4285-885b-1035cf57475e-trusted-ca\") pod \"8959c2ac-69f0-4285-885b-1035cf57475e\" (UID: \"8959c2ac-69f0-4285-885b-1035cf57475e\") " May 28 15:11:17.684702 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:17.684688 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8959c2ac-69f0-4285-885b-1035cf57475e-bound-sa-token\") pod \"8959c2ac-69f0-4285-885b-1035cf57475e\" (UID: \"8959c2ac-69f0-4285-885b-1035cf57475e\") " May 28 15:11:17.684973 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:17.684713 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8959c2ac-69f0-4285-885b-1035cf57475e-installation-pull-secrets\") pod \"8959c2ac-69f0-4285-885b-1035cf57475e\" (UID: \"8959c2ac-69f0-4285-885b-1035cf57475e\") " May 28 15:11:17.684973 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:17.684742 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8959c2ac-69f0-4285-885b-1035cf57475e-registry-tls\") pod \"8959c2ac-69f0-4285-885b-1035cf57475e\" (UID: \"8959c2ac-69f0-4285-885b-1035cf57475e\") " May 28 15:11:17.684973 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:17.684792 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-924p2\" (UniqueName: \"kubernetes.io/projected/8959c2ac-69f0-4285-885b-1035cf57475e-kube-api-access-924p2\") pod \"8959c2ac-69f0-4285-885b-1035cf57475e\" (UID: \"8959c2ac-69f0-4285-885b-1035cf57475e\") " May 28 15:11:17.684973 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:17.684815 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8959c2ac-69f0-4285-885b-1035cf57475e-registry-certificates\") pod \"8959c2ac-69f0-4285-885b-1035cf57475e\" (UID: \"8959c2ac-69f0-4285-885b-1035cf57475e\") " May 28 15:11:17.684973 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:17.684845 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8959c2ac-69f0-4285-885b-1035cf57475e-ca-trust-extracted\") pod \"8959c2ac-69f0-4285-885b-1035cf57475e\" (UID: \"8959c2ac-69f0-4285-885b-1035cf57475e\") " May 28 15:11:17.685293 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:17.685171 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8959c2ac-69f0-4285-885b-1035cf57475e-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "8959c2ac-69f0-4285-885b-1035cf57475e" (UID: "8959c2ac-69f0-4285-885b-1035cf57475e"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 28 15:11:17.685557 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:17.685525 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8959c2ac-69f0-4285-885b-1035cf57475e-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "8959c2ac-69f0-4285-885b-1035cf57475e" (UID: "8959c2ac-69f0-4285-885b-1035cf57475e"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 28 15:11:17.687137 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:17.687107 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8959c2ac-69f0-4285-885b-1035cf57475e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "8959c2ac-69f0-4285-885b-1035cf57475e" (UID: "8959c2ac-69f0-4285-885b-1035cf57475e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 15:11:17.687290 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:17.687264 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8959c2ac-69f0-4285-885b-1035cf57475e-kube-api-access-924p2" (OuterVolumeSpecName: "kube-api-access-924p2") pod "8959c2ac-69f0-4285-885b-1035cf57475e" (UID: "8959c2ac-69f0-4285-885b-1035cf57475e"). InnerVolumeSpecName "kube-api-access-924p2". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 15:11:17.687426 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:17.687402 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8959c2ac-69f0-4285-885b-1035cf57475e-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "8959c2ac-69f0-4285-885b-1035cf57475e" (UID: "8959c2ac-69f0-4285-885b-1035cf57475e"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 28 15:11:17.687471 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:17.687405 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8959c2ac-69f0-4285-885b-1035cf57475e-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "8959c2ac-69f0-4285-885b-1035cf57475e" (UID: "8959c2ac-69f0-4285-885b-1035cf57475e"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 15:11:17.687703 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:17.687679 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8959c2ac-69f0-4285-885b-1035cf57475e-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "8959c2ac-69f0-4285-885b-1035cf57475e" (UID: "8959c2ac-69f0-4285-885b-1035cf57475e"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 28 15:11:17.693806 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:17.693784 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8959c2ac-69f0-4285-885b-1035cf57475e-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "8959c2ac-69f0-4285-885b-1035cf57475e" (UID: "8959c2ac-69f0-4285-885b-1035cf57475e"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 28 15:11:17.786441 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:17.786411 2559 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8959c2ac-69f0-4285-885b-1035cf57475e-image-registry-private-configuration\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:11:17.786441 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:17.786438 2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8959c2ac-69f0-4285-885b-1035cf57475e-trusted-ca\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:11:17.786441 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:17.786448 2559 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8959c2ac-69f0-4285-885b-1035cf57475e-bound-sa-token\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:11:17.786649 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:17.786457 2559 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8959c2ac-69f0-4285-885b-1035cf57475e-installation-pull-secrets\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:11:17.786649 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:17.786467 2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8959c2ac-69f0-4285-885b-1035cf57475e-registry-tls\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:11:17.786649 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:17.786476 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-924p2\" (UniqueName: \"kubernetes.io/projected/8959c2ac-69f0-4285-885b-1035cf57475e-kube-api-access-924p2\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:11:17.786649 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:17.786506 2559 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8959c2ac-69f0-4285-885b-1035cf57475e-registry-certificates\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:11:17.786649 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:17.786514 2559 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8959c2ac-69f0-4285-885b-1035cf57475e-ca-trust-extracted\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:11:18.376615 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:18.376577 2559 generic.go:358] "Generic (PLEG): container finished" podID="8959c2ac-69f0-4285-885b-1035cf57475e" containerID="45c10a936b0bce931735a806f64bb467d650bd1965d0a5b3e1375721adb9e361" exitCode=0 May 28 15:11:18.377044 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:18.376628 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" event={"ID":"8959c2ac-69f0-4285-885b-1035cf57475e","Type":"ContainerDied","Data":"45c10a936b0bce931735a806f64bb467d650bd1965d0a5b3e1375721adb9e361"} May 28 15:11:18.377044 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:18.376634 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" May 28 15:11:18.377044 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:18.376653 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bd878d5c4-fx69p" event={"ID":"8959c2ac-69f0-4285-885b-1035cf57475e","Type":"ContainerDied","Data":"4eca11c6335def40d5e8295d8aa8241f795fe7cca109e7a6196e1165b624cb4d"} May 28 15:11:18.377044 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:18.376670 2559 scope.go:117] "RemoveContainer" containerID="45c10a936b0bce931735a806f64bb467d650bd1965d0a5b3e1375721adb9e361" May 28 15:11:18.384449 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:18.384428 2559 scope.go:117] "RemoveContainer" containerID="45c10a936b0bce931735a806f64bb467d650bd1965d0a5b3e1375721adb9e361" May 28 15:11:18.384726 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:11:18.384707 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45c10a936b0bce931735a806f64bb467d650bd1965d0a5b3e1375721adb9e361\": container with ID starting with 45c10a936b0bce931735a806f64bb467d650bd1965d0a5b3e1375721adb9e361 not found: ID does not exist" containerID="45c10a936b0bce931735a806f64bb467d650bd1965d0a5b3e1375721adb9e361" May 28 15:11:18.384780 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:18.384735 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45c10a936b0bce931735a806f64bb467d650bd1965d0a5b3e1375721adb9e361"} err="failed to get container status \"45c10a936b0bce931735a806f64bb467d650bd1965d0a5b3e1375721adb9e361\": rpc error: code = NotFound desc = could not find container \"45c10a936b0bce931735a806f64bb467d650bd1965d0a5b3e1375721adb9e361\": container with ID starting with 45c10a936b0bce931735a806f64bb467d650bd1965d0a5b3e1375721adb9e361 not found: ID does not exist" May 28 15:11:18.393114 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:18.393092 2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5bd878d5c4-fx69p"] May 28 15:11:18.398380 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:18.398360 2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5bd878d5c4-fx69p"] May 28 15:11:19.712265 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:19.712229 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8959c2ac-69f0-4285-885b-1035cf57475e" path="/var/lib/kubelet/pods/8959c2ac-69f0-4285-885b-1035cf57475e/volumes" May 28 15:11:41.578277 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:41.578243 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3a78917-de69-42d0-8fbe-85f346276bf9-metrics-certs\") pod \"network-metrics-daemon-l9p7b\" (UID: \"e3a78917-de69-42d0-8fbe-85f346276bf9\") " pod="openshift-multus/network-metrics-daemon-l9p7b" May 28 15:11:41.580528 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:41.580509 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3a78917-de69-42d0-8fbe-85f346276bf9-metrics-certs\") pod \"network-metrics-daemon-l9p7b\" (UID: \"e3a78917-de69-42d0-8fbe-85f346276bf9\") " pod="openshift-multus/network-metrics-daemon-l9p7b" May 28 15:11:41.611759 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:41.611732 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8wjkp\"" May 28 15:11:41.619660 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:41.619643 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9p7b" May 28 15:11:41.743642 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:41.743536 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-l9p7b"] May 28 15:11:41.746294 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:11:41.746262 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3a78917_de69_42d0_8fbe_85f346276bf9.slice/crio-6d4b7c5d9c6e91ecf68ac1831770c1aeb30bb9b65cb801eb2737ae750243cb91 WatchSource:0}: Error finding container 6d4b7c5d9c6e91ecf68ac1831770c1aeb30bb9b65cb801eb2737ae750243cb91: Status 404 returned error can't find the container with id 6d4b7c5d9c6e91ecf68ac1831770c1aeb30bb9b65cb801eb2737ae750243cb91 May 28 15:11:42.442372 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:42.442333 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l9p7b" event={"ID":"e3a78917-de69-42d0-8fbe-85f346276bf9","Type":"ContainerStarted","Data":"6d4b7c5d9c6e91ecf68ac1831770c1aeb30bb9b65cb801eb2737ae750243cb91"} May 28 15:11:43.446704 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:43.446666 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l9p7b" event={"ID":"e3a78917-de69-42d0-8fbe-85f346276bf9","Type":"ContainerStarted","Data":"8a2d4f3869154b0837ca2cda8c6129af41590737dfea6b9d1ba38e73c3dbc62b"} May 28 15:11:43.447076 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:43.446713 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l9p7b" event={"ID":"e3a78917-de69-42d0-8fbe-85f346276bf9","Type":"ContainerStarted","Data":"3eb305152eac34a97963dd711df39db0b25e2496a5ae8ad0b8983f75eda4d6e3"} May 28 15:11:43.462420 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:43.462378 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-l9p7b" podStartSLOduration=253.271481329 podStartE2EDuration="4m14.462365388s" podCreationTimestamp="2026-05-28 15:07:29 +0000 UTC" firstStartedPulling="2026-05-28 15:11:41.748131113 +0000 UTC m=+252.637118175" lastFinishedPulling="2026-05-28 15:11:42.939015172 +0000 UTC m=+253.828002234" observedRunningTime="2026-05-28 15:11:43.461213722 +0000 UTC m=+254.350200807" watchObservedRunningTime="2026-05-28 15:11:43.462365388 +0000 UTC m=+254.351352471" May 28 15:11:54.958435 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:54.958400 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-6448c57698-9bw6c"] May 28 15:11:54.958861 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:54.958767 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1" containerName="console" May 28 15:11:54.958861 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:54.958782 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1" containerName="console" May 28 15:11:54.958861 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:54.958799 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8959c2ac-69f0-4285-885b-1035cf57475e" containerName="registry" May 28 15:11:54.958861 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:54.958805 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8959c2ac-69f0-4285-885b-1035cf57475e" containerName="registry" May 28 15:11:54.958861 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:54.958815 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7660a27-79df-4fa5-8785-2d9ceead8c61" containerName="console" May 28 15:11:54.958861 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:54.958820 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7660a27-79df-4fa5-8785-2d9ceead8c61" containerName="console" May 28 15:11:54.959074 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:54.958867 2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="43bd88c6-c6e6-49f4-8f8b-fbf05fba75c1" containerName="console" May 28 15:11:54.959074 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:54.958874 2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="e7660a27-79df-4fa5-8785-2d9ceead8c61" containerName="console" May 28 15:11:54.959074 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:54.958883 2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="8959c2ac-69f0-4285-885b-1035cf57475e" containerName="registry" May 28 15:11:54.960753 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:54.960738 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6448c57698-9bw6c" May 28 15:11:54.974933 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:54.974909 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6448c57698-9bw6c"] May 28 15:11:55.077025 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:55.077002 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bf978273-5024-4c57-baca-e30abd850138-service-ca\") pod \"console-6448c57698-9bw6c\" (UID: \"bf978273-5024-4c57-baca-e30abd850138\") " pod="openshift-console/console-6448c57698-9bw6c" May 28 15:11:55.077120 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:55.077029 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bf978273-5024-4c57-baca-e30abd850138-console-serving-cert\") pod \"console-6448c57698-9bw6c\" (UID: \"bf978273-5024-4c57-baca-e30abd850138\") " pod="openshift-console/console-6448c57698-9bw6c" May 28 15:11:55.077120 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:55.077057 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2n282\" (UniqueName: \"kubernetes.io/projected/bf978273-5024-4c57-baca-e30abd850138-kube-api-access-2n282\") pod \"console-6448c57698-9bw6c\" (UID: \"bf978273-5024-4c57-baca-e30abd850138\") " pod="openshift-console/console-6448c57698-9bw6c" May 28 15:11:55.077120 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:55.077076 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf978273-5024-4c57-baca-e30abd850138-trusted-ca-bundle\") pod \"console-6448c57698-9bw6c\" (UID: \"bf978273-5024-4c57-baca-e30abd850138\") " pod="openshift-console/console-6448c57698-9bw6c" May 28 15:11:55.077236 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:55.077151 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bf978273-5024-4c57-baca-e30abd850138-oauth-serving-cert\") pod \"console-6448c57698-9bw6c\" (UID: \"bf978273-5024-4c57-baca-e30abd850138\") " pod="openshift-console/console-6448c57698-9bw6c" May 28 15:11:55.077236 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:55.077180 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bf978273-5024-4c57-baca-e30abd850138-console-config\") pod \"console-6448c57698-9bw6c\" (UID: \"bf978273-5024-4c57-baca-e30abd850138\") " pod="openshift-console/console-6448c57698-9bw6c" May 28 15:11:55.077236 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:55.077225 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bf978273-5024-4c57-baca-e30abd850138-console-oauth-config\") pod \"console-6448c57698-9bw6c\" (UID: \"bf978273-5024-4c57-baca-e30abd850138\") " pod="openshift-console/console-6448c57698-9bw6c" May 28 15:11:55.178374 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:55.178352 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf978273-5024-4c57-baca-e30abd850138-trusted-ca-bundle\") pod \"console-6448c57698-9bw6c\" (UID: \"bf978273-5024-4c57-baca-e30abd850138\") " pod="openshift-console/console-6448c57698-9bw6c" May 28 15:11:55.178500 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:55.178388 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bf978273-5024-4c57-baca-e30abd850138-oauth-serving-cert\") pod \"console-6448c57698-9bw6c\" (UID: \"bf978273-5024-4c57-baca-e30abd850138\") " pod="openshift-console/console-6448c57698-9bw6c" May 28 15:11:55.178500 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:55.178405 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bf978273-5024-4c57-baca-e30abd850138-console-config\") pod \"console-6448c57698-9bw6c\" (UID: \"bf978273-5024-4c57-baca-e30abd850138\") " pod="openshift-console/console-6448c57698-9bw6c" May 28 15:11:55.178500 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:55.178432 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bf978273-5024-4c57-baca-e30abd850138-console-oauth-config\") pod \"console-6448c57698-9bw6c\" (UID: \"bf978273-5024-4c57-baca-e30abd850138\") " pod="openshift-console/console-6448c57698-9bw6c" May 28 15:11:55.178500 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:55.178452 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bf978273-5024-4c57-baca-e30abd850138-service-ca\") pod \"console-6448c57698-9bw6c\" (UID: \"bf978273-5024-4c57-baca-e30abd850138\") " pod="openshift-console/console-6448c57698-9bw6c" May 28 15:11:55.178726 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:55.178641 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bf978273-5024-4c57-baca-e30abd850138-console-serving-cert\") pod \"console-6448c57698-9bw6c\" (UID: \"bf978273-5024-4c57-baca-e30abd850138\") " pod="openshift-console/console-6448c57698-9bw6c" May 28 15:11:55.178726 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:55.178677 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2n282\" (UniqueName: \"kubernetes.io/projected/bf978273-5024-4c57-baca-e30abd850138-kube-api-access-2n282\") pod \"console-6448c57698-9bw6c\" (UID: \"bf978273-5024-4c57-baca-e30abd850138\") " pod="openshift-console/console-6448c57698-9bw6c" May 28 15:11:55.179126 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:55.179105 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bf978273-5024-4c57-baca-e30abd850138-oauth-serving-cert\") pod \"console-6448c57698-9bw6c\" (UID: \"bf978273-5024-4c57-baca-e30abd850138\") " pod="openshift-console/console-6448c57698-9bw6c" May 28 15:11:55.179126 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:55.179118 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bf978273-5024-4c57-baca-e30abd850138-console-config\") pod \"console-6448c57698-9bw6c\" (UID: \"bf978273-5024-4c57-baca-e30abd850138\") " pod="openshift-console/console-6448c57698-9bw6c" May 28 15:11:55.179248 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:55.179219 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf978273-5024-4c57-baca-e30abd850138-trusted-ca-bundle\") pod \"console-6448c57698-9bw6c\" (UID: \"bf978273-5024-4c57-baca-e30abd850138\") " pod="openshift-console/console-6448c57698-9bw6c" May 28 15:11:55.179297 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:55.179259 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bf978273-5024-4c57-baca-e30abd850138-service-ca\") pod \"console-6448c57698-9bw6c\" (UID: \"bf978273-5024-4c57-baca-e30abd850138\") " pod="openshift-console/console-6448c57698-9bw6c" May 28 15:11:55.180802 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:55.180777 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bf978273-5024-4c57-baca-e30abd850138-console-oauth-config\") pod \"console-6448c57698-9bw6c\" (UID: \"bf978273-5024-4c57-baca-e30abd850138\") " pod="openshift-console/console-6448c57698-9bw6c" May 28 15:11:55.181075 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:55.181054 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bf978273-5024-4c57-baca-e30abd850138-console-serving-cert\") pod \"console-6448c57698-9bw6c\" (UID: \"bf978273-5024-4c57-baca-e30abd850138\") " pod="openshift-console/console-6448c57698-9bw6c" May 28 15:11:55.187754 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:55.187734 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2n282\" (UniqueName: \"kubernetes.io/projected/bf978273-5024-4c57-baca-e30abd850138-kube-api-access-2n282\") pod \"console-6448c57698-9bw6c\" (UID: \"bf978273-5024-4c57-baca-e30abd850138\") " pod="openshift-console/console-6448c57698-9bw6c" May 28 15:11:55.270016 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:55.269994 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6448c57698-9bw6c" May 28 15:11:55.392779 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:55.392755 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6448c57698-9bw6c"] May 28 15:11:55.394887 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:11:55.394855 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf978273_5024_4c57_baca_e30abd850138.slice/crio-580cf26ac3134c7006e5dfc2f67269b51d828e93fac372fafe99e778e54bb098 WatchSource:0}: Error finding container 580cf26ac3134c7006e5dfc2f67269b51d828e93fac372fafe99e778e54bb098: Status 404 returned error can't find the container with id 580cf26ac3134c7006e5dfc2f67269b51d828e93fac372fafe99e778e54bb098 May 28 15:11:55.481865 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:55.481838 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6448c57698-9bw6c" event={"ID":"bf978273-5024-4c57-baca-e30abd850138","Type":"ContainerStarted","Data":"5cf3dfc00e85c24ed0ff6f406d8ed206391e2249a5d41ab0ceffa41d2663e1aa"} May 28 15:11:55.481865 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:55.481868 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6448c57698-9bw6c" event={"ID":"bf978273-5024-4c57-baca-e30abd850138","Type":"ContainerStarted","Data":"580cf26ac3134c7006e5dfc2f67269b51d828e93fac372fafe99e778e54bb098"} May 28 15:11:55.504660 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:11:55.504608 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-6448c57698-9bw6c" podStartSLOduration=1.504592559 podStartE2EDuration="1.504592559s" podCreationTimestamp="2026-05-28 15:11:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 15:11:55.500808062 +0000 UTC m=+266.389795147" watchObservedRunningTime="2026-05-28 15:11:55.504592559 +0000 UTC m=+266.393579643" May 28 15:12:05.270501 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:12:05.270449 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6448c57698-9bw6c" May 28 15:12:05.270501 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:12:05.270511 2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6448c57698-9bw6c" May 28 15:12:05.275306 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:12:05.275284 2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6448c57698-9bw6c" May 28 15:12:05.513159 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:12:05.513133 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6448c57698-9bw6c" May 28 15:12:05.557556 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:12:05.557471 2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bb7c5ff46-c77cf"] May 28 15:12:29.596045 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:12:29.596019 2559 kubelet.go:1628] "Image garbage collection succeeded" May 28 15:12:30.577766 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:12:30.577726 2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-bb7c5ff46-c77cf" podUID="66a1691c-0ec4-45c0-a465-e481b07a1f39" containerName="console" containerID="cri-o://4c943cb14b3ec260a81ac0cc313bc8484c82bfd8279a619d51de26b33daa747b" gracePeriod=15 May 28 15:12:30.814958 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:12:30.814934 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-bb7c5ff46-c77cf_66a1691c-0ec4-45c0-a465-e481b07a1f39/console/0.log" May 28 15:12:30.815235 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:12:30.814996 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bb7c5ff46-c77cf" May 28 15:12:30.941523 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:12:30.941439 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/66a1691c-0ec4-45c0-a465-e481b07a1f39-oauth-serving-cert\") pod \"66a1691c-0ec4-45c0-a465-e481b07a1f39\" (UID: \"66a1691c-0ec4-45c0-a465-e481b07a1f39\") " May 28 15:12:30.941523 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:12:30.941510 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/66a1691c-0ec4-45c0-a465-e481b07a1f39-console-serving-cert\") pod \"66a1691c-0ec4-45c0-a465-e481b07a1f39\" (UID: \"66a1691c-0ec4-45c0-a465-e481b07a1f39\") " May 28 15:12:30.941693 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:12:30.941549 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/66a1691c-0ec4-45c0-a465-e481b07a1f39-console-oauth-config\") pod \"66a1691c-0ec4-45c0-a465-e481b07a1f39\" (UID: \"66a1691c-0ec4-45c0-a465-e481b07a1f39\") " May 28 15:12:30.941693 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:12:30.941568 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66a1691c-0ec4-45c0-a465-e481b07a1f39-trusted-ca-bundle\") pod \"66a1691c-0ec4-45c0-a465-e481b07a1f39\" (UID: \"66a1691c-0ec4-45c0-a465-e481b07a1f39\") " May 28 15:12:30.941693 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:12:30.941596 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/66a1691c-0ec4-45c0-a465-e481b07a1f39-service-ca\") pod \"66a1691c-0ec4-45c0-a465-e481b07a1f39\" (UID: \"66a1691c-0ec4-45c0-a465-e481b07a1f39\") " May 28 15:12:30.941693 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:12:30.941629 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/66a1691c-0ec4-45c0-a465-e481b07a1f39-console-config\") pod \"66a1691c-0ec4-45c0-a465-e481b07a1f39\" (UID: \"66a1691c-0ec4-45c0-a465-e481b07a1f39\") " May 28 15:12:30.941693 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:12:30.941653 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dzxwm\" (UniqueName: \"kubernetes.io/projected/66a1691c-0ec4-45c0-a465-e481b07a1f39-kube-api-access-dzxwm\") pod \"66a1691c-0ec4-45c0-a465-e481b07a1f39\" (UID: \"66a1691c-0ec4-45c0-a465-e481b07a1f39\") " May 28 15:12:30.941938 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:12:30.941889 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66a1691c-0ec4-45c0-a465-e481b07a1f39-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "66a1691c-0ec4-45c0-a465-e481b07a1f39" (UID: "66a1691c-0ec4-45c0-a465-e481b07a1f39"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 28 15:12:30.942029 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:12:30.942000 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66a1691c-0ec4-45c0-a465-e481b07a1f39-service-ca" (OuterVolumeSpecName: "service-ca") pod "66a1691c-0ec4-45c0-a465-e481b07a1f39" (UID: "66a1691c-0ec4-45c0-a465-e481b07a1f39"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 28 15:12:30.942102 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:12:30.942051 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66a1691c-0ec4-45c0-a465-e481b07a1f39-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "66a1691c-0ec4-45c0-a465-e481b07a1f39" (UID: "66a1691c-0ec4-45c0-a465-e481b07a1f39"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 28 15:12:30.942102 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:12:30.942069 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66a1691c-0ec4-45c0-a465-e481b07a1f39-console-config" (OuterVolumeSpecName: "console-config") pod "66a1691c-0ec4-45c0-a465-e481b07a1f39" (UID: "66a1691c-0ec4-45c0-a465-e481b07a1f39"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 28 15:12:30.943748 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:12:30.943721 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66a1691c-0ec4-45c0-a465-e481b07a1f39-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "66a1691c-0ec4-45c0-a465-e481b07a1f39" (UID: "66a1691c-0ec4-45c0-a465-e481b07a1f39"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 28 15:12:30.943853 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:12:30.943747 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66a1691c-0ec4-45c0-a465-e481b07a1f39-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "66a1691c-0ec4-45c0-a465-e481b07a1f39" (UID: "66a1691c-0ec4-45c0-a465-e481b07a1f39"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 28 15:12:30.943853 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:12:30.943727 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66a1691c-0ec4-45c0-a465-e481b07a1f39-kube-api-access-dzxwm" (OuterVolumeSpecName: "kube-api-access-dzxwm") pod "66a1691c-0ec4-45c0-a465-e481b07a1f39" (UID: "66a1691c-0ec4-45c0-a465-e481b07a1f39"). InnerVolumeSpecName "kube-api-access-dzxwm". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 15:12:31.042330 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:12:31.042306 2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/66a1691c-0ec4-45c0-a465-e481b07a1f39-console-config\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:12:31.042330 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:12:31.042325 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dzxwm\" (UniqueName: \"kubernetes.io/projected/66a1691c-0ec4-45c0-a465-e481b07a1f39-kube-api-access-dzxwm\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:12:31.042462 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:12:31.042334 2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/66a1691c-0ec4-45c0-a465-e481b07a1f39-oauth-serving-cert\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:12:31.042462 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:12:31.042346 2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/66a1691c-0ec4-45c0-a465-e481b07a1f39-console-serving-cert\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:12:31.042462 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:12:31.042354 2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/66a1691c-0ec4-45c0-a465-e481b07a1f39-console-oauth-config\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:12:31.042462 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:12:31.042363 2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66a1691c-0ec4-45c0-a465-e481b07a1f39-trusted-ca-bundle\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:12:31.042462 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:12:31.042372 2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/66a1691c-0ec4-45c0-a465-e481b07a1f39-service-ca\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:12:31.583125 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:12:31.583100 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-bb7c5ff46-c77cf_66a1691c-0ec4-45c0-a465-e481b07a1f39/console/0.log" May 28 15:12:31.583301 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:12:31.583141 2559 generic.go:358] "Generic (PLEG): container finished" podID="66a1691c-0ec4-45c0-a465-e481b07a1f39" containerID="4c943cb14b3ec260a81ac0cc313bc8484c82bfd8279a619d51de26b33daa747b" exitCode=2 May 28 15:12:31.583301 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:12:31.583213 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bb7c5ff46-c77cf" May 28 15:12:31.583301 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:12:31.583233 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bb7c5ff46-c77cf" event={"ID":"66a1691c-0ec4-45c0-a465-e481b07a1f39","Type":"ContainerDied","Data":"4c943cb14b3ec260a81ac0cc313bc8484c82bfd8279a619d51de26b33daa747b"} May 28 15:12:31.583301 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:12:31.583278 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bb7c5ff46-c77cf" event={"ID":"66a1691c-0ec4-45c0-a465-e481b07a1f39","Type":"ContainerDied","Data":"cbf34b18258c4b082c53344ef5fd3bacc822bb14d3963dac716ab2ab21db3819"} May 28 15:12:31.583301 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:12:31.583299 2559 scope.go:117] "RemoveContainer" containerID="4c943cb14b3ec260a81ac0cc313bc8484c82bfd8279a619d51de26b33daa747b" May 28 15:12:31.591881 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:12:31.591855 2559 scope.go:117] "RemoveContainer" containerID="4c943cb14b3ec260a81ac0cc313bc8484c82bfd8279a619d51de26b33daa747b" May 28 15:12:31.592100 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:12:31.592079 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c943cb14b3ec260a81ac0cc313bc8484c82bfd8279a619d51de26b33daa747b\": container with ID starting with 4c943cb14b3ec260a81ac0cc313bc8484c82bfd8279a619d51de26b33daa747b not found: ID does not exist" containerID="4c943cb14b3ec260a81ac0cc313bc8484c82bfd8279a619d51de26b33daa747b" May 28 15:12:31.592150 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:12:31.592107 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c943cb14b3ec260a81ac0cc313bc8484c82bfd8279a619d51de26b33daa747b"} err="failed to get container status \"4c943cb14b3ec260a81ac0cc313bc8484c82bfd8279a619d51de26b33daa747b\": rpc error: code = NotFound desc = could not find container \"4c943cb14b3ec260a81ac0cc313bc8484c82bfd8279a619d51de26b33daa747b\": container with ID starting with 4c943cb14b3ec260a81ac0cc313bc8484c82bfd8279a619d51de26b33daa747b not found: ID does not exist" May 28 15:12:31.604645 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:12:31.604622 2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bb7c5ff46-c77cf"] May 28 15:12:31.607849 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:12:31.607830 2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-bb7c5ff46-c77cf"] May 28 15:12:31.712125 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:12:31.712102 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66a1691c-0ec4-45c0-a465-e481b07a1f39" path="/var/lib/kubelet/pods/66a1691c-0ec4-45c0-a465-e481b07a1f39/volumes" May 28 15:13:03.321872 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:03.321835 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68csf"] May 28 15:13:03.322407 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:03.322164 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66a1691c-0ec4-45c0-a465-e481b07a1f39" containerName="console" May 28 15:13:03.322407 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:03.322175 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="66a1691c-0ec4-45c0-a465-e481b07a1f39" containerName="console" May 28 15:13:03.322407 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:03.322231 2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="66a1691c-0ec4-45c0-a465-e481b07a1f39" containerName="console" May 28 15:13:03.324585 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:03.324568 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68csf" May 28 15:13:03.327013 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:03.326988 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" May 28 15:13:03.327978 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:03.327957 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" May 28 15:13:03.328067 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:03.327980 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ldgjz\"" May 28 15:13:03.332984 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:03.332962 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68csf"] May 28 15:13:03.384321 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:03.384284 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86b25\" (UniqueName: \"kubernetes.io/projected/8d63320f-bbbb-4c75-9d5d-c5070665482d-kube-api-access-86b25\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68csf\" (UID: \"8d63320f-bbbb-4c75-9d5d-c5070665482d\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68csf" May 28 15:13:03.384508 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:03.384394 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8d63320f-bbbb-4c75-9d5d-c5070665482d-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68csf\" (UID: \"8d63320f-bbbb-4c75-9d5d-c5070665482d\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68csf" May 28 15:13:03.384508 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:03.384425 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8d63320f-bbbb-4c75-9d5d-c5070665482d-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68csf\" (UID: \"8d63320f-bbbb-4c75-9d5d-c5070665482d\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68csf" May 28 15:13:03.485721 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:03.485675 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8d63320f-bbbb-4c75-9d5d-c5070665482d-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68csf\" (UID: \"8d63320f-bbbb-4c75-9d5d-c5070665482d\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68csf" May 28 15:13:03.485721 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:03.485719 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8d63320f-bbbb-4c75-9d5d-c5070665482d-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68csf\" (UID: \"8d63320f-bbbb-4c75-9d5d-c5070665482d\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68csf" May 28 15:13:03.485966 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:03.485882 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-86b25\" (UniqueName: \"kubernetes.io/projected/8d63320f-bbbb-4c75-9d5d-c5070665482d-kube-api-access-86b25\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68csf\" (UID: \"8d63320f-bbbb-4c75-9d5d-c5070665482d\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68csf" May 28 15:13:03.486057 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:03.486037 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8d63320f-bbbb-4c75-9d5d-c5070665482d-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68csf\" (UID: \"8d63320f-bbbb-4c75-9d5d-c5070665482d\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68csf" May 28 15:13:03.486120 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:03.486070 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8d63320f-bbbb-4c75-9d5d-c5070665482d-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68csf\" (UID: \"8d63320f-bbbb-4c75-9d5d-c5070665482d\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68csf" May 28 15:13:03.493838 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:03.493815 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86b25\" (UniqueName: \"kubernetes.io/projected/8d63320f-bbbb-4c75-9d5d-c5070665482d-kube-api-access-86b25\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68csf\" (UID: \"8d63320f-bbbb-4c75-9d5d-c5070665482d\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68csf" May 28 15:13:03.634850 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:03.634769 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68csf" May 28 15:13:03.760421 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:03.760394 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68csf"] May 28 15:13:03.763216 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:13:03.763190 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d63320f_bbbb_4c75_9d5d_c5070665482d.slice/crio-831afb787506a2898cb9840309ce66a927146d9fd13730342998e96da7258135 WatchSource:0}: Error finding container 831afb787506a2898cb9840309ce66a927146d9fd13730342998e96da7258135: Status 404 returned error can't find the container with id 831afb787506a2898cb9840309ce66a927146d9fd13730342998e96da7258135 May 28 15:13:03.764994 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:03.764974 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 28 15:13:04.678894 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:04.678857 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68csf" event={"ID":"8d63320f-bbbb-4c75-9d5d-c5070665482d","Type":"ContainerStarted","Data":"831afb787506a2898cb9840309ce66a927146d9fd13730342998e96da7258135"} May 28 15:13:10.701559 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:10.701533 2559 generic.go:358] "Generic (PLEG): container finished" podID="8d63320f-bbbb-4c75-9d5d-c5070665482d" containerID="a4cd9e07625e71064d5eb58b5f43b05ac20d6edec5815e1f4f1f12ca874efa9b" exitCode=0 May 28 15:13:10.701835 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:10.701578 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68csf" event={"ID":"8d63320f-bbbb-4c75-9d5d-c5070665482d","Type":"ContainerDied","Data":"a4cd9e07625e71064d5eb58b5f43b05ac20d6edec5815e1f4f1f12ca874efa9b"} May 28 15:13:12.711742 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:12.711710 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68csf" event={"ID":"8d63320f-bbbb-4c75-9d5d-c5070665482d","Type":"ContainerStarted","Data":"634882b12041630424ddd12be7a713859f6bd5e7cc772db5af1eb06d4dcf2672"} May 28 15:13:13.715412 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:13.715373 2559 generic.go:358] "Generic (PLEG): container finished" podID="8d63320f-bbbb-4c75-9d5d-c5070665482d" containerID="634882b12041630424ddd12be7a713859f6bd5e7cc772db5af1eb06d4dcf2672" exitCode=0 May 28 15:13:13.715867 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:13.715459 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68csf" event={"ID":"8d63320f-bbbb-4c75-9d5d-c5070665482d","Type":"ContainerDied","Data":"634882b12041630424ddd12be7a713859f6bd5e7cc772db5af1eb06d4dcf2672"} May 28 15:13:19.734623 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:19.734546 2559 generic.go:358] "Generic (PLEG): container finished" podID="8d63320f-bbbb-4c75-9d5d-c5070665482d" containerID="276de5fb1830ca093f29f040adcfe20b62c6e6ab1439a0c0858a31d0fcf95a62" exitCode=0 May 28 15:13:19.734926 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:19.734634 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68csf" event={"ID":"8d63320f-bbbb-4c75-9d5d-c5070665482d","Type":"ContainerDied","Data":"276de5fb1830ca093f29f040adcfe20b62c6e6ab1439a0c0858a31d0fcf95a62"} May 28 15:13:20.856363 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:20.856339 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68csf" May 28 15:13:20.929556 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:20.929535 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-86b25\" (UniqueName: \"kubernetes.io/projected/8d63320f-bbbb-4c75-9d5d-c5070665482d-kube-api-access-86b25\") pod \"8d63320f-bbbb-4c75-9d5d-c5070665482d\" (UID: \"8d63320f-bbbb-4c75-9d5d-c5070665482d\") " May 28 15:13:20.929692 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:20.929591 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8d63320f-bbbb-4c75-9d5d-c5070665482d-util\") pod \"8d63320f-bbbb-4c75-9d5d-c5070665482d\" (UID: \"8d63320f-bbbb-4c75-9d5d-c5070665482d\") " May 28 15:13:20.929692 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:20.929619 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8d63320f-bbbb-4c75-9d5d-c5070665482d-bundle\") pod \"8d63320f-bbbb-4c75-9d5d-c5070665482d\" (UID: \"8d63320f-bbbb-4c75-9d5d-c5070665482d\") " May 28 15:13:20.930284 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:20.930247 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d63320f-bbbb-4c75-9d5d-c5070665482d-bundle" (OuterVolumeSpecName: "bundle") pod "8d63320f-bbbb-4c75-9d5d-c5070665482d" (UID: "8d63320f-bbbb-4c75-9d5d-c5070665482d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 28 15:13:20.931642 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:20.931620 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d63320f-bbbb-4c75-9d5d-c5070665482d-kube-api-access-86b25" (OuterVolumeSpecName: "kube-api-access-86b25") pod "8d63320f-bbbb-4c75-9d5d-c5070665482d" (UID: "8d63320f-bbbb-4c75-9d5d-c5070665482d"). InnerVolumeSpecName "kube-api-access-86b25". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 15:13:20.933578 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:20.933560 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d63320f-bbbb-4c75-9d5d-c5070665482d-util" (OuterVolumeSpecName: "util") pod "8d63320f-bbbb-4c75-9d5d-c5070665482d" (UID: "8d63320f-bbbb-4c75-9d5d-c5070665482d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 28 15:13:21.030677 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:21.030647 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-86b25\" (UniqueName: \"kubernetes.io/projected/8d63320f-bbbb-4c75-9d5d-c5070665482d-kube-api-access-86b25\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:13:21.030677 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:21.030679 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8d63320f-bbbb-4c75-9d5d-c5070665482d-util\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:13:21.030797 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:21.030709 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8d63320f-bbbb-4c75-9d5d-c5070665482d-bundle\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:13:21.742119 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:21.742083 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68csf" event={"ID":"8d63320f-bbbb-4c75-9d5d-c5070665482d","Type":"ContainerDied","Data":"831afb787506a2898cb9840309ce66a927146d9fd13730342998e96da7258135"} May 28 15:13:21.742119 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:21.742121 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="831afb787506a2898cb9840309ce66a927146d9fd13730342998e96da7258135" May 28 15:13:21.742286 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:21.742142 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68csf" May 28 15:13:30.810601 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:30.810568 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-587ccfb98-gxlgz"] May 28 15:13:30.810978 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:30.810853 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d63320f-bbbb-4c75-9d5d-c5070665482d" containerName="util" May 28 15:13:30.810978 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:30.810864 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d63320f-bbbb-4c75-9d5d-c5070665482d" containerName="util" May 28 15:13:30.810978 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:30.810878 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d63320f-bbbb-4c75-9d5d-c5070665482d" containerName="extract" May 28 15:13:30.810978 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:30.810884 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d63320f-bbbb-4c75-9d5d-c5070665482d" containerName="extract" May 28 15:13:30.810978 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:30.810900 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d63320f-bbbb-4c75-9d5d-c5070665482d" containerName="pull" May 28 15:13:30.810978 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:30.810906 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d63320f-bbbb-4c75-9d5d-c5070665482d" containerName="pull" May 28 15:13:30.810978 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:30.810955 2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="8d63320f-bbbb-4c75-9d5d-c5070665482d" containerName="extract" May 28 15:13:30.815146 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:30.815128 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-587ccfb98-gxlgz" May 28 15:13:30.817706 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:30.817685 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" May 28 15:13:30.817782 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:30.817685 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-7bf5r\"" May 28 15:13:30.818587 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:30.818571 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" May 28 15:13:30.820741 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:30.820719 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-587ccfb98-gxlgz"] May 28 15:13:30.906775 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:30.906751 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7f91ec24-fab9-4bbc-b8c7-9d8a7c6c81ef-bound-sa-token\") pod \"cert-manager-webhook-587ccfb98-gxlgz\" (UID: \"7f91ec24-fab9-4bbc-b8c7-9d8a7c6c81ef\") " pod="cert-manager/cert-manager-webhook-587ccfb98-gxlgz" May 28 15:13:30.906884 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:30.906804 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ngdk\" (UniqueName: \"kubernetes.io/projected/7f91ec24-fab9-4bbc-b8c7-9d8a7c6c81ef-kube-api-access-2ngdk\") pod \"cert-manager-webhook-587ccfb98-gxlgz\" (UID: \"7f91ec24-fab9-4bbc-b8c7-9d8a7c6c81ef\") " pod="cert-manager/cert-manager-webhook-587ccfb98-gxlgz" May 28 15:13:31.007288 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:31.007263 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7f91ec24-fab9-4bbc-b8c7-9d8a7c6c81ef-bound-sa-token\") pod \"cert-manager-webhook-587ccfb98-gxlgz\" (UID: \"7f91ec24-fab9-4bbc-b8c7-9d8a7c6c81ef\") " pod="cert-manager/cert-manager-webhook-587ccfb98-gxlgz" May 28 15:13:31.007412 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:31.007317 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2ngdk\" (UniqueName: \"kubernetes.io/projected/7f91ec24-fab9-4bbc-b8c7-9d8a7c6c81ef-kube-api-access-2ngdk\") pod \"cert-manager-webhook-587ccfb98-gxlgz\" (UID: \"7f91ec24-fab9-4bbc-b8c7-9d8a7c6c81ef\") " pod="cert-manager/cert-manager-webhook-587ccfb98-gxlgz" May 28 15:13:31.015085 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:31.015065 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7f91ec24-fab9-4bbc-b8c7-9d8a7c6c81ef-bound-sa-token\") pod \"cert-manager-webhook-587ccfb98-gxlgz\" (UID: \"7f91ec24-fab9-4bbc-b8c7-9d8a7c6c81ef\") " pod="cert-manager/cert-manager-webhook-587ccfb98-gxlgz" May 28 15:13:31.015180 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:31.015128 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ngdk\" (UniqueName: \"kubernetes.io/projected/7f91ec24-fab9-4bbc-b8c7-9d8a7c6c81ef-kube-api-access-2ngdk\") pod \"cert-manager-webhook-587ccfb98-gxlgz\" (UID: \"7f91ec24-fab9-4bbc-b8c7-9d8a7c6c81ef\") " pod="cert-manager/cert-manager-webhook-587ccfb98-gxlgz" May 28 15:13:31.132283 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:31.132226 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-587ccfb98-gxlgz" May 28 15:13:31.254268 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:31.254237 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-587ccfb98-gxlgz"] May 28 15:13:31.256547 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:13:31.256506 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7f91ec24_fab9_4bbc_b8c7_9d8a7c6c81ef.slice/crio-2b6f0b09e164016cd61c77f4cf87815031529c5402398ff65a0b874d519395aa WatchSource:0}: Error finding container 2b6f0b09e164016cd61c77f4cf87815031529c5402398ff65a0b874d519395aa: Status 404 returned error can't find the container with id 2b6f0b09e164016cd61c77f4cf87815031529c5402398ff65a0b874d519395aa May 28 15:13:31.770782 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:31.770744 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-587ccfb98-gxlgz" event={"ID":"7f91ec24-fab9-4bbc-b8c7-9d8a7c6c81ef","Type":"ContainerStarted","Data":"2b6f0b09e164016cd61c77f4cf87815031529c5402398ff65a0b874d519395aa"} May 28 15:13:34.785807 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:34.785767 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-587ccfb98-gxlgz" event={"ID":"7f91ec24-fab9-4bbc-b8c7-9d8a7c6c81ef","Type":"ContainerStarted","Data":"8b9a7ddf08d34093ca3ca070291ad46ba1e31535e7ad091c7400c645b8c57da2"} May 28 15:13:34.786180 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:34.785828 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-587ccfb98-gxlgz" May 28 15:13:34.799955 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:34.799899 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-587ccfb98-gxlgz" podStartSLOduration=2.182297713 podStartE2EDuration="4.799886048s" podCreationTimestamp="2026-05-28 15:13:30 +0000 UTC" firstStartedPulling="2026-05-28 15:13:31.258260882 +0000 UTC m=+362.147247944" lastFinishedPulling="2026-05-28 15:13:33.875849202 +0000 UTC m=+364.764836279" observedRunningTime="2026-05-28 15:13:34.799790513 +0000 UTC m=+365.688777599" watchObservedRunningTime="2026-05-28 15:13:34.799886048 +0000 UTC m=+365.688873128" May 28 15:13:40.790064 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:40.790034 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-587ccfb98-gxlgz" May 28 15:13:42.330711 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:42.330680 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-79c8d999ff-ps4kf"] May 28 15:13:42.357153 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:42.357126 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-79c8d999ff-ps4kf"] May 28 15:13:42.357297 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:42.357225 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-79c8d999ff-ps4kf" May 28 15:13:42.359855 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:42.359830 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-rsb4h\"" May 28 15:13:42.500965 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:42.500938 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhnw4\" (UniqueName: \"kubernetes.io/projected/329e8339-f810-40a2-a40f-842bc9837b2d-kube-api-access-fhnw4\") pod \"cert-manager-79c8d999ff-ps4kf\" (UID: \"329e8339-f810-40a2-a40f-842bc9837b2d\") " pod="cert-manager/cert-manager-79c8d999ff-ps4kf" May 28 15:13:42.501105 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:42.500971 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/329e8339-f810-40a2-a40f-842bc9837b2d-bound-sa-token\") pod \"cert-manager-79c8d999ff-ps4kf\" (UID: \"329e8339-f810-40a2-a40f-842bc9837b2d\") " pod="cert-manager/cert-manager-79c8d999ff-ps4kf" May 28 15:13:42.601502 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:42.601430 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fhnw4\" (UniqueName: \"kubernetes.io/projected/329e8339-f810-40a2-a40f-842bc9837b2d-kube-api-access-fhnw4\") pod \"cert-manager-79c8d999ff-ps4kf\" (UID: \"329e8339-f810-40a2-a40f-842bc9837b2d\") " pod="cert-manager/cert-manager-79c8d999ff-ps4kf" May 28 15:13:42.601502 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:42.601462 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/329e8339-f810-40a2-a40f-842bc9837b2d-bound-sa-token\") pod \"cert-manager-79c8d999ff-ps4kf\" (UID: \"329e8339-f810-40a2-a40f-842bc9837b2d\") " pod="cert-manager/cert-manager-79c8d999ff-ps4kf" May 28 15:13:42.609360 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:42.609341 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/329e8339-f810-40a2-a40f-842bc9837b2d-bound-sa-token\") pod \"cert-manager-79c8d999ff-ps4kf\" (UID: \"329e8339-f810-40a2-a40f-842bc9837b2d\") " pod="cert-manager/cert-manager-79c8d999ff-ps4kf" May 28 15:13:42.609514 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:42.609477 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhnw4\" (UniqueName: \"kubernetes.io/projected/329e8339-f810-40a2-a40f-842bc9837b2d-kube-api-access-fhnw4\") pod \"cert-manager-79c8d999ff-ps4kf\" (UID: \"329e8339-f810-40a2-a40f-842bc9837b2d\") " pod="cert-manager/cert-manager-79c8d999ff-ps4kf" May 28 15:13:42.666305 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:42.666286 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-79c8d999ff-ps4kf" May 28 15:13:42.784174 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:42.784052 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-79c8d999ff-ps4kf"] May 28 15:13:42.786892 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:13:42.786868 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod329e8339_f810_40a2_a40f_842bc9837b2d.slice/crio-c1eecdb6f00339dbdb34e80115b6a38c5518a01800879ae2145f0ea4ede96808 WatchSource:0}: Error finding container c1eecdb6f00339dbdb34e80115b6a38c5518a01800879ae2145f0ea4ede96808: Status 404 returned error can't find the container with id c1eecdb6f00339dbdb34e80115b6a38c5518a01800879ae2145f0ea4ede96808 May 28 15:13:42.807908 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:42.807880 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-79c8d999ff-ps4kf" event={"ID":"329e8339-f810-40a2-a40f-842bc9837b2d","Type":"ContainerStarted","Data":"c1eecdb6f00339dbdb34e80115b6a38c5518a01800879ae2145f0ea4ede96808"} May 28 15:13:43.812439 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:43.812404 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-79c8d999ff-ps4kf" event={"ID":"329e8339-f810-40a2-a40f-842bc9837b2d","Type":"ContainerStarted","Data":"90aa5e507d2a35ed32ff6269d21d22dac4c9f9ed03c4500e152b11d0675a2ed3"} May 28 15:13:43.829633 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:13:43.829586 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-79c8d999ff-ps4kf" podStartSLOduration=1.829574076 podStartE2EDuration="1.829574076s" podCreationTimestamp="2026-05-28 15:13:42 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 15:13:43.827925551 +0000 UTC m=+374.716912634" watchObservedRunningTime="2026-05-28 15:13:43.829574076 +0000 UTC m=+374.718561161" May 28 15:14:00.518076 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:00.518045 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-b8g2p"] May 28 15:14:00.520162 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:00.520145 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-b8g2p" May 28 15:14:00.522532 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:00.522514 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" May 28 15:14:00.523445 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:00.523423 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" May 28 15:14:00.523578 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:00.523426 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-ngth7\"" May 28 15:14:00.530860 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:00.530837 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-b8g2p"] May 28 15:14:00.635496 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:00.635462 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgzkb\" (UniqueName: \"kubernetes.io/projected/4ef17cd1-d142-4170-9a5f-9509ea6c20dc-kube-api-access-xgzkb\") pod \"openshift-lws-operator-bfc7f696d-b8g2p\" (UID: \"4ef17cd1-d142-4170-9a5f-9509ea6c20dc\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-b8g2p" May 28 15:14:00.635582 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:00.635508 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4ef17cd1-d142-4170-9a5f-9509ea6c20dc-tmp\") pod \"openshift-lws-operator-bfc7f696d-b8g2p\" (UID: \"4ef17cd1-d142-4170-9a5f-9509ea6c20dc\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-b8g2p" May 28 15:14:00.735848 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:00.735821 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xgzkb\" (UniqueName: \"kubernetes.io/projected/4ef17cd1-d142-4170-9a5f-9509ea6c20dc-kube-api-access-xgzkb\") pod \"openshift-lws-operator-bfc7f696d-b8g2p\" (UID: \"4ef17cd1-d142-4170-9a5f-9509ea6c20dc\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-b8g2p" May 28 15:14:00.735926 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:00.735854 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4ef17cd1-d142-4170-9a5f-9509ea6c20dc-tmp\") pod \"openshift-lws-operator-bfc7f696d-b8g2p\" (UID: \"4ef17cd1-d142-4170-9a5f-9509ea6c20dc\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-b8g2p" May 28 15:14:00.736205 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:00.736190 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4ef17cd1-d142-4170-9a5f-9509ea6c20dc-tmp\") pod \"openshift-lws-operator-bfc7f696d-b8g2p\" (UID: \"4ef17cd1-d142-4170-9a5f-9509ea6c20dc\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-b8g2p" May 28 15:14:00.751458 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:00.751433 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgzkb\" (UniqueName: \"kubernetes.io/projected/4ef17cd1-d142-4170-9a5f-9509ea6c20dc-kube-api-access-xgzkb\") pod \"openshift-lws-operator-bfc7f696d-b8g2p\" (UID: \"4ef17cd1-d142-4170-9a5f-9509ea6c20dc\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-b8g2p" May 28 15:14:00.828499 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:00.828434 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-b8g2p" May 28 15:14:00.951020 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:00.950997 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-b8g2p"] May 28 15:14:00.953025 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:14:00.952995 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4ef17cd1_d142_4170_9a5f_9509ea6c20dc.slice/crio-9ad1a7a396314c38511849d82cadc3ded1d6432d2a0d3b80a5bd360f5a9ca470 WatchSource:0}: Error finding container 9ad1a7a396314c38511849d82cadc3ded1d6432d2a0d3b80a5bd360f5a9ca470: Status 404 returned error can't find the container with id 9ad1a7a396314c38511849d82cadc3ded1d6432d2a0d3b80a5bd360f5a9ca470 May 28 15:14:01.879767 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:01.879722 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-b8g2p" event={"ID":"4ef17cd1-d142-4170-9a5f-9509ea6c20dc","Type":"ContainerStarted","Data":"9ad1a7a396314c38511849d82cadc3ded1d6432d2a0d3b80a5bd360f5a9ca470"} May 28 15:14:05.897032 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:05.896999 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-b8g2p" event={"ID":"4ef17cd1-d142-4170-9a5f-9509ea6c20dc","Type":"ContainerStarted","Data":"1bf1f56c70f08710555f29dfb496715b0d9712f3d2741abe0517ae144c409acb"} May 28 15:14:05.913184 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:05.913138 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-b8g2p" podStartSLOduration=1.901213373 podStartE2EDuration="5.913124668s" podCreationTimestamp="2026-05-28 15:14:00 +0000 UTC" firstStartedPulling="2026-05-28 15:14:00.954925033 +0000 UTC m=+391.843912095" lastFinishedPulling="2026-05-28 15:14:04.966836326 +0000 UTC m=+395.855823390" observedRunningTime="2026-05-28 15:14:05.912916683 +0000 UTC m=+396.801903773" watchObservedRunningTime="2026-05-28 15:14:05.913124668 +0000 UTC m=+396.802111751" May 28 15:14:10.699274 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:10.699237 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5bm6cb"] May 28 15:14:10.702665 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:10.702646 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5bm6cb" May 28 15:14:10.705350 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:10.705315 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" May 28 15:14:10.705535 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:10.705361 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" May 28 15:14:10.706241 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:10.706223 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ldgjz\"" May 28 15:14:10.710528 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:10.710342 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5bm6cb"] May 28 15:14:10.710657 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:10.710633 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rq9c\" (UniqueName: \"kubernetes.io/projected/3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304-kube-api-access-7rq9c\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5bm6cb\" (UID: \"3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5bm6cb" May 28 15:14:10.710794 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:10.710728 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304-bundle\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5bm6cb\" (UID: \"3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5bm6cb" May 28 15:14:10.710794 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:10.710759 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304-util\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5bm6cb\" (UID: \"3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5bm6cb" May 28 15:14:10.811613 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:10.811581 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7rq9c\" (UniqueName: \"kubernetes.io/projected/3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304-kube-api-access-7rq9c\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5bm6cb\" (UID: \"3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5bm6cb" May 28 15:14:10.811737 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:10.811627 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304-bundle\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5bm6cb\" (UID: \"3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5bm6cb" May 28 15:14:10.811737 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:10.811672 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304-util\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5bm6cb\" (UID: \"3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5bm6cb" May 28 15:14:10.811998 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:10.811982 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304-util\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5bm6cb\" (UID: \"3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5bm6cb" May 28 15:14:10.812082 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:10.812065 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304-bundle\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5bm6cb\" (UID: \"3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5bm6cb" May 28 15:14:10.821362 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:10.821341 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rq9c\" (UniqueName: \"kubernetes.io/projected/3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304-kube-api-access-7rq9c\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5bm6cb\" (UID: \"3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5bm6cb" May 28 15:14:11.013843 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:11.013821 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5bm6cb" May 28 15:14:11.140459 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:11.140436 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5bm6cb"] May 28 15:14:11.143038 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:14:11.143009 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cc6a6c5_e47b_4ffc_a9e9_c336b5a5f304.slice/crio-053548d6ae3c379bb96dae9f261b8bd100b2965a4329b1759241a618e7298e24 WatchSource:0}: Error finding container 053548d6ae3c379bb96dae9f261b8bd100b2965a4329b1759241a618e7298e24: Status 404 returned error can't find the container with id 053548d6ae3c379bb96dae9f261b8bd100b2965a4329b1759241a618e7298e24 May 28 15:14:11.915798 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:11.915761 2559 generic.go:358] "Generic (PLEG): container finished" podID="3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304" containerID="506789b918040d2ae29c18ea86cf525a10cd166fff0cc17cb91a17d3f7394d26" exitCode=0 May 28 15:14:11.916115 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:11.915844 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5bm6cb" event={"ID":"3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304","Type":"ContainerDied","Data":"506789b918040d2ae29c18ea86cf525a10cd166fff0cc17cb91a17d3f7394d26"} May 28 15:14:11.916115 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:11.915880 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5bm6cb" event={"ID":"3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304","Type":"ContainerStarted","Data":"053548d6ae3c379bb96dae9f261b8bd100b2965a4329b1759241a618e7298e24"} May 28 15:14:12.921299 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:12.921229 2559 generic.go:358] "Generic (PLEG): container finished" podID="3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304" containerID="611b042447aed99b625ab5b0a77069d8e3e3c54aa7e195f030d130cce8696400" exitCode=0 May 28 15:14:12.921299 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:12.921278 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5bm6cb" event={"ID":"3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304","Type":"ContainerDied","Data":"611b042447aed99b625ab5b0a77069d8e3e3c54aa7e195f030d130cce8696400"} May 28 15:14:13.927455 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:13.927419 2559 generic.go:358] "Generic (PLEG): container finished" podID="3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304" containerID="bb200eeb849ba38e2067269f904f8bc76c2c46aec9694d44bacabf6cbb81c55b" exitCode=0 May 28 15:14:13.927886 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:13.927530 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5bm6cb" event={"ID":"3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304","Type":"ContainerDied","Data":"bb200eeb849ba38e2067269f904f8bc76c2c46aec9694d44bacabf6cbb81c55b"} May 28 15:14:15.063074 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:15.063052 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5bm6cb" May 28 15:14:15.144792 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:15.144763 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7rq9c\" (UniqueName: \"kubernetes.io/projected/3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304-kube-api-access-7rq9c\") pod \"3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304\" (UID: \"3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304\") " May 28 15:14:15.144936 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:15.144844 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304-bundle\") pod \"3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304\" (UID: \"3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304\") " May 28 15:14:15.144936 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:15.144872 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304-util\") pod \"3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304\" (UID: \"3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304\") " May 28 15:14:15.145542 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:15.145520 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304-bundle" (OuterVolumeSpecName: "bundle") pod "3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304" (UID: "3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 28 15:14:15.146901 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:15.146875 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304-kube-api-access-7rq9c" (OuterVolumeSpecName: "kube-api-access-7rq9c") pod "3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304" (UID: "3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304"). InnerVolumeSpecName "kube-api-access-7rq9c". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 15:14:15.150434 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:15.150399 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304-util" (OuterVolumeSpecName: "util") pod "3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304" (UID: "3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 28 15:14:15.245626 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:15.245605 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304-bundle\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:14:15.245626 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:15.245625 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304-util\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:14:15.245737 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:15.245635 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7rq9c\" (UniqueName: \"kubernetes.io/projected/3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304-kube-api-access-7rq9c\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:14:15.935759 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:15.935729 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5bm6cb" event={"ID":"3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304","Type":"ContainerDied","Data":"053548d6ae3c379bb96dae9f261b8bd100b2965a4329b1759241a618e7298e24"} May 28 15:14:15.935759 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:15.935759 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="053548d6ae3c379bb96dae9f261b8bd100b2965a4329b1759241a618e7298e24" May 28 15:14:15.935911 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:15.935787 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5bm6cb" May 28 15:14:18.518764 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:18.518732 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5657b98b9f-gxd6s"] May 28 15:14:18.519134 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:18.519056 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304" containerName="extract" May 28 15:14:18.519134 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:18.519068 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304" containerName="extract" May 28 15:14:18.519134 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:18.519079 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304" containerName="util" May 28 15:14:18.519134 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:18.519085 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304" containerName="util" May 28 15:14:18.519134 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:18.519093 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304" containerName="pull" May 28 15:14:18.519134 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:18.519098 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304" containerName="pull" May 28 15:14:18.519383 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:18.519149 2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="3cc6a6c5-e47b-4ffc-a9e9-c336b5a5f304" containerName="extract" May 28 15:14:18.523262 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:18.523246 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5657b98b9f-gxd6s" May 28 15:14:18.525946 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:18.525926 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-mqt6c\"" May 28 15:14:18.527116 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:18.527098 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" May 28 15:14:18.527219 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:18.527114 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" May 28 15:14:18.527219 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:18.527200 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" May 28 15:14:18.529930 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:18.529906 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5657b98b9f-gxd6s"] May 28 15:14:18.570817 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:18.570796 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mjhk\" (UniqueName: \"kubernetes.io/projected/33db757d-c1e2-4459-b3e9-c7847fbea04b-kube-api-access-4mjhk\") pod \"lws-controller-manager-5657b98b9f-gxd6s\" (UID: \"33db757d-c1e2-4459-b3e9-c7847fbea04b\") " pod="openshift-lws-operator/lws-controller-manager-5657b98b9f-gxd6s" May 28 15:14:18.570906 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:18.570832 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/33db757d-c1e2-4459-b3e9-c7847fbea04b-manager-config\") pod \"lws-controller-manager-5657b98b9f-gxd6s\" (UID: \"33db757d-c1e2-4459-b3e9-c7847fbea04b\") " pod="openshift-lws-operator/lws-controller-manager-5657b98b9f-gxd6s" May 28 15:14:18.570906 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:18.570899 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/33db757d-c1e2-4459-b3e9-c7847fbea04b-metrics-cert\") pod \"lws-controller-manager-5657b98b9f-gxd6s\" (UID: \"33db757d-c1e2-4459-b3e9-c7847fbea04b\") " pod="openshift-lws-operator/lws-controller-manager-5657b98b9f-gxd6s" May 28 15:14:18.570975 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:18.570930 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/33db757d-c1e2-4459-b3e9-c7847fbea04b-cert\") pod \"lws-controller-manager-5657b98b9f-gxd6s\" (UID: \"33db757d-c1e2-4459-b3e9-c7847fbea04b\") " pod="openshift-lws-operator/lws-controller-manager-5657b98b9f-gxd6s" May 28 15:14:18.672122 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:18.672096 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/33db757d-c1e2-4459-b3e9-c7847fbea04b-manager-config\") pod \"lws-controller-manager-5657b98b9f-gxd6s\" (UID: \"33db757d-c1e2-4459-b3e9-c7847fbea04b\") " pod="openshift-lws-operator/lws-controller-manager-5657b98b9f-gxd6s" May 28 15:14:18.672206 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:18.672142 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/33db757d-c1e2-4459-b3e9-c7847fbea04b-metrics-cert\") pod \"lws-controller-manager-5657b98b9f-gxd6s\" (UID: \"33db757d-c1e2-4459-b3e9-c7847fbea04b\") " pod="openshift-lws-operator/lws-controller-manager-5657b98b9f-gxd6s" May 28 15:14:18.672206 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:18.672168 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/33db757d-c1e2-4459-b3e9-c7847fbea04b-cert\") pod \"lws-controller-manager-5657b98b9f-gxd6s\" (UID: \"33db757d-c1e2-4459-b3e9-c7847fbea04b\") " pod="openshift-lws-operator/lws-controller-manager-5657b98b9f-gxd6s" May 28 15:14:18.672206 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:18.672200 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4mjhk\" (UniqueName: \"kubernetes.io/projected/33db757d-c1e2-4459-b3e9-c7847fbea04b-kube-api-access-4mjhk\") pod \"lws-controller-manager-5657b98b9f-gxd6s\" (UID: \"33db757d-c1e2-4459-b3e9-c7847fbea04b\") " pod="openshift-lws-operator/lws-controller-manager-5657b98b9f-gxd6s" May 28 15:14:18.672722 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:18.672693 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/33db757d-c1e2-4459-b3e9-c7847fbea04b-manager-config\") pod \"lws-controller-manager-5657b98b9f-gxd6s\" (UID: \"33db757d-c1e2-4459-b3e9-c7847fbea04b\") " pod="openshift-lws-operator/lws-controller-manager-5657b98b9f-gxd6s" May 28 15:14:18.674545 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:18.674527 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/33db757d-c1e2-4459-b3e9-c7847fbea04b-metrics-cert\") pod \"lws-controller-manager-5657b98b9f-gxd6s\" (UID: \"33db757d-c1e2-4459-b3e9-c7847fbea04b\") " pod="openshift-lws-operator/lws-controller-manager-5657b98b9f-gxd6s" May 28 15:14:18.674696 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:18.674678 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/33db757d-c1e2-4459-b3e9-c7847fbea04b-cert\") pod \"lws-controller-manager-5657b98b9f-gxd6s\" (UID: \"33db757d-c1e2-4459-b3e9-c7847fbea04b\") " pod="openshift-lws-operator/lws-controller-manager-5657b98b9f-gxd6s" May 28 15:14:18.686261 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:18.686236 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mjhk\" (UniqueName: \"kubernetes.io/projected/33db757d-c1e2-4459-b3e9-c7847fbea04b-kube-api-access-4mjhk\") pod \"lws-controller-manager-5657b98b9f-gxd6s\" (UID: \"33db757d-c1e2-4459-b3e9-c7847fbea04b\") " pod="openshift-lws-operator/lws-controller-manager-5657b98b9f-gxd6s" May 28 15:14:18.833658 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:18.833588 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5657b98b9f-gxd6s" May 28 15:14:18.958352 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:18.958329 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5657b98b9f-gxd6s"] May 28 15:14:18.960664 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:14:18.960637 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33db757d_c1e2_4459_b3e9_c7847fbea04b.slice/crio-ffd37bf3ef252a0e9bb67a0b0ac252b0da526b733909c0ea1a99e6e66d09991b WatchSource:0}: Error finding container ffd37bf3ef252a0e9bb67a0b0ac252b0da526b733909c0ea1a99e6e66d09991b: Status 404 returned error can't find the container with id ffd37bf3ef252a0e9bb67a0b0ac252b0da526b733909c0ea1a99e6e66d09991b May 28 15:14:19.950340 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:19.950302 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5657b98b9f-gxd6s" event={"ID":"33db757d-c1e2-4459-b3e9-c7847fbea04b","Type":"ContainerStarted","Data":"ffd37bf3ef252a0e9bb67a0b0ac252b0da526b733909c0ea1a99e6e66d09991b"} May 28 15:14:20.954798 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:20.954719 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5657b98b9f-gxd6s" event={"ID":"33db757d-c1e2-4459-b3e9-c7847fbea04b","Type":"ContainerStarted","Data":"e0a10197574e430f62f3997e096802d0ee58df98f024ad6d3ff96484202d0ee7"} May 28 15:14:20.954798 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:20.954768 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5657b98b9f-gxd6s" May 28 15:14:20.974565 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:20.974513 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5657b98b9f-gxd6s" podStartSLOduration=1.294154078 podStartE2EDuration="2.974474641s" podCreationTimestamp="2026-05-28 15:14:18 +0000 UTC" firstStartedPulling="2026-05-28 15:14:18.962415491 +0000 UTC m=+409.851402555" lastFinishedPulling="2026-05-28 15:14:20.642736043 +0000 UTC m=+411.531723118" observedRunningTime="2026-05-28 15:14:20.974228093 +0000 UTC m=+411.863215179" watchObservedRunningTime="2026-05-28 15:14:20.974474641 +0000 UTC m=+411.863461725" May 28 15:14:21.838216 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:21.838180 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-57c44ff6d8-b2zj5"] May 28 15:14:21.845241 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:21.842292 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-57c44ff6d8-b2zj5" May 28 15:14:21.845241 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:21.845211 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-c8lf9\"" May 28 15:14:21.846031 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:21.845996 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" May 28 15:14:21.846031 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:21.846021 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" May 28 15:14:21.846234 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:21.846088 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" May 28 15:14:21.846234 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:21.846095 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" May 28 15:14:21.863189 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:21.863166 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-57c44ff6d8-b2zj5"] May 28 15:14:21.894780 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:21.894756 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2s7z\" (UniqueName: \"kubernetes.io/projected/0dbad4af-3d6a-4836-bee5-0c9b0e0a2013-kube-api-access-h2s7z\") pod \"opendatahub-operator-controller-manager-57c44ff6d8-b2zj5\" (UID: \"0dbad4af-3d6a-4836-bee5-0c9b0e0a2013\") " pod="opendatahub/opendatahub-operator-controller-manager-57c44ff6d8-b2zj5" May 28 15:14:21.894877 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:21.894790 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0dbad4af-3d6a-4836-bee5-0c9b0e0a2013-apiservice-cert\") pod \"opendatahub-operator-controller-manager-57c44ff6d8-b2zj5\" (UID: \"0dbad4af-3d6a-4836-bee5-0c9b0e0a2013\") " pod="opendatahub/opendatahub-operator-controller-manager-57c44ff6d8-b2zj5" May 28 15:14:21.894877 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:21.894807 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0dbad4af-3d6a-4836-bee5-0c9b0e0a2013-webhook-cert\") pod \"opendatahub-operator-controller-manager-57c44ff6d8-b2zj5\" (UID: \"0dbad4af-3d6a-4836-bee5-0c9b0e0a2013\") " pod="opendatahub/opendatahub-operator-controller-manager-57c44ff6d8-b2zj5" May 28 15:14:21.996048 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:21.996025 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-h2s7z\" (UniqueName: \"kubernetes.io/projected/0dbad4af-3d6a-4836-bee5-0c9b0e0a2013-kube-api-access-h2s7z\") pod \"opendatahub-operator-controller-manager-57c44ff6d8-b2zj5\" (UID: \"0dbad4af-3d6a-4836-bee5-0c9b0e0a2013\") " pod="opendatahub/opendatahub-operator-controller-manager-57c44ff6d8-b2zj5" May 28 15:14:21.996407 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:21.996124 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0dbad4af-3d6a-4836-bee5-0c9b0e0a2013-apiservice-cert\") pod \"opendatahub-operator-controller-manager-57c44ff6d8-b2zj5\" (UID: \"0dbad4af-3d6a-4836-bee5-0c9b0e0a2013\") " pod="opendatahub/opendatahub-operator-controller-manager-57c44ff6d8-b2zj5" May 28 15:14:21.996407 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:21.996159 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0dbad4af-3d6a-4836-bee5-0c9b0e0a2013-webhook-cert\") pod \"opendatahub-operator-controller-manager-57c44ff6d8-b2zj5\" (UID: \"0dbad4af-3d6a-4836-bee5-0c9b0e0a2013\") " pod="opendatahub/opendatahub-operator-controller-manager-57c44ff6d8-b2zj5" May 28 15:14:21.998621 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:21.998599 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0dbad4af-3d6a-4836-bee5-0c9b0e0a2013-webhook-cert\") pod \"opendatahub-operator-controller-manager-57c44ff6d8-b2zj5\" (UID: \"0dbad4af-3d6a-4836-bee5-0c9b0e0a2013\") " pod="opendatahub/opendatahub-operator-controller-manager-57c44ff6d8-b2zj5" May 28 15:14:21.998695 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:21.998678 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0dbad4af-3d6a-4836-bee5-0c9b0e0a2013-apiservice-cert\") pod \"opendatahub-operator-controller-manager-57c44ff6d8-b2zj5\" (UID: \"0dbad4af-3d6a-4836-bee5-0c9b0e0a2013\") " pod="opendatahub/opendatahub-operator-controller-manager-57c44ff6d8-b2zj5" May 28 15:14:22.009003 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:22.008984 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2s7z\" (UniqueName: \"kubernetes.io/projected/0dbad4af-3d6a-4836-bee5-0c9b0e0a2013-kube-api-access-h2s7z\") pod \"opendatahub-operator-controller-manager-57c44ff6d8-b2zj5\" (UID: \"0dbad4af-3d6a-4836-bee5-0c9b0e0a2013\") " pod="opendatahub/opendatahub-operator-controller-manager-57c44ff6d8-b2zj5" May 28 15:14:22.153507 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:22.153437 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-57c44ff6d8-b2zj5" May 28 15:14:22.296443 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:22.296413 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-57c44ff6d8-b2zj5"] May 28 15:14:22.298812 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:14:22.298786 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dbad4af_3d6a_4836_bee5_0c9b0e0a2013.slice/crio-0687945086a29d5eea91e194c34a87b83eb14f8fc4e7dd076b195293651b919d WatchSource:0}: Error finding container 0687945086a29d5eea91e194c34a87b83eb14f8fc4e7dd076b195293651b919d: Status 404 returned error can't find the container with id 0687945086a29d5eea91e194c34a87b83eb14f8fc4e7dd076b195293651b919d May 28 15:14:22.962946 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:22.962906 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57c44ff6d8-b2zj5" event={"ID":"0dbad4af-3d6a-4836-bee5-0c9b0e0a2013","Type":"ContainerStarted","Data":"0687945086a29d5eea91e194c34a87b83eb14f8fc4e7dd076b195293651b919d"} May 28 15:14:24.971931 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:24.971896 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57c44ff6d8-b2zj5" event={"ID":"0dbad4af-3d6a-4836-bee5-0c9b0e0a2013","Type":"ContainerStarted","Data":"8f2437bf1624b2adf84f62ed2947375eb1dcd9fcd84a589bf1974def4f90c5d1"} May 28 15:14:24.972298 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:24.972063 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-57c44ff6d8-b2zj5" May 28 15:14:24.996794 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:24.996745 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-57c44ff6d8-b2zj5" podStartSLOduration=1.523451295 podStartE2EDuration="3.996730249s" podCreationTimestamp="2026-05-28 15:14:21 +0000 UTC" firstStartedPulling="2026-05-28 15:14:22.300510337 +0000 UTC m=+413.189497402" lastFinishedPulling="2026-05-28 15:14:24.773789281 +0000 UTC m=+415.662776356" observedRunningTime="2026-05-28 15:14:24.994888906 +0000 UTC m=+415.883876025" watchObservedRunningTime="2026-05-28 15:14:24.996730249 +0000 UTC m=+415.885717334" May 28 15:14:31.959721 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:31.959696 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5657b98b9f-gxd6s" May 28 15:14:35.976734 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:14:35.976703 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-57c44ff6d8-b2zj5" May 28 15:15:16.766270 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:16.766239 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b45p4l"] May 28 15:15:16.772120 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:16.772101 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b45p4l" May 28 15:15:16.774777 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:16.774756 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" May 28 15:15:16.774777 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:16.774757 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-2wdx8\"" May 28 15:15:16.781768 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:16.781744 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b45p4l"] May 28 15:15:16.910596 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:16.910566 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46389ad5-3b8d-4802-8fee-d069c467ab50-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b45p4l\" (UID: \"46389ad5-3b8d-4802-8fee-d069c467ab50\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b45p4l" May 28 15:15:16.910745 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:16.910627 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46389ad5-3b8d-4802-8fee-d069c467ab50-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b45p4l\" (UID: \"46389ad5-3b8d-4802-8fee-d069c467ab50\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b45p4l" May 28 15:15:16.910745 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:16.910677 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46389ad5-3b8d-4802-8fee-d069c467ab50-istio-data\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b45p4l\" (UID: \"46389ad5-3b8d-4802-8fee-d069c467ab50\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b45p4l" May 28 15:15:16.910745 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:16.910706 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nw6zq\" (UniqueName: \"kubernetes.io/projected/46389ad5-3b8d-4802-8fee-d069c467ab50-kube-api-access-nw6zq\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b45p4l\" (UID: \"46389ad5-3b8d-4802-8fee-d069c467ab50\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b45p4l" May 28 15:15:16.910745 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:16.910725 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46389ad5-3b8d-4802-8fee-d069c467ab50-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b45p4l\" (UID: \"46389ad5-3b8d-4802-8fee-d069c467ab50\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b45p4l" May 28 15:15:16.910882 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:16.910749 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46389ad5-3b8d-4802-8fee-d069c467ab50-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b45p4l\" (UID: \"46389ad5-3b8d-4802-8fee-d069c467ab50\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b45p4l" May 28 15:15:16.910882 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:16.910769 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46389ad5-3b8d-4802-8fee-d069c467ab50-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b45p4l\" (UID: \"46389ad5-3b8d-4802-8fee-d069c467ab50\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b45p4l" May 28 15:15:16.910882 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:16.910857 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46389ad5-3b8d-4802-8fee-d069c467ab50-istio-token\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b45p4l\" (UID: \"46389ad5-3b8d-4802-8fee-d069c467ab50\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b45p4l" May 28 15:15:16.910972 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:16.910889 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46389ad5-3b8d-4802-8fee-d069c467ab50-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b45p4l\" (UID: \"46389ad5-3b8d-4802-8fee-d069c467ab50\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b45p4l" May 28 15:15:17.012030 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:17.011997 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46389ad5-3b8d-4802-8fee-d069c467ab50-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b45p4l\" (UID: \"46389ad5-3b8d-4802-8fee-d069c467ab50\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b45p4l" May 28 15:15:17.012175 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:17.012071 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46389ad5-3b8d-4802-8fee-d069c467ab50-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b45p4l\" (UID: \"46389ad5-3b8d-4802-8fee-d069c467ab50\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b45p4l" May 28 15:15:17.012175 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:17.012124 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46389ad5-3b8d-4802-8fee-d069c467ab50-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b45p4l\" (UID: \"46389ad5-3b8d-4802-8fee-d069c467ab50\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b45p4l" May 28 15:15:17.012249 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:17.012182 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46389ad5-3b8d-4802-8fee-d069c467ab50-istio-data\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b45p4l\" (UID: \"46389ad5-3b8d-4802-8fee-d069c467ab50\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b45p4l" May 28 15:15:17.012249 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:17.012211 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nw6zq\" (UniqueName: \"kubernetes.io/projected/46389ad5-3b8d-4802-8fee-d069c467ab50-kube-api-access-nw6zq\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b45p4l\" (UID: \"46389ad5-3b8d-4802-8fee-d069c467ab50\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b45p4l" May 28 15:15:17.012249 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:17.012182 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46389ad5-3b8d-4802-8fee-d069c467ab50-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b45p4l\" (UID: \"46389ad5-3b8d-4802-8fee-d069c467ab50\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b45p4l" May 28 15:15:17.012249 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:17.012239 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46389ad5-3b8d-4802-8fee-d069c467ab50-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b45p4l\" (UID: \"46389ad5-3b8d-4802-8fee-d069c467ab50\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b45p4l" May 28 15:15:17.012418 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:17.012287 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46389ad5-3b8d-4802-8fee-d069c467ab50-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b45p4l\" (UID: \"46389ad5-3b8d-4802-8fee-d069c467ab50\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b45p4l" May 28 15:15:17.012418 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:17.012319 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46389ad5-3b8d-4802-8fee-d069c467ab50-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b45p4l\" (UID: \"46389ad5-3b8d-4802-8fee-d069c467ab50\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b45p4l" May 28 15:15:17.012928 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:17.012706 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46389ad5-3b8d-4802-8fee-d069c467ab50-istio-token\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b45p4l\" (UID: \"46389ad5-3b8d-4802-8fee-d069c467ab50\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b45p4l" May 28 15:15:17.012928 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:17.012808 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46389ad5-3b8d-4802-8fee-d069c467ab50-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b45p4l\" (UID: \"46389ad5-3b8d-4802-8fee-d069c467ab50\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b45p4l" May 28 15:15:17.012928 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:17.012869 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46389ad5-3b8d-4802-8fee-d069c467ab50-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b45p4l\" (UID: \"46389ad5-3b8d-4802-8fee-d069c467ab50\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b45p4l" May 28 15:15:17.012928 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:17.012911 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46389ad5-3b8d-4802-8fee-d069c467ab50-istio-data\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b45p4l\" (UID: \"46389ad5-3b8d-4802-8fee-d069c467ab50\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b45p4l" May 28 15:15:17.013790 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:17.013759 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46389ad5-3b8d-4802-8fee-d069c467ab50-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b45p4l\" (UID: \"46389ad5-3b8d-4802-8fee-d069c467ab50\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b45p4l" May 28 15:15:17.015343 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:17.015310 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46389ad5-3b8d-4802-8fee-d069c467ab50-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b45p4l\" (UID: \"46389ad5-3b8d-4802-8fee-d069c467ab50\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b45p4l" May 28 15:15:17.020288 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:17.020230 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46389ad5-3b8d-4802-8fee-d069c467ab50-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b45p4l\" (UID: \"46389ad5-3b8d-4802-8fee-d069c467ab50\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b45p4l" May 28 15:15:17.021650 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:17.021624 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46389ad5-3b8d-4802-8fee-d069c467ab50-istio-token\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b45p4l\" (UID: \"46389ad5-3b8d-4802-8fee-d069c467ab50\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b45p4l" May 28 15:15:17.021785 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:17.021769 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nw6zq\" (UniqueName: \"kubernetes.io/projected/46389ad5-3b8d-4802-8fee-d069c467ab50-kube-api-access-nw6zq\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b45p4l\" (UID: \"46389ad5-3b8d-4802-8fee-d069c467ab50\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b45p4l" May 28 15:15:17.083900 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:17.083879 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b45p4l" May 28 15:15:17.212674 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:17.212647 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b45p4l"] May 28 15:15:17.214110 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:15:17.214081 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46389ad5_3b8d_4802_8fee_d069c467ab50.slice/crio-c0fa719aabcf4c0abb76c00e226c19bf22c93c153e2cd3598c4fa4aa9966cee3 WatchSource:0}: Error finding container c0fa719aabcf4c0abb76c00e226c19bf22c93c153e2cd3598c4fa4aa9966cee3: Status 404 returned error can't find the container with id c0fa719aabcf4c0abb76c00e226c19bf22c93c153e2cd3598c4fa4aa9966cee3 May 28 15:15:18.148830 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:18.148789 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b45p4l" event={"ID":"46389ad5-3b8d-4802-8fee-d069c467ab50","Type":"ContainerStarted","Data":"c0fa719aabcf4c0abb76c00e226c19bf22c93c153e2cd3598c4fa4aa9966cee3"} May 28 15:15:19.675732 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:19.675691 2559 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} May 28 15:15:19.676115 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:19.675793 2559 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} May 28 15:15:19.676115 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:19.675890 2559 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} May 28 15:15:20.158292 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:20.158240 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b45p4l" event={"ID":"46389ad5-3b8d-4802-8fee-d069c467ab50","Type":"ContainerStarted","Data":"7447b7265cab07c56f9f3acfa53210c9993c7c968111fc52e64644ecb411a1a6"} May 28 15:15:20.179072 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:20.179008 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b45p4l" podStartSLOduration=1.719426135 podStartE2EDuration="4.17899103s" podCreationTimestamp="2026-05-28 15:15:16 +0000 UTC" firstStartedPulling="2026-05-28 15:15:17.215884594 +0000 UTC m=+468.104871657" lastFinishedPulling="2026-05-28 15:15:19.675449487 +0000 UTC m=+470.564436552" observedRunningTime="2026-05-28 15:15:20.177371729 +0000 UTC m=+471.066358823" watchObservedRunningTime="2026-05-28 15:15:20.17899103 +0000 UTC m=+471.067978116" May 28 15:15:21.084451 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:21.084409 2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b45p4l" May 28 15:15:21.089225 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:21.089193 2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b45p4l" May 28 15:15:21.161448 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:21.161415 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b45p4l" May 28 15:15:21.162370 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:21.162351 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b45p4l" May 28 15:15:39.654520 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:39.654473 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ksfv9"] May 28 15:15:39.658137 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:39.658115 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ksfv9" May 28 15:15:39.660657 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:39.660633 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" May 28 15:15:39.660766 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:39.660633 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" May 28 15:15:39.660766 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:39.660723 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-bgn5c\"" May 28 15:15:39.665412 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:39.665385 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ksfv9"] May 28 15:15:39.694075 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:39.694045 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ht5rs\" (UniqueName: \"kubernetes.io/projected/899539bf-f715-4843-bc4b-b439e64a5760-kube-api-access-ht5rs\") pod \"kuadrant-operator-catalog-ksfv9\" (UID: \"899539bf-f715-4843-bc4b-b439e64a5760\") " pod="kuadrant-system/kuadrant-operator-catalog-ksfv9" May 28 15:15:39.795231 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:39.795199 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ht5rs\" (UniqueName: \"kubernetes.io/projected/899539bf-f715-4843-bc4b-b439e64a5760-kube-api-access-ht5rs\") pod \"kuadrant-operator-catalog-ksfv9\" (UID: \"899539bf-f715-4843-bc4b-b439e64a5760\") " pod="kuadrant-system/kuadrant-operator-catalog-ksfv9" May 28 15:15:39.803237 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:39.803203 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ht5rs\" (UniqueName: \"kubernetes.io/projected/899539bf-f715-4843-bc4b-b439e64a5760-kube-api-access-ht5rs\") pod \"kuadrant-operator-catalog-ksfv9\" (UID: \"899539bf-f715-4843-bc4b-b439e64a5760\") " pod="kuadrant-system/kuadrant-operator-catalog-ksfv9" May 28 15:15:39.968188 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:39.968089 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ksfv9" May 28 15:15:40.025819 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:40.025785 2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ksfv9"] May 28 15:15:40.094262 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:40.094241 2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ksfv9"] May 28 15:15:40.097104 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:15:40.097080 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod899539bf_f715_4843_bc4b_b439e64a5760.slice/crio-c90d2a05df00d233c61551947997d2cf6895e0712efa0747410b824eb1fcd78a WatchSource:0}: Error finding container c90d2a05df00d233c61551947997d2cf6895e0712efa0747410b824eb1fcd78a: Status 404 returned error can't find the container with id c90d2a05df00d233c61551947997d2cf6895e0712efa0747410b824eb1fcd78a May 28 15:15:40.227303 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:40.227210 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ksfv9" event={"ID":"899539bf-f715-4843-bc4b-b439e64a5760","Type":"ContainerStarted","Data":"c90d2a05df00d233c61551947997d2cf6895e0712efa0747410b824eb1fcd78a"} May 28 15:15:40.234376 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:40.234347 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-k7g4d"] May 28 15:15:40.238701 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:40.238684 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-k7g4d" May 28 15:15:40.244386 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:40.244362 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-k7g4d"] May 28 15:15:40.299215 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:40.299177 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqjhp\" (UniqueName: \"kubernetes.io/projected/01b6eda7-dbb7-4d91-bb5b-0a78899d6596-kube-api-access-sqjhp\") pod \"kuadrant-operator-catalog-k7g4d\" (UID: \"01b6eda7-dbb7-4d91-bb5b-0a78899d6596\") " pod="kuadrant-system/kuadrant-operator-catalog-k7g4d" May 28 15:15:40.400001 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:40.399950 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sqjhp\" (UniqueName: \"kubernetes.io/projected/01b6eda7-dbb7-4d91-bb5b-0a78899d6596-kube-api-access-sqjhp\") pod \"kuadrant-operator-catalog-k7g4d\" (UID: \"01b6eda7-dbb7-4d91-bb5b-0a78899d6596\") " pod="kuadrant-system/kuadrant-operator-catalog-k7g4d" May 28 15:15:40.408158 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:40.408137 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqjhp\" (UniqueName: \"kubernetes.io/projected/01b6eda7-dbb7-4d91-bb5b-0a78899d6596-kube-api-access-sqjhp\") pod \"kuadrant-operator-catalog-k7g4d\" (UID: \"01b6eda7-dbb7-4d91-bb5b-0a78899d6596\") " pod="kuadrant-system/kuadrant-operator-catalog-k7g4d" May 28 15:15:40.550132 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:40.550103 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-k7g4d" May 28 15:15:40.696535 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:40.696510 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-k7g4d"] May 28 15:15:40.698296 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:15:40.698267 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01b6eda7_dbb7_4d91_bb5b_0a78899d6596.slice/crio-68214f92f98c3e5596e61817d9be6c71bea454f4514996cf026b31f7e0382097 WatchSource:0}: Error finding container 68214f92f98c3e5596e61817d9be6c71bea454f4514996cf026b31f7e0382097: Status 404 returned error can't find the container with id 68214f92f98c3e5596e61817d9be6c71bea454f4514996cf026b31f7e0382097 May 28 15:15:41.215471 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:41.215439 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-84855f4f96-qp4rb"] May 28 15:15:41.218754 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:41.218731 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84855f4f96-qp4rb" May 28 15:15:41.229990 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:41.229957 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84855f4f96-qp4rb"] May 28 15:15:41.232725 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:41.232699 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-k7g4d" event={"ID":"01b6eda7-dbb7-4d91-bb5b-0a78899d6596","Type":"ContainerStarted","Data":"68214f92f98c3e5596e61817d9be6c71bea454f4514996cf026b31f7e0382097"} May 28 15:15:41.309177 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:41.309133 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3-console-oauth-config\") pod \"console-84855f4f96-qp4rb\" (UID: \"bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3\") " pod="openshift-console/console-84855f4f96-qp4rb" May 28 15:15:41.309343 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:41.309263 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3-console-serving-cert\") pod \"console-84855f4f96-qp4rb\" (UID: \"bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3\") " pod="openshift-console/console-84855f4f96-qp4rb" May 28 15:15:41.309343 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:41.309294 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3-service-ca\") pod \"console-84855f4f96-qp4rb\" (UID: \"bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3\") " pod="openshift-console/console-84855f4f96-qp4rb" May 28 15:15:41.309426 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:41.309356 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3-console-config\") pod \"console-84855f4f96-qp4rb\" (UID: \"bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3\") " pod="openshift-console/console-84855f4f96-qp4rb" May 28 15:15:41.309503 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:41.309462 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljqwp\" (UniqueName: \"kubernetes.io/projected/bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3-kube-api-access-ljqwp\") pod \"console-84855f4f96-qp4rb\" (UID: \"bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3\") " pod="openshift-console/console-84855f4f96-qp4rb" May 28 15:15:41.309576 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:41.309560 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3-trusted-ca-bundle\") pod \"console-84855f4f96-qp4rb\" (UID: \"bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3\") " pod="openshift-console/console-84855f4f96-qp4rb" May 28 15:15:41.309615 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:41.309585 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3-oauth-serving-cert\") pod \"console-84855f4f96-qp4rb\" (UID: \"bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3\") " pod="openshift-console/console-84855f4f96-qp4rb" May 28 15:15:41.410976 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:41.410944 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3-console-oauth-config\") pod \"console-84855f4f96-qp4rb\" (UID: \"bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3\") " pod="openshift-console/console-84855f4f96-qp4rb" May 28 15:15:41.411134 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:41.410994 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3-console-serving-cert\") pod \"console-84855f4f96-qp4rb\" (UID: \"bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3\") " pod="openshift-console/console-84855f4f96-qp4rb" May 28 15:15:41.411134 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:41.411010 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3-service-ca\") pod \"console-84855f4f96-qp4rb\" (UID: \"bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3\") " pod="openshift-console/console-84855f4f96-qp4rb" May 28 15:15:41.411134 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:41.411035 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3-console-config\") pod \"console-84855f4f96-qp4rb\" (UID: \"bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3\") " pod="openshift-console/console-84855f4f96-qp4rb" May 28 15:15:41.411134 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:41.411073 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ljqwp\" (UniqueName: \"kubernetes.io/projected/bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3-kube-api-access-ljqwp\") pod \"console-84855f4f96-qp4rb\" (UID: \"bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3\") " pod="openshift-console/console-84855f4f96-qp4rb" May 28 15:15:41.411134 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:41.411104 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3-trusted-ca-bundle\") pod \"console-84855f4f96-qp4rb\" (UID: \"bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3\") " pod="openshift-console/console-84855f4f96-qp4rb" May 28 15:15:41.411134 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:41.411128 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3-oauth-serving-cert\") pod \"console-84855f4f96-qp4rb\" (UID: \"bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3\") " pod="openshift-console/console-84855f4f96-qp4rb" May 28 15:15:41.411761 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:41.411737 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3-service-ca\") pod \"console-84855f4f96-qp4rb\" (UID: \"bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3\") " pod="openshift-console/console-84855f4f96-qp4rb" May 28 15:15:41.411918 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:41.411891 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3-console-config\") pod \"console-84855f4f96-qp4rb\" (UID: \"bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3\") " pod="openshift-console/console-84855f4f96-qp4rb" May 28 15:15:41.411966 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:41.411906 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3-oauth-serving-cert\") pod \"console-84855f4f96-qp4rb\" (UID: \"bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3\") " pod="openshift-console/console-84855f4f96-qp4rb" May 28 15:15:41.412197 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:41.412176 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3-trusted-ca-bundle\") pod \"console-84855f4f96-qp4rb\" (UID: \"bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3\") " pod="openshift-console/console-84855f4f96-qp4rb" May 28 15:15:41.413336 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:41.413298 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3-console-oauth-config\") pod \"console-84855f4f96-qp4rb\" (UID: \"bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3\") " pod="openshift-console/console-84855f4f96-qp4rb" May 28 15:15:41.413514 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:41.413474 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3-console-serving-cert\") pod \"console-84855f4f96-qp4rb\" (UID: \"bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3\") " pod="openshift-console/console-84855f4f96-qp4rb" May 28 15:15:41.418798 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:41.418778 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljqwp\" (UniqueName: \"kubernetes.io/projected/bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3-kube-api-access-ljqwp\") pod \"console-84855f4f96-qp4rb\" (UID: \"bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3\") " pod="openshift-console/console-84855f4f96-qp4rb" May 28 15:15:41.529792 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:41.529758 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84855f4f96-qp4rb" May 28 15:15:41.887140 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:41.887089 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84855f4f96-qp4rb"] May 28 15:15:41.890241 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:15:41.890017 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcb3263b_a6f4_46a2_9da8_2d7b09ed29c3.slice/crio-b1e53cf8853dba8133563801f8120d25802cd069892a904009ff55b3d2301ab8 WatchSource:0}: Error finding container b1e53cf8853dba8133563801f8120d25802cd069892a904009ff55b3d2301ab8: Status 404 returned error can't find the container with id b1e53cf8853dba8133563801f8120d25802cd069892a904009ff55b3d2301ab8 May 28 15:15:42.238558 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:42.238451 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84855f4f96-qp4rb" event={"ID":"bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3","Type":"ContainerStarted","Data":"90bd6bd54a7aa4caab7c9098a437efc54c65c56a7b273fe6ba09af3405e7bedd"} May 28 15:15:42.238558 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:42.238507 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84855f4f96-qp4rb" event={"ID":"bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3","Type":"ContainerStarted","Data":"b1e53cf8853dba8133563801f8120d25802cd069892a904009ff55b3d2301ab8"} May 28 15:15:42.256882 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:42.256795 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-84855f4f96-qp4rb" podStartSLOduration=1.25677737 podStartE2EDuration="1.25677737s" podCreationTimestamp="2026-05-28 15:15:41 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 15:15:42.256072037 +0000 UTC m=+493.145059145" watchObservedRunningTime="2026-05-28 15:15:42.25677737 +0000 UTC m=+493.145764455" May 28 15:15:43.245643 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:43.245606 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-k7g4d" event={"ID":"01b6eda7-dbb7-4d91-bb5b-0a78899d6596","Type":"ContainerStarted","Data":"b3a4f575b9cb4764b1bd7583fad60a99ce51fed8e1febd390207143e359926d2"} May 28 15:15:43.246814 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:43.246787 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ksfv9" event={"ID":"899539bf-f715-4843-bc4b-b439e64a5760","Type":"ContainerStarted","Data":"bc4b5944f28b60d92c1e9ef5e39f34aceacf80677dbd749ac3b7ae18f63d1806"} May 28 15:15:43.246916 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:43.246815 2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-ksfv9" podUID="899539bf-f715-4843-bc4b-b439e64a5760" containerName="registry-server" containerID="cri-o://bc4b5944f28b60d92c1e9ef5e39f34aceacf80677dbd749ac3b7ae18f63d1806" gracePeriod=2 May 28 15:15:43.261009 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:43.260970 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-k7g4d" podStartSLOduration=1.478360328 podStartE2EDuration="3.260953595s" podCreationTimestamp="2026-05-28 15:15:40 +0000 UTC" firstStartedPulling="2026-05-28 15:15:40.699644344 +0000 UTC m=+491.588631410" lastFinishedPulling="2026-05-28 15:15:42.482237614 +0000 UTC m=+493.371224677" observedRunningTime="2026-05-28 15:15:43.260044003 +0000 UTC m=+494.149031088" watchObservedRunningTime="2026-05-28 15:15:43.260953595 +0000 UTC m=+494.149940679" May 28 15:15:43.275891 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:43.275846 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-ksfv9" podStartSLOduration=1.8953224560000002 podStartE2EDuration="4.275832534s" podCreationTimestamp="2026-05-28 15:15:39 +0000 UTC" firstStartedPulling="2026-05-28 15:15:40.098609641 +0000 UTC m=+490.987596720" lastFinishedPulling="2026-05-28 15:15:42.479119733 +0000 UTC m=+493.368106798" observedRunningTime="2026-05-28 15:15:43.27420099 +0000 UTC m=+494.163188077" watchObservedRunningTime="2026-05-28 15:15:43.275832534 +0000 UTC m=+494.164819617" May 28 15:15:43.490551 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:43.490527 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ksfv9" May 28 15:15:43.529627 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:43.529543 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ht5rs\" (UniqueName: \"kubernetes.io/projected/899539bf-f715-4843-bc4b-b439e64a5760-kube-api-access-ht5rs\") pod \"899539bf-f715-4843-bc4b-b439e64a5760\" (UID: \"899539bf-f715-4843-bc4b-b439e64a5760\") " May 28 15:15:43.531689 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:43.531661 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/899539bf-f715-4843-bc4b-b439e64a5760-kube-api-access-ht5rs" (OuterVolumeSpecName: "kube-api-access-ht5rs") pod "899539bf-f715-4843-bc4b-b439e64a5760" (UID: "899539bf-f715-4843-bc4b-b439e64a5760"). InnerVolumeSpecName "kube-api-access-ht5rs". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 15:15:43.630687 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:43.630655 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ht5rs\" (UniqueName: \"kubernetes.io/projected/899539bf-f715-4843-bc4b-b439e64a5760-kube-api-access-ht5rs\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:15:44.251446 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:44.251412 2559 generic.go:358] "Generic (PLEG): container finished" podID="899539bf-f715-4843-bc4b-b439e64a5760" containerID="bc4b5944f28b60d92c1e9ef5e39f34aceacf80677dbd749ac3b7ae18f63d1806" exitCode=0 May 28 15:15:44.251916 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:44.251469 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ksfv9" May 28 15:15:44.251916 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:44.251513 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ksfv9" event={"ID":"899539bf-f715-4843-bc4b-b439e64a5760","Type":"ContainerDied","Data":"bc4b5944f28b60d92c1e9ef5e39f34aceacf80677dbd749ac3b7ae18f63d1806"} May 28 15:15:44.251916 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:44.251554 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ksfv9" event={"ID":"899539bf-f715-4843-bc4b-b439e64a5760","Type":"ContainerDied","Data":"c90d2a05df00d233c61551947997d2cf6895e0712efa0747410b824eb1fcd78a"} May 28 15:15:44.251916 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:44.251571 2559 scope.go:117] "RemoveContainer" containerID="bc4b5944f28b60d92c1e9ef5e39f34aceacf80677dbd749ac3b7ae18f63d1806" May 28 15:15:44.259853 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:44.259838 2559 scope.go:117] "RemoveContainer" containerID="bc4b5944f28b60d92c1e9ef5e39f34aceacf80677dbd749ac3b7ae18f63d1806" May 28 15:15:44.260081 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:15:44.260061 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc4b5944f28b60d92c1e9ef5e39f34aceacf80677dbd749ac3b7ae18f63d1806\": container with ID starting with bc4b5944f28b60d92c1e9ef5e39f34aceacf80677dbd749ac3b7ae18f63d1806 not found: ID does not exist" containerID="bc4b5944f28b60d92c1e9ef5e39f34aceacf80677dbd749ac3b7ae18f63d1806" May 28 15:15:44.260144 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:44.260087 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc4b5944f28b60d92c1e9ef5e39f34aceacf80677dbd749ac3b7ae18f63d1806"} err="failed to get container status \"bc4b5944f28b60d92c1e9ef5e39f34aceacf80677dbd749ac3b7ae18f63d1806\": rpc error: code = NotFound desc = could not find container \"bc4b5944f28b60d92c1e9ef5e39f34aceacf80677dbd749ac3b7ae18f63d1806\": container with ID starting with bc4b5944f28b60d92c1e9ef5e39f34aceacf80677dbd749ac3b7ae18f63d1806 not found: ID does not exist" May 28 15:15:44.267754 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:44.267729 2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ksfv9"] May 28 15:15:44.272982 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:44.272963 2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ksfv9"] May 28 15:15:45.712834 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:45.712803 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="899539bf-f715-4843-bc4b-b439e64a5760" path="/var/lib/kubelet/pods/899539bf-f715-4843-bc4b-b439e64a5760/volumes" May 28 15:15:50.550212 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:50.550171 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-k7g4d" May 28 15:15:50.550685 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:50.550233 2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-k7g4d" May 28 15:15:50.571573 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:50.571547 2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-k7g4d" May 28 15:15:51.296288 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:51.296261 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-k7g4d" May 28 15:15:51.530661 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:51.530610 2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-84855f4f96-qp4rb" May 28 15:15:51.530661 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:51.530663 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-84855f4f96-qp4rb" May 28 15:15:51.535326 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:51.535303 2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-84855f4f96-qp4rb" May 28 15:15:52.282150 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:52.282123 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-84855f4f96-qp4rb" May 28 15:15:52.334734 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:52.334704 2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6448c57698-9bw6c"] May 28 15:15:55.266848 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:55.266809 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732246x"] May 28 15:15:55.267280 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:55.267140 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="899539bf-f715-4843-bc4b-b439e64a5760" containerName="registry-server" May 28 15:15:55.267280 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:55.267151 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="899539bf-f715-4843-bc4b-b439e64a5760" containerName="registry-server" May 28 15:15:55.267280 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:55.267220 2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="899539bf-f715-4843-bc4b-b439e64a5760" containerName="registry-server" May 28 15:15:55.272567 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:55.272548 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732246x" May 28 15:15:55.276089 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:55.276059 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-jxjrw\"" May 28 15:15:55.277352 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:55.277330 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732246x"] May 28 15:15:55.327763 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:55.327731 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bf2518bc-3be2-4f08-9567-2a4e4f6aa833-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732246x\" (UID: \"bf2518bc-3be2-4f08-9567-2a4e4f6aa833\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732246x" May 28 15:15:55.327898 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:55.327777 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bf2518bc-3be2-4f08-9567-2a4e4f6aa833-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732246x\" (UID: \"bf2518bc-3be2-4f08-9567-2a4e4f6aa833\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732246x" May 28 15:15:55.327936 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:55.327893 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-244j9\" (UniqueName: \"kubernetes.io/projected/bf2518bc-3be2-4f08-9567-2a4e4f6aa833-kube-api-access-244j9\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732246x\" (UID: \"bf2518bc-3be2-4f08-9567-2a4e4f6aa833\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732246x" May 28 15:15:55.428751 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:55.428720 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-244j9\" (UniqueName: \"kubernetes.io/projected/bf2518bc-3be2-4f08-9567-2a4e4f6aa833-kube-api-access-244j9\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732246x\" (UID: \"bf2518bc-3be2-4f08-9567-2a4e4f6aa833\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732246x" May 28 15:15:55.428904 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:55.428776 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bf2518bc-3be2-4f08-9567-2a4e4f6aa833-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732246x\" (UID: \"bf2518bc-3be2-4f08-9567-2a4e4f6aa833\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732246x" May 28 15:15:55.428904 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:55.428807 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bf2518bc-3be2-4f08-9567-2a4e4f6aa833-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732246x\" (UID: \"bf2518bc-3be2-4f08-9567-2a4e4f6aa833\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732246x" May 28 15:15:55.429142 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:55.429126 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bf2518bc-3be2-4f08-9567-2a4e4f6aa833-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732246x\" (UID: \"bf2518bc-3be2-4f08-9567-2a4e4f6aa833\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732246x" May 28 15:15:55.429186 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:55.429163 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bf2518bc-3be2-4f08-9567-2a4e4f6aa833-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732246x\" (UID: \"bf2518bc-3be2-4f08-9567-2a4e4f6aa833\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732246x" May 28 15:15:55.436862 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:55.436837 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-244j9\" (UniqueName: \"kubernetes.io/projected/bf2518bc-3be2-4f08-9567-2a4e4f6aa833-kube-api-access-244j9\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732246x\" (UID: \"bf2518bc-3be2-4f08-9567-2a4e4f6aa833\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732246x" May 28 15:15:55.582759 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:55.582693 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732246x" May 28 15:15:55.706897 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:55.706874 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732246x"] May 28 15:15:55.709044 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:15:55.709015 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf2518bc_3be2_4f08_9567_2a4e4f6aa833.slice/crio-064ad61121ccabe58d7fd37f8ce91756adf53ba92095d50c0c94dfe8662be083 WatchSource:0}: Error finding container 064ad61121ccabe58d7fd37f8ce91756adf53ba92095d50c0c94dfe8662be083: Status 404 returned error can't find the container with id 064ad61121ccabe58d7fd37f8ce91756adf53ba92095d50c0c94dfe8662be083 May 28 15:15:55.868555 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:55.868445 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bz2jl"] May 28 15:15:55.871028 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:55.871009 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bz2jl" May 28 15:15:55.879394 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:55.879367 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bz2jl"] May 28 15:15:55.933852 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:55.933826 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1cd08bac-f55e-4be0-b887-a50543cbba4b-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bz2jl\" (UID: \"1cd08bac-f55e-4be0-b887-a50543cbba4b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bz2jl" May 28 15:15:55.933978 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:55.933886 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tldh\" (UniqueName: \"kubernetes.io/projected/1cd08bac-f55e-4be0-b887-a50543cbba4b-kube-api-access-4tldh\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bz2jl\" (UID: \"1cd08bac-f55e-4be0-b887-a50543cbba4b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bz2jl" May 28 15:15:55.933978 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:55.933946 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1cd08bac-f55e-4be0-b887-a50543cbba4b-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bz2jl\" (UID: \"1cd08bac-f55e-4be0-b887-a50543cbba4b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bz2jl" May 28 15:15:56.034944 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:56.034916 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4tldh\" (UniqueName: \"kubernetes.io/projected/1cd08bac-f55e-4be0-b887-a50543cbba4b-kube-api-access-4tldh\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bz2jl\" (UID: \"1cd08bac-f55e-4be0-b887-a50543cbba4b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bz2jl" May 28 15:15:56.035053 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:56.034959 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1cd08bac-f55e-4be0-b887-a50543cbba4b-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bz2jl\" (UID: \"1cd08bac-f55e-4be0-b887-a50543cbba4b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bz2jl" May 28 15:15:56.035113 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:56.035093 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1cd08bac-f55e-4be0-b887-a50543cbba4b-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bz2jl\" (UID: \"1cd08bac-f55e-4be0-b887-a50543cbba4b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bz2jl" May 28 15:15:56.035306 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:56.035292 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1cd08bac-f55e-4be0-b887-a50543cbba4b-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bz2jl\" (UID: \"1cd08bac-f55e-4be0-b887-a50543cbba4b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bz2jl" May 28 15:15:56.035391 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:56.035374 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1cd08bac-f55e-4be0-b887-a50543cbba4b-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bz2jl\" (UID: \"1cd08bac-f55e-4be0-b887-a50543cbba4b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bz2jl" May 28 15:15:56.042945 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:56.042922 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tldh\" (UniqueName: \"kubernetes.io/projected/1cd08bac-f55e-4be0-b887-a50543cbba4b-kube-api-access-4tldh\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bz2jl\" (UID: \"1cd08bac-f55e-4be0-b887-a50543cbba4b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bz2jl" May 28 15:15:56.179942 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:56.179876 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bz2jl" May 28 15:15:56.292667 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:56.292633 2559 generic.go:358] "Generic (PLEG): container finished" podID="bf2518bc-3be2-4f08-9567-2a4e4f6aa833" containerID="af9a634ec6e980613c66d049b269c6bea0440b27b0aae56f332aa781dd497cb9" exitCode=0 May 28 15:15:56.292988 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:56.292675 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732246x" event={"ID":"bf2518bc-3be2-4f08-9567-2a4e4f6aa833","Type":"ContainerDied","Data":"af9a634ec6e980613c66d049b269c6bea0440b27b0aae56f332aa781dd497cb9"} May 28 15:15:56.292988 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:56.292698 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732246x" event={"ID":"bf2518bc-3be2-4f08-9567-2a4e4f6aa833","Type":"ContainerStarted","Data":"064ad61121ccabe58d7fd37f8ce91756adf53ba92095d50c0c94dfe8662be083"} May 28 15:15:56.299108 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:56.299084 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bz2jl"] May 28 15:15:56.301423 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:15:56.301393 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cd08bac_f55e_4be0_b887_a50543cbba4b.slice/crio-6f47bc0f4fb34676f08ce5302ceba152e7f9990d94f5aba8204893281a9e1e8f WatchSource:0}: Error finding container 6f47bc0f4fb34676f08ce5302ceba152e7f9990d94f5aba8204893281a9e1e8f: Status 404 returned error can't find the container with id 6f47bc0f4fb34676f08ce5302ceba152e7f9990d94f5aba8204893281a9e1e8f May 28 15:15:56.864887 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:56.864856 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759btbj4"] May 28 15:15:56.867741 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:56.867716 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759btbj4" May 28 15:15:56.875673 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:56.875644 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759btbj4"] May 28 15:15:56.943713 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:56.943690 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1f2fed77-e2ed-4149-86f4-767f909314fc-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759btbj4\" (UID: \"1f2fed77-e2ed-4149-86f4-767f909314fc\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759btbj4" May 28 15:15:56.943811 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:56.943738 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rcdf\" (UniqueName: \"kubernetes.io/projected/1f2fed77-e2ed-4149-86f4-767f909314fc-kube-api-access-2rcdf\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759btbj4\" (UID: \"1f2fed77-e2ed-4149-86f4-767f909314fc\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759btbj4" May 28 15:15:56.943811 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:56.943797 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1f2fed77-e2ed-4149-86f4-767f909314fc-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759btbj4\" (UID: \"1f2fed77-e2ed-4149-86f4-767f909314fc\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759btbj4" May 28 15:15:57.044996 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:57.044972 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1f2fed77-e2ed-4149-86f4-767f909314fc-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759btbj4\" (UID: \"1f2fed77-e2ed-4149-86f4-767f909314fc\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759btbj4" May 28 15:15:57.045085 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:57.045056 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1f2fed77-e2ed-4149-86f4-767f909314fc-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759btbj4\" (UID: \"1f2fed77-e2ed-4149-86f4-767f909314fc\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759btbj4" May 28 15:15:57.045127 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:57.045097 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2rcdf\" (UniqueName: \"kubernetes.io/projected/1f2fed77-e2ed-4149-86f4-767f909314fc-kube-api-access-2rcdf\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759btbj4\" (UID: \"1f2fed77-e2ed-4149-86f4-767f909314fc\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759btbj4" May 28 15:15:57.045266 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:57.045248 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1f2fed77-e2ed-4149-86f4-767f909314fc-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759btbj4\" (UID: \"1f2fed77-e2ed-4149-86f4-767f909314fc\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759btbj4" May 28 15:15:57.045376 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:57.045342 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1f2fed77-e2ed-4149-86f4-767f909314fc-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759btbj4\" (UID: \"1f2fed77-e2ed-4149-86f4-767f909314fc\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759btbj4" May 28 15:15:57.053331 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:57.053307 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rcdf\" (UniqueName: \"kubernetes.io/projected/1f2fed77-e2ed-4149-86f4-767f909314fc-kube-api-access-2rcdf\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759btbj4\" (UID: \"1f2fed77-e2ed-4149-86f4-767f909314fc\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759btbj4" May 28 15:15:57.180120 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:57.180023 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759btbj4" May 28 15:15:57.297921 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:57.297884 2559 generic.go:358] "Generic (PLEG): container finished" podID="1cd08bac-f55e-4be0-b887-a50543cbba4b" containerID="9504d145a78d60c15a0d6c509cb3cb99cf584015306c6bf47a558b8a553e02b0" exitCode=0 May 28 15:15:57.298357 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:57.297963 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bz2jl" event={"ID":"1cd08bac-f55e-4be0-b887-a50543cbba4b","Type":"ContainerDied","Data":"9504d145a78d60c15a0d6c509cb3cb99cf584015306c6bf47a558b8a553e02b0"} May 28 15:15:57.298357 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:57.298005 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bz2jl" event={"ID":"1cd08bac-f55e-4be0-b887-a50543cbba4b","Type":"ContainerStarted","Data":"6f47bc0f4fb34676f08ce5302ceba152e7f9990d94f5aba8204893281a9e1e8f"} May 28 15:15:57.299707 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:57.299685 2559 generic.go:358] "Generic (PLEG): container finished" podID="bf2518bc-3be2-4f08-9567-2a4e4f6aa833" containerID="42fe18b49fe4a5c46634cafb3fbc10d9d1433d0a6903c1b10fc46eb683ef8b60" exitCode=0 May 28 15:15:57.299797 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:57.299770 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732246x" event={"ID":"bf2518bc-3be2-4f08-9567-2a4e4f6aa833","Type":"ContainerDied","Data":"42fe18b49fe4a5c46634cafb3fbc10d9d1433d0a6903c1b10fc46eb683ef8b60"} May 28 15:15:57.304278 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:57.304259 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759btbj4"] May 28 15:15:57.306606 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:15:57.306584 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f2fed77_e2ed_4149_86f4_767f909314fc.slice/crio-4e3f1213be1820d42fbe44f3c9f14de1234eaa10933fc074c908ed7157d317c0 WatchSource:0}: Error finding container 4e3f1213be1820d42fbe44f3c9f14de1234eaa10933fc074c908ed7157d317c0: Status 404 returned error can't find the container with id 4e3f1213be1820d42fbe44f3c9f14de1234eaa10933fc074c908ed7157d317c0 May 28 15:15:58.305009 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:58.304973 2559 generic.go:358] "Generic (PLEG): container finished" podID="1cd08bac-f55e-4be0-b887-a50543cbba4b" containerID="8c83dfeb89638c71f88c9852989ea40311b641bac51efc91e7e43d985ae7d23f" exitCode=0 May 28 15:15:58.305346 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:58.305022 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bz2jl" event={"ID":"1cd08bac-f55e-4be0-b887-a50543cbba4b","Type":"ContainerDied","Data":"8c83dfeb89638c71f88c9852989ea40311b641bac51efc91e7e43d985ae7d23f"} May 28 15:15:58.306395 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:58.306369 2559 generic.go:358] "Generic (PLEG): container finished" podID="1f2fed77-e2ed-4149-86f4-767f909314fc" containerID="a26286fe5893d887ca3d5f9e1f8783b3332d0987c6cc0648c3ebbd85afea8b3a" exitCode=0 May 28 15:15:58.306519 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:58.306454 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759btbj4" event={"ID":"1f2fed77-e2ed-4149-86f4-767f909314fc","Type":"ContainerDied","Data":"a26286fe5893d887ca3d5f9e1f8783b3332d0987c6cc0648c3ebbd85afea8b3a"} May 28 15:15:58.306519 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:58.306497 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759btbj4" event={"ID":"1f2fed77-e2ed-4149-86f4-767f909314fc","Type":"ContainerStarted","Data":"4e3f1213be1820d42fbe44f3c9f14de1234eaa10933fc074c908ed7157d317c0"} May 28 15:15:58.308519 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:58.308390 2559 generic.go:358] "Generic (PLEG): container finished" podID="bf2518bc-3be2-4f08-9567-2a4e4f6aa833" containerID="0526758e334d962275af233a642e8b7b5a565cfbbdd63b9a91ac895234353c8b" exitCode=0 May 28 15:15:58.308519 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:58.308463 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732246x" event={"ID":"bf2518bc-3be2-4f08-9567-2a4e4f6aa833","Type":"ContainerDied","Data":"0526758e334d962275af233a642e8b7b5a565cfbbdd63b9a91ac895234353c8b"} May 28 15:15:59.313440 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:59.313403 2559 generic.go:358] "Generic (PLEG): container finished" podID="1cd08bac-f55e-4be0-b887-a50543cbba4b" containerID="b9ee6af658098f342360516ca503e6bb47abd9f6ae929e0dc7f8709bd312c309" exitCode=0 May 28 15:15:59.313872 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:59.313450 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bz2jl" event={"ID":"1cd08bac-f55e-4be0-b887-a50543cbba4b","Type":"ContainerDied","Data":"b9ee6af658098f342360516ca503e6bb47abd9f6ae929e0dc7f8709bd312c309"} May 28 15:15:59.315027 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:59.315005 2559 generic.go:358] "Generic (PLEG): container finished" podID="1f2fed77-e2ed-4149-86f4-767f909314fc" containerID="43e71cd3f79046384be552fe69d38778d072b230a3f571367840072a346e8456" exitCode=0 May 28 15:15:59.315154 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:59.315080 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759btbj4" event={"ID":"1f2fed77-e2ed-4149-86f4-767f909314fc","Type":"ContainerDied","Data":"43e71cd3f79046384be552fe69d38778d072b230a3f571367840072a346e8456"} May 28 15:15:59.450084 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:59.450062 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732246x" May 28 15:15:59.566580 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:59.566550 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bf2518bc-3be2-4f08-9567-2a4e4f6aa833-bundle\") pod \"bf2518bc-3be2-4f08-9567-2a4e4f6aa833\" (UID: \"bf2518bc-3be2-4f08-9567-2a4e4f6aa833\") " May 28 15:15:59.566703 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:59.566601 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bf2518bc-3be2-4f08-9567-2a4e4f6aa833-util\") pod \"bf2518bc-3be2-4f08-9567-2a4e4f6aa833\" (UID: \"bf2518bc-3be2-4f08-9567-2a4e4f6aa833\") " May 28 15:15:59.566703 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:59.566652 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-244j9\" (UniqueName: \"kubernetes.io/projected/bf2518bc-3be2-4f08-9567-2a4e4f6aa833-kube-api-access-244j9\") pod \"bf2518bc-3be2-4f08-9567-2a4e4f6aa833\" (UID: \"bf2518bc-3be2-4f08-9567-2a4e4f6aa833\") " May 28 15:15:59.567306 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:59.567278 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf2518bc-3be2-4f08-9567-2a4e4f6aa833-bundle" (OuterVolumeSpecName: "bundle") pod "bf2518bc-3be2-4f08-9567-2a4e4f6aa833" (UID: "bf2518bc-3be2-4f08-9567-2a4e4f6aa833"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 28 15:15:59.568794 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:59.568767 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf2518bc-3be2-4f08-9567-2a4e4f6aa833-kube-api-access-244j9" (OuterVolumeSpecName: "kube-api-access-244j9") pod "bf2518bc-3be2-4f08-9567-2a4e4f6aa833" (UID: "bf2518bc-3be2-4f08-9567-2a4e4f6aa833"). InnerVolumeSpecName "kube-api-access-244j9". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 15:15:59.572064 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:59.572044 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf2518bc-3be2-4f08-9567-2a4e4f6aa833-util" (OuterVolumeSpecName: "util") pod "bf2518bc-3be2-4f08-9567-2a4e4f6aa833" (UID: "bf2518bc-3be2-4f08-9567-2a4e4f6aa833"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 28 15:15:59.667998 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:59.667974 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-244j9\" (UniqueName: \"kubernetes.io/projected/bf2518bc-3be2-4f08-9567-2a4e4f6aa833-kube-api-access-244j9\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:15:59.667998 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:59.667995 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bf2518bc-3be2-4f08-9567-2a4e4f6aa833-bundle\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:15:59.668143 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:15:59.668004 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bf2518bc-3be2-4f08-9567-2a4e4f6aa833-util\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:16:00.321760 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:00.321731 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732246x" May 28 15:16:00.322209 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:00.321736 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732246x" event={"ID":"bf2518bc-3be2-4f08-9567-2a4e4f6aa833","Type":"ContainerDied","Data":"064ad61121ccabe58d7fd37f8ce91756adf53ba92095d50c0c94dfe8662be083"} May 28 15:16:00.322209 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:00.321827 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="064ad61121ccabe58d7fd37f8ce91756adf53ba92095d50c0c94dfe8662be083" May 28 15:16:00.323649 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:00.323622 2559 generic.go:358] "Generic (PLEG): container finished" podID="1f2fed77-e2ed-4149-86f4-767f909314fc" containerID="f23828828de589b24bf7bd6c53167345d169988e3e33f2dfe1c83cc58b271b0b" exitCode=0 May 28 15:16:00.323772 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:00.323708 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759btbj4" event={"ID":"1f2fed77-e2ed-4149-86f4-767f909314fc","Type":"ContainerDied","Data":"f23828828de589b24bf7bd6c53167345d169988e3e33f2dfe1c83cc58b271b0b"} May 28 15:16:00.449076 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:00.449048 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bz2jl" May 28 15:16:00.574886 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:00.574831 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1cd08bac-f55e-4be0-b887-a50543cbba4b-util\") pod \"1cd08bac-f55e-4be0-b887-a50543cbba4b\" (UID: \"1cd08bac-f55e-4be0-b887-a50543cbba4b\") " May 28 15:16:00.574992 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:00.574937 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1cd08bac-f55e-4be0-b887-a50543cbba4b-bundle\") pod \"1cd08bac-f55e-4be0-b887-a50543cbba4b\" (UID: \"1cd08bac-f55e-4be0-b887-a50543cbba4b\") " May 28 15:16:00.574992 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:00.574971 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4tldh\" (UniqueName: \"kubernetes.io/projected/1cd08bac-f55e-4be0-b887-a50543cbba4b-kube-api-access-4tldh\") pod \"1cd08bac-f55e-4be0-b887-a50543cbba4b\" (UID: \"1cd08bac-f55e-4be0-b887-a50543cbba4b\") " May 28 15:16:00.575360 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:00.575338 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cd08bac-f55e-4be0-b887-a50543cbba4b-bundle" (OuterVolumeSpecName: "bundle") pod "1cd08bac-f55e-4be0-b887-a50543cbba4b" (UID: "1cd08bac-f55e-4be0-b887-a50543cbba4b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 28 15:16:00.576892 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:00.576866 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cd08bac-f55e-4be0-b887-a50543cbba4b-kube-api-access-4tldh" (OuterVolumeSpecName: "kube-api-access-4tldh") pod "1cd08bac-f55e-4be0-b887-a50543cbba4b" (UID: "1cd08bac-f55e-4be0-b887-a50543cbba4b"). InnerVolumeSpecName "kube-api-access-4tldh". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 15:16:00.580292 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:00.580272 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cd08bac-f55e-4be0-b887-a50543cbba4b-util" (OuterVolumeSpecName: "util") pod "1cd08bac-f55e-4be0-b887-a50543cbba4b" (UID: "1cd08bac-f55e-4be0-b887-a50543cbba4b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 28 15:16:00.676390 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:00.676369 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1cd08bac-f55e-4be0-b887-a50543cbba4b-bundle\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:16:00.676390 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:00.676390 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4tldh\" (UniqueName: \"kubernetes.io/projected/1cd08bac-f55e-4be0-b887-a50543cbba4b-kube-api-access-4tldh\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:16:00.676521 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:00.676399 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1cd08bac-f55e-4be0-b887-a50543cbba4b-util\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:16:01.328308 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:01.328283 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bz2jl" May 28 15:16:01.328684 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:01.328326 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bz2jl" event={"ID":"1cd08bac-f55e-4be0-b887-a50543cbba4b","Type":"ContainerDied","Data":"6f47bc0f4fb34676f08ce5302ceba152e7f9990d94f5aba8204893281a9e1e8f"} May 28 15:16:01.328684 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:01.328358 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6f47bc0f4fb34676f08ce5302ceba152e7f9990d94f5aba8204893281a9e1e8f" May 28 15:16:01.456104 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:01.456080 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759btbj4" May 28 15:16:01.583778 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:01.583698 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1f2fed77-e2ed-4149-86f4-767f909314fc-bundle\") pod \"1f2fed77-e2ed-4149-86f4-767f909314fc\" (UID: \"1f2fed77-e2ed-4149-86f4-767f909314fc\") " May 28 15:16:01.583928 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:01.583810 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2rcdf\" (UniqueName: \"kubernetes.io/projected/1f2fed77-e2ed-4149-86f4-767f909314fc-kube-api-access-2rcdf\") pod \"1f2fed77-e2ed-4149-86f4-767f909314fc\" (UID: \"1f2fed77-e2ed-4149-86f4-767f909314fc\") " May 28 15:16:01.583928 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:01.583834 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1f2fed77-e2ed-4149-86f4-767f909314fc-util\") pod \"1f2fed77-e2ed-4149-86f4-767f909314fc\" (UID: \"1f2fed77-e2ed-4149-86f4-767f909314fc\") " May 28 15:16:01.584210 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:01.584185 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f2fed77-e2ed-4149-86f4-767f909314fc-bundle" (OuterVolumeSpecName: "bundle") pod "1f2fed77-e2ed-4149-86f4-767f909314fc" (UID: "1f2fed77-e2ed-4149-86f4-767f909314fc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 28 15:16:01.585895 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:01.585864 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f2fed77-e2ed-4149-86f4-767f909314fc-kube-api-access-2rcdf" (OuterVolumeSpecName: "kube-api-access-2rcdf") pod "1f2fed77-e2ed-4149-86f4-767f909314fc" (UID: "1f2fed77-e2ed-4149-86f4-767f909314fc"). InnerVolumeSpecName "kube-api-access-2rcdf". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 15:16:01.588738 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:01.588713 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f2fed77-e2ed-4149-86f4-767f909314fc-util" (OuterVolumeSpecName: "util") pod "1f2fed77-e2ed-4149-86f4-767f909314fc" (UID: "1f2fed77-e2ed-4149-86f4-767f909314fc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 28 15:16:01.684514 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:01.684466 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2rcdf\" (UniqueName: \"kubernetes.io/projected/1f2fed77-e2ed-4149-86f4-767f909314fc-kube-api-access-2rcdf\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:16:01.684514 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:01.684510 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1f2fed77-e2ed-4149-86f4-767f909314fc-util\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:16:01.684707 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:01.684524 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1f2fed77-e2ed-4149-86f4-767f909314fc-bundle\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:16:02.333244 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:02.333220 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759btbj4" May 28 15:16:02.333629 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:02.333206 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759btbj4" event={"ID":"1f2fed77-e2ed-4149-86f4-767f909314fc","Type":"ContainerDied","Data":"4e3f1213be1820d42fbe44f3c9f14de1234eaa10933fc074c908ed7157d317c0"} May 28 15:16:02.333629 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:02.333331 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4e3f1213be1820d42fbe44f3c9f14de1234eaa10933fc074c908ed7157d317c0" May 28 15:16:10.095028 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:10.094991 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-mrdzf"] May 28 15:16:10.095426 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:10.095332 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf2518bc-3be2-4f08-9567-2a4e4f6aa833" containerName="util" May 28 15:16:10.095426 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:10.095342 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf2518bc-3be2-4f08-9567-2a4e4f6aa833" containerName="util" May 28 15:16:10.095426 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:10.095352 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf2518bc-3be2-4f08-9567-2a4e4f6aa833" containerName="pull" May 28 15:16:10.095426 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:10.095357 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf2518bc-3be2-4f08-9567-2a4e4f6aa833" containerName="pull" May 28 15:16:10.095426 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:10.095366 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cd08bac-f55e-4be0-b887-a50543cbba4b" containerName="util" May 28 15:16:10.095426 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:10.095372 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cd08bac-f55e-4be0-b887-a50543cbba4b" containerName="util" May 28 15:16:10.095426 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:10.095379 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f2fed77-e2ed-4149-86f4-767f909314fc" containerName="extract" May 28 15:16:10.095426 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:10.095385 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f2fed77-e2ed-4149-86f4-767f909314fc" containerName="extract" May 28 15:16:10.095426 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:10.095396 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cd08bac-f55e-4be0-b887-a50543cbba4b" containerName="pull" May 28 15:16:10.095426 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:10.095401 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cd08bac-f55e-4be0-b887-a50543cbba4b" containerName="pull" May 28 15:16:10.095426 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:10.095407 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f2fed77-e2ed-4149-86f4-767f909314fc" containerName="pull" May 28 15:16:10.095426 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:10.095411 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f2fed77-e2ed-4149-86f4-767f909314fc" containerName="pull" May 28 15:16:10.095426 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:10.095424 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f2fed77-e2ed-4149-86f4-767f909314fc" containerName="util" May 28 15:16:10.095426 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:10.095429 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f2fed77-e2ed-4149-86f4-767f909314fc" containerName="util" May 28 15:16:10.095426 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:10.095434 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cd08bac-f55e-4be0-b887-a50543cbba4b" containerName="extract" May 28 15:16:10.095878 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:10.095439 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cd08bac-f55e-4be0-b887-a50543cbba4b" containerName="extract" May 28 15:16:10.095878 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:10.095447 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf2518bc-3be2-4f08-9567-2a4e4f6aa833" containerName="extract" May 28 15:16:10.095878 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:10.095452 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf2518bc-3be2-4f08-9567-2a4e4f6aa833" containerName="extract" May 28 15:16:10.095878 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:10.095515 2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="bf2518bc-3be2-4f08-9567-2a4e4f6aa833" containerName="extract" May 28 15:16:10.095878 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:10.095527 2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="1f2fed77-e2ed-4149-86f4-767f909314fc" containerName="extract" May 28 15:16:10.095878 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:10.095534 2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="1cd08bac-f55e-4be0-b887-a50543cbba4b" containerName="extract" May 28 15:16:10.099778 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:10.099762 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-mrdzf" May 28 15:16:10.102732 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:10.102711 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" May 28 15:16:10.102832 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:10.102781 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-b5qfw\"" May 28 15:16:10.109111 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:10.109090 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-mrdzf"] May 28 15:16:10.249720 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:10.249689 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-485gr\" (UniqueName: \"kubernetes.io/projected/f2253641-dc2e-4536-9ac0-9999e647ea53-kube-api-access-485gr\") pod \"dns-operator-controller-manager-648d5c98bc-mrdzf\" (UID: \"f2253641-dc2e-4536-9ac0-9999e647ea53\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-mrdzf" May 28 15:16:10.351096 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:10.351059 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-485gr\" (UniqueName: \"kubernetes.io/projected/f2253641-dc2e-4536-9ac0-9999e647ea53-kube-api-access-485gr\") pod \"dns-operator-controller-manager-648d5c98bc-mrdzf\" (UID: \"f2253641-dc2e-4536-9ac0-9999e647ea53\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-mrdzf" May 28 15:16:10.364553 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:10.364528 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-485gr\" (UniqueName: \"kubernetes.io/projected/f2253641-dc2e-4536-9ac0-9999e647ea53-kube-api-access-485gr\") pod \"dns-operator-controller-manager-648d5c98bc-mrdzf\" (UID: \"f2253641-dc2e-4536-9ac0-9999e647ea53\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-mrdzf" May 28 15:16:10.410433 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:10.410415 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-mrdzf" May 28 15:16:10.538767 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:10.538693 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-mrdzf"] May 28 15:16:10.541106 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:16:10.541080 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2253641_dc2e_4536_9ac0_9999e647ea53.slice/crio-b2753de6731a26495a6850ecc763e701dabef091bf2e92686191cb8545cc2789 WatchSource:0}: Error finding container b2753de6731a26495a6850ecc763e701dabef091bf2e92686191cb8545cc2789: Status 404 returned error can't find the container with id b2753de6731a26495a6850ecc763e701dabef091bf2e92686191cb8545cc2789 May 28 15:16:11.364272 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:11.364242 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-mrdzf" event={"ID":"f2253641-dc2e-4536-9ac0-9999e647ea53","Type":"ContainerStarted","Data":"b2753de6731a26495a6850ecc763e701dabef091bf2e92686191cb8545cc2789"} May 28 15:16:13.379409 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:13.379322 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-mrdzf" event={"ID":"f2253641-dc2e-4536-9ac0-9999e647ea53","Type":"ContainerStarted","Data":"9ce175026f981d395af1937dd5996323f62977e0405ed91226d7913be7e0b8cd"} May 28 15:16:13.379791 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:13.379449 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-mrdzf" May 28 15:16:13.395834 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:13.395792 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-mrdzf" podStartSLOduration=0.927037236 podStartE2EDuration="3.395779858s" podCreationTimestamp="2026-05-28 15:16:10 +0000 UTC" firstStartedPulling="2026-05-28 15:16:10.542954742 +0000 UTC m=+521.431941804" lastFinishedPulling="2026-05-28 15:16:13.011697361 +0000 UTC m=+523.900684426" observedRunningTime="2026-05-28 15:16:13.395065144 +0000 UTC m=+524.284052247" watchObservedRunningTime="2026-05-28 15:16:13.395779858 +0000 UTC m=+524.284766939" May 28 15:16:17.357575 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:17.357532 2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-6448c57698-9bw6c" podUID="bf978273-5024-4c57-baca-e30abd850138" containerName="console" containerID="cri-o://5cf3dfc00e85c24ed0ff6f406d8ed206391e2249a5d41ab0ceffa41d2663e1aa" gracePeriod=15 May 28 15:16:17.605154 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:17.605131 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6448c57698-9bw6c_bf978273-5024-4c57-baca-e30abd850138/console/0.log" May 28 15:16:17.605265 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:17.605188 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6448c57698-9bw6c" May 28 15:16:17.715893 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:17.715816 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2n282\" (UniqueName: \"kubernetes.io/projected/bf978273-5024-4c57-baca-e30abd850138-kube-api-access-2n282\") pod \"bf978273-5024-4c57-baca-e30abd850138\" (UID: \"bf978273-5024-4c57-baca-e30abd850138\") " May 28 15:16:17.715893 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:17.715856 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bf978273-5024-4c57-baca-e30abd850138-console-config\") pod \"bf978273-5024-4c57-baca-e30abd850138\" (UID: \"bf978273-5024-4c57-baca-e30abd850138\") " May 28 15:16:17.715893 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:17.715875 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf978273-5024-4c57-baca-e30abd850138-trusted-ca-bundle\") pod \"bf978273-5024-4c57-baca-e30abd850138\" (UID: \"bf978273-5024-4c57-baca-e30abd850138\") " May 28 15:16:17.716152 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:17.715900 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bf978273-5024-4c57-baca-e30abd850138-console-serving-cert\") pod \"bf978273-5024-4c57-baca-e30abd850138\" (UID: \"bf978273-5024-4c57-baca-e30abd850138\") " May 28 15:16:17.716152 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:17.715951 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bf978273-5024-4c57-baca-e30abd850138-oauth-serving-cert\") pod \"bf978273-5024-4c57-baca-e30abd850138\" (UID: \"bf978273-5024-4c57-baca-e30abd850138\") " May 28 15:16:17.716152 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:17.715977 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bf978273-5024-4c57-baca-e30abd850138-console-oauth-config\") pod \"bf978273-5024-4c57-baca-e30abd850138\" (UID: \"bf978273-5024-4c57-baca-e30abd850138\") " May 28 15:16:17.716152 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:17.715998 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bf978273-5024-4c57-baca-e30abd850138-service-ca\") pod \"bf978273-5024-4c57-baca-e30abd850138\" (UID: \"bf978273-5024-4c57-baca-e30abd850138\") " May 28 15:16:17.716346 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:17.716215 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf978273-5024-4c57-baca-e30abd850138-console-config" (OuterVolumeSpecName: "console-config") pod "bf978273-5024-4c57-baca-e30abd850138" (UID: "bf978273-5024-4c57-baca-e30abd850138"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 28 15:16:17.716346 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:17.716286 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf978273-5024-4c57-baca-e30abd850138-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "bf978273-5024-4c57-baca-e30abd850138" (UID: "bf978273-5024-4c57-baca-e30abd850138"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 28 15:16:17.716460 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:17.716379 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf978273-5024-4c57-baca-e30abd850138-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "bf978273-5024-4c57-baca-e30abd850138" (UID: "bf978273-5024-4c57-baca-e30abd850138"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 28 15:16:17.716531 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:17.716514 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf978273-5024-4c57-baca-e30abd850138-service-ca" (OuterVolumeSpecName: "service-ca") pod "bf978273-5024-4c57-baca-e30abd850138" (UID: "bf978273-5024-4c57-baca-e30abd850138"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 28 15:16:17.717976 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:17.717954 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf978273-5024-4c57-baca-e30abd850138-kube-api-access-2n282" (OuterVolumeSpecName: "kube-api-access-2n282") pod "bf978273-5024-4c57-baca-e30abd850138" (UID: "bf978273-5024-4c57-baca-e30abd850138"). InnerVolumeSpecName "kube-api-access-2n282". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 15:16:17.718445 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:17.718416 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf978273-5024-4c57-baca-e30abd850138-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "bf978273-5024-4c57-baca-e30abd850138" (UID: "bf978273-5024-4c57-baca-e30abd850138"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 28 15:16:17.718445 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:17.718432 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf978273-5024-4c57-baca-e30abd850138-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "bf978273-5024-4c57-baca-e30abd850138" (UID: "bf978273-5024-4c57-baca-e30abd850138"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 28 15:16:17.816642 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:17.816600 2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bf978273-5024-4c57-baca-e30abd850138-oauth-serving-cert\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:16:17.816642 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:17.816638 2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bf978273-5024-4c57-baca-e30abd850138-console-oauth-config\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:16:17.816642 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:17.816648 2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bf978273-5024-4c57-baca-e30abd850138-service-ca\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:16:17.816642 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:17.816659 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2n282\" (UniqueName: \"kubernetes.io/projected/bf978273-5024-4c57-baca-e30abd850138-kube-api-access-2n282\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:16:17.816888 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:17.816670 2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bf978273-5024-4c57-baca-e30abd850138-console-config\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:16:17.816888 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:17.816678 2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf978273-5024-4c57-baca-e30abd850138-trusted-ca-bundle\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:16:17.816888 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:17.816686 2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bf978273-5024-4c57-baca-e30abd850138-console-serving-cert\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:16:18.403167 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:18.403139 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6448c57698-9bw6c_bf978273-5024-4c57-baca-e30abd850138/console/0.log" May 28 15:16:18.403629 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:18.403186 2559 generic.go:358] "Generic (PLEG): container finished" podID="bf978273-5024-4c57-baca-e30abd850138" containerID="5cf3dfc00e85c24ed0ff6f406d8ed206391e2249a5d41ab0ceffa41d2663e1aa" exitCode=2 May 28 15:16:18.403629 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:18.403264 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6448c57698-9bw6c" May 28 15:16:18.403629 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:18.403277 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6448c57698-9bw6c" event={"ID":"bf978273-5024-4c57-baca-e30abd850138","Type":"ContainerDied","Data":"5cf3dfc00e85c24ed0ff6f406d8ed206391e2249a5d41ab0ceffa41d2663e1aa"} May 28 15:16:18.403629 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:18.403325 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6448c57698-9bw6c" event={"ID":"bf978273-5024-4c57-baca-e30abd850138","Type":"ContainerDied","Data":"580cf26ac3134c7006e5dfc2f67269b51d828e93fac372fafe99e778e54bb098"} May 28 15:16:18.403629 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:18.403346 2559 scope.go:117] "RemoveContainer" containerID="5cf3dfc00e85c24ed0ff6f406d8ed206391e2249a5d41ab0ceffa41d2663e1aa" May 28 15:16:18.412779 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:18.412762 2559 scope.go:117] "RemoveContainer" containerID="5cf3dfc00e85c24ed0ff6f406d8ed206391e2249a5d41ab0ceffa41d2663e1aa" May 28 15:16:18.413013 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:16:18.412994 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5cf3dfc00e85c24ed0ff6f406d8ed206391e2249a5d41ab0ceffa41d2663e1aa\": container with ID starting with 5cf3dfc00e85c24ed0ff6f406d8ed206391e2249a5d41ab0ceffa41d2663e1aa not found: ID does not exist" containerID="5cf3dfc00e85c24ed0ff6f406d8ed206391e2249a5d41ab0ceffa41d2663e1aa" May 28 15:16:18.413075 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:18.413024 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cf3dfc00e85c24ed0ff6f406d8ed206391e2249a5d41ab0ceffa41d2663e1aa"} err="failed to get container status \"5cf3dfc00e85c24ed0ff6f406d8ed206391e2249a5d41ab0ceffa41d2663e1aa\": rpc error: code = NotFound desc = could not find container \"5cf3dfc00e85c24ed0ff6f406d8ed206391e2249a5d41ab0ceffa41d2663e1aa\": container with ID starting with 5cf3dfc00e85c24ed0ff6f406d8ed206391e2249a5d41ab0ceffa41d2663e1aa not found: ID does not exist" May 28 15:16:18.436298 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:18.436275 2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6448c57698-9bw6c"] May 28 15:16:18.441151 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:18.441129 2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6448c57698-9bw6c"] May 28 15:16:18.647918 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:18.647889 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-l22fb"] May 28 15:16:18.648258 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:18.648246 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf978273-5024-4c57-baca-e30abd850138" containerName="console" May 28 15:16:18.648300 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:18.648260 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf978273-5024-4c57-baca-e30abd850138" containerName="console" May 28 15:16:18.648334 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:18.648323 2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="bf978273-5024-4c57-baca-e30abd850138" containerName="console" May 28 15:16:18.653318 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:18.653275 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-l22fb" May 28 15:16:18.656668 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:18.656647 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-lgmkq\"" May 28 15:16:18.664379 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:18.664359 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-l22fb"] May 28 15:16:18.822939 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:18.822911 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9441c18-6732-4471-a2b4-f71e471bbed8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-l22fb\" (UID: \"f9441c18-6732-4471-a2b4-f71e471bbed8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-l22fb" May 28 15:16:18.823050 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:18.822943 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42r8f\" (UniqueName: \"kubernetes.io/projected/f9441c18-6732-4471-a2b4-f71e471bbed8-kube-api-access-42r8f\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-l22fb\" (UID: \"f9441c18-6732-4471-a2b4-f71e471bbed8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-l22fb" May 28 15:16:18.923853 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:18.923797 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9441c18-6732-4471-a2b4-f71e471bbed8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-l22fb\" (UID: \"f9441c18-6732-4471-a2b4-f71e471bbed8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-l22fb" May 28 15:16:18.923853 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:18.923829 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-42r8f\" (UniqueName: \"kubernetes.io/projected/f9441c18-6732-4471-a2b4-f71e471bbed8-kube-api-access-42r8f\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-l22fb\" (UID: \"f9441c18-6732-4471-a2b4-f71e471bbed8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-l22fb" May 28 15:16:18.924123 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:18.924104 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9441c18-6732-4471-a2b4-f71e471bbed8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-l22fb\" (UID: \"f9441c18-6732-4471-a2b4-f71e471bbed8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-l22fb" May 28 15:16:18.931581 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:18.931554 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-42r8f\" (UniqueName: \"kubernetes.io/projected/f9441c18-6732-4471-a2b4-f71e471bbed8-kube-api-access-42r8f\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-l22fb\" (UID: \"f9441c18-6732-4471-a2b4-f71e471bbed8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-l22fb" May 28 15:16:18.964327 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:18.964298 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-l22fb" May 28 15:16:19.101512 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:19.101460 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-l22fb"] May 28 15:16:19.103788 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:16:19.103758 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9441c18_6732_4471_a2b4_f71e471bbed8.slice/crio-17fa693b441cf15b82eb2b63fc11d27caf4216dd007c434dc77d6f3f87e19aed WatchSource:0}: Error finding container 17fa693b441cf15b82eb2b63fc11d27caf4216dd007c434dc77d6f3f87e19aed: Status 404 returned error can't find the container with id 17fa693b441cf15b82eb2b63fc11d27caf4216dd007c434dc77d6f3f87e19aed May 28 15:16:19.408918 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:19.408888 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-l22fb" event={"ID":"f9441c18-6732-4471-a2b4-f71e471bbed8","Type":"ContainerStarted","Data":"17fa693b441cf15b82eb2b63fc11d27caf4216dd007c434dc77d6f3f87e19aed"} May 28 15:16:19.713673 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:19.713596 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf978273-5024-4c57-baca-e30abd850138" path="/var/lib/kubelet/pods/bf978273-5024-4c57-baca-e30abd850138/volumes" May 28 15:16:24.384742 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:24.384716 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-mrdzf" May 28 15:16:24.430639 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:24.430608 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-l22fb" event={"ID":"f9441c18-6732-4471-a2b4-f71e471bbed8","Type":"ContainerStarted","Data":"3c51ca767548d8ba7754b0041c0e1d64923ed979f8f5c29b9a04dde2fe9ccbed"} May 28 15:16:24.430770 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:24.430728 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-l22fb" May 28 15:16:24.450687 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:24.450633 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-l22fb" podStartSLOduration=1.217628108 podStartE2EDuration="6.450616753s" podCreationTimestamp="2026-05-28 15:16:18 +0000 UTC" firstStartedPulling="2026-05-28 15:16:19.106225358 +0000 UTC m=+529.995212424" lastFinishedPulling="2026-05-28 15:16:24.339214007 +0000 UTC m=+535.228201069" observedRunningTime="2026-05-28 15:16:24.449104629 +0000 UTC m=+535.338091714" watchObservedRunningTime="2026-05-28 15:16:24.450616753 +0000 UTC m=+535.339603839" May 28 15:16:27.135145 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:27.135114 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-946np"] May 28 15:16:27.138662 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:27.138641 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-946np" May 28 15:16:27.140980 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:27.140960 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" May 28 15:16:27.141198 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:27.141183 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" May 28 15:16:27.141340 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:27.141318 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-jxjrw\"" May 28 15:16:27.144970 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:27.144949 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-946np"] May 28 15:16:27.298463 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:27.298432 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7b734797-e0f8-4bef-8537-d564d53d545e-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-946np\" (UID: \"7b734797-e0f8-4bef-8537-d564d53d545e\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-946np" May 28 15:16:27.298626 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:27.298536 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7b734797-e0f8-4bef-8537-d564d53d545e-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-946np\" (UID: \"7b734797-e0f8-4bef-8537-d564d53d545e\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-946np" May 28 15:16:27.298626 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:27.298570 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wlb8\" (UniqueName: \"kubernetes.io/projected/7b734797-e0f8-4bef-8537-d564d53d545e-kube-api-access-4wlb8\") pod \"kuadrant-console-plugin-6cb54b5c86-946np\" (UID: \"7b734797-e0f8-4bef-8537-d564d53d545e\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-946np" May 28 15:16:27.399998 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:27.399915 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7b734797-e0f8-4bef-8537-d564d53d545e-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-946np\" (UID: \"7b734797-e0f8-4bef-8537-d564d53d545e\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-946np" May 28 15:16:27.400148 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:27.399996 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7b734797-e0f8-4bef-8537-d564d53d545e-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-946np\" (UID: \"7b734797-e0f8-4bef-8537-d564d53d545e\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-946np" May 28 15:16:27.400148 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:27.400030 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4wlb8\" (UniqueName: \"kubernetes.io/projected/7b734797-e0f8-4bef-8537-d564d53d545e-kube-api-access-4wlb8\") pod \"kuadrant-console-plugin-6cb54b5c86-946np\" (UID: \"7b734797-e0f8-4bef-8537-d564d53d545e\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-946np" May 28 15:16:27.400665 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:27.400642 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7b734797-e0f8-4bef-8537-d564d53d545e-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-946np\" (UID: \"7b734797-e0f8-4bef-8537-d564d53d545e\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-946np" May 28 15:16:27.402354 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:27.402337 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7b734797-e0f8-4bef-8537-d564d53d545e-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-946np\" (UID: \"7b734797-e0f8-4bef-8537-d564d53d545e\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-946np" May 28 15:16:27.407799 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:27.407780 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wlb8\" (UniqueName: \"kubernetes.io/projected/7b734797-e0f8-4bef-8537-d564d53d545e-kube-api-access-4wlb8\") pod \"kuadrant-console-plugin-6cb54b5c86-946np\" (UID: \"7b734797-e0f8-4bef-8537-d564d53d545e\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-946np" May 28 15:16:27.448568 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:27.448544 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-946np" May 28 15:16:27.575839 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:27.575813 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-946np"] May 28 15:16:27.577356 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:16:27.577329 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7b734797_e0f8_4bef_8537_d564d53d545e.slice/crio-fe89e84d57926fb59e6240a36c42bcccc01815b0582119380007662bf6e502a5 WatchSource:0}: Error finding container fe89e84d57926fb59e6240a36c42bcccc01815b0582119380007662bf6e502a5: Status 404 returned error can't find the container with id fe89e84d57926fb59e6240a36c42bcccc01815b0582119380007662bf6e502a5 May 28 15:16:28.448983 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:28.448909 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-946np" event={"ID":"7b734797-e0f8-4bef-8537-d564d53d545e","Type":"ContainerStarted","Data":"fe89e84d57926fb59e6240a36c42bcccc01815b0582119380007662bf6e502a5"} May 28 15:16:35.437859 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:35.437829 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-l22fb" May 28 15:16:36.459593 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:36.459554 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-47mbq"] May 28 15:16:36.466248 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:36.466218 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-47mbq" May 28 15:16:36.477659 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:36.477634 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-47mbq"] May 28 15:16:36.483319 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:36.483292 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/83d18a45-5298-49fa-a8e9-64565cb293e1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-47mbq\" (UID: \"83d18a45-5298-49fa-a8e9-64565cb293e1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-47mbq" May 28 15:16:36.483429 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:36.483354 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlcxx\" (UniqueName: \"kubernetes.io/projected/83d18a45-5298-49fa-a8e9-64565cb293e1-kube-api-access-tlcxx\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-47mbq\" (UID: \"83d18a45-5298-49fa-a8e9-64565cb293e1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-47mbq" May 28 15:16:36.584451 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:36.584071 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/83d18a45-5298-49fa-a8e9-64565cb293e1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-47mbq\" (UID: \"83d18a45-5298-49fa-a8e9-64565cb293e1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-47mbq" May 28 15:16:36.584451 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:36.584119 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tlcxx\" (UniqueName: \"kubernetes.io/projected/83d18a45-5298-49fa-a8e9-64565cb293e1-kube-api-access-tlcxx\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-47mbq\" (UID: \"83d18a45-5298-49fa-a8e9-64565cb293e1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-47mbq" May 28 15:16:36.584695 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:36.584507 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/83d18a45-5298-49fa-a8e9-64565cb293e1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-47mbq\" (UID: \"83d18a45-5298-49fa-a8e9-64565cb293e1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-47mbq" May 28 15:16:36.599960 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:36.599920 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlcxx\" (UniqueName: \"kubernetes.io/projected/83d18a45-5298-49fa-a8e9-64565cb293e1-kube-api-access-tlcxx\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-47mbq\" (UID: \"83d18a45-5298-49fa-a8e9-64565cb293e1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-47mbq" May 28 15:16:36.780373 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:36.780336 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-47mbq" May 28 15:16:37.139244 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:37.139164 2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-l22fb"] May 28 15:16:37.139780 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:37.139737 2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-l22fb" podUID="f9441c18-6732-4471-a2b4-f71e471bbed8" containerName="manager" containerID="cri-o://3c51ca767548d8ba7754b0041c0e1d64923ed979f8f5c29b9a04dde2fe9ccbed" gracePeriod=2 May 28 15:16:37.142064 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:37.142036 2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-l22fb"] May 28 15:16:37.142276 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:37.142248 2559 status_manager.go:895] "Failed to get status for pod" podUID="f9441c18-6732-4471-a2b4-f71e471bbed8" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-l22fb" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-l22fb\" is forbidden: User \"system:node:ip-10-0-140-141.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-141.ec2.internal' and this object" May 28 15:16:37.148928 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:37.148901 2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-47mbq"] May 28 15:16:37.155915 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:37.155893 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ld65f"] May 28 15:16:37.156591 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:37.156571 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9441c18-6732-4471-a2b4-f71e471bbed8" containerName="manager" May 28 15:16:37.156713 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:37.156703 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9441c18-6732-4471-a2b4-f71e471bbed8" containerName="manager" May 28 15:16:37.156928 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:37.156914 2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="f9441c18-6732-4471-a2b4-f71e471bbed8" containerName="manager" May 28 15:16:37.161008 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:37.160986 2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-47mbq"] May 28 15:16:37.161148 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:37.161133 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ld65f" May 28 15:16:37.163266 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:37.163234 2559 status_manager.go:895] "Failed to get status for pod" podUID="f9441c18-6732-4471-a2b4-f71e471bbed8" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-l22fb" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-l22fb\" is forbidden: User \"system:node:ip-10-0-140-141.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-141.ec2.internal' and this object" May 28 15:16:37.173163 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:37.173117 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-6sw42"] May 28 15:16:37.176729 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:37.176621 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ld65f"] May 28 15:16:37.176828 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:37.176732 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-6sw42" May 28 15:16:37.190064 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:37.190017 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zczw\" (UniqueName: \"kubernetes.io/projected/fe94dbc4-ee3d-4f71-b68d-99823bd4f594-kube-api-access-7zczw\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-ld65f\" (UID: \"fe94dbc4-ee3d-4f71-b68d-99823bd4f594\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ld65f" May 28 15:16:37.190178 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:37.190117 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/264d077b-757b-470a-85e5-fe4257145180-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-6sw42\" (UID: \"264d077b-757b-470a-85e5-fe4257145180\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-6sw42" May 28 15:16:37.190244 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:37.190223 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47q2h\" (UniqueName: \"kubernetes.io/projected/264d077b-757b-470a-85e5-fe4257145180-kube-api-access-47q2h\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-6sw42\" (UID: \"264d077b-757b-470a-85e5-fe4257145180\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-6sw42" May 28 15:16:37.190302 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:37.190255 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fe94dbc4-ee3d-4f71-b68d-99823bd4f594-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-ld65f\" (UID: \"fe94dbc4-ee3d-4f71-b68d-99823bd4f594\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ld65f" May 28 15:16:37.194070 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:37.194045 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-6sw42"] May 28 15:16:37.198431 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:37.198394 2559 status_manager.go:895] "Failed to get status for pod" podUID="f9441c18-6732-4471-a2b4-f71e471bbed8" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-l22fb" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-l22fb\" is forbidden: User \"system:node:ip-10-0-140-141.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-141.ec2.internal' and this object" May 28 15:16:37.291679 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:37.291637 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/264d077b-757b-470a-85e5-fe4257145180-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-6sw42\" (UID: \"264d077b-757b-470a-85e5-fe4257145180\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-6sw42" May 28 15:16:37.291860 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:37.291740 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-47q2h\" (UniqueName: \"kubernetes.io/projected/264d077b-757b-470a-85e5-fe4257145180-kube-api-access-47q2h\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-6sw42\" (UID: \"264d077b-757b-470a-85e5-fe4257145180\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-6sw42" May 28 15:16:37.291860 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:37.291766 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fe94dbc4-ee3d-4f71-b68d-99823bd4f594-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-ld65f\" (UID: \"fe94dbc4-ee3d-4f71-b68d-99823bd4f594\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ld65f" May 28 15:16:37.291860 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:37.291806 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7zczw\" (UniqueName: \"kubernetes.io/projected/fe94dbc4-ee3d-4f71-b68d-99823bd4f594-kube-api-access-7zczw\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-ld65f\" (UID: \"fe94dbc4-ee3d-4f71-b68d-99823bd4f594\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ld65f" May 28 15:16:37.292098 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:37.292074 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/264d077b-757b-470a-85e5-fe4257145180-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-6sw42\" (UID: \"264d077b-757b-470a-85e5-fe4257145180\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-6sw42" May 28 15:16:37.292150 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:37.292131 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fe94dbc4-ee3d-4f71-b68d-99823bd4f594-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-ld65f\" (UID: \"fe94dbc4-ee3d-4f71-b68d-99823bd4f594\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ld65f" May 28 15:16:37.303247 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:37.303221 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47q2h\" (UniqueName: \"kubernetes.io/projected/264d077b-757b-470a-85e5-fe4257145180-kube-api-access-47q2h\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-6sw42\" (UID: \"264d077b-757b-470a-85e5-fe4257145180\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-6sw42" May 28 15:16:37.303376 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:37.303317 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zczw\" (UniqueName: \"kubernetes.io/projected/fe94dbc4-ee3d-4f71-b68d-99823bd4f594-kube-api-access-7zczw\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-ld65f\" (UID: \"fe94dbc4-ee3d-4f71-b68d-99823bd4f594\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ld65f" May 28 15:16:37.510809 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:37.510781 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ld65f" May 28 15:16:37.517546 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:37.517521 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-6sw42" May 28 15:16:49.867393 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:49.867370 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-l22fb" May 28 15:16:49.869834 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:49.869803 2559 status_manager.go:895] "Failed to get status for pod" podUID="f9441c18-6732-4471-a2b4-f71e471bbed8" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-l22fb" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-l22fb\" is forbidden: User \"system:node:ip-10-0-140-141.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-141.ec2.internal' and this object" May 28 15:16:49.903897 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:49.903788 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ld65f"] May 28 15:16:49.905917 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:16:49.905879 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe94dbc4_ee3d_4f71_b68d_99823bd4f594.slice/crio-605ccf6ef1f38651466a2b68ded759d73ecdc73beaf5dffb44e59e08ac77f3c4 WatchSource:0}: Error finding container 605ccf6ef1f38651466a2b68ded759d73ecdc73beaf5dffb44e59e08ac77f3c4: Status 404 returned error can't find the container with id 605ccf6ef1f38651466a2b68ded759d73ecdc73beaf5dffb44e59e08ac77f3c4 May 28 15:16:49.918136 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:49.918115 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9441c18-6732-4471-a2b4-f71e471bbed8-extensions-socket-volume\") pod \"f9441c18-6732-4471-a2b4-f71e471bbed8\" (UID: \"f9441c18-6732-4471-a2b4-f71e471bbed8\") " May 28 15:16:49.918234 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:49.918187 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-42r8f\" (UniqueName: \"kubernetes.io/projected/f9441c18-6732-4471-a2b4-f71e471bbed8-kube-api-access-42r8f\") pod \"f9441c18-6732-4471-a2b4-f71e471bbed8\" (UID: \"f9441c18-6732-4471-a2b4-f71e471bbed8\") " May 28 15:16:49.918547 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:49.918514 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9441c18-6732-4471-a2b4-f71e471bbed8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f9441c18-6732-4471-a2b4-f71e471bbed8" (UID: "f9441c18-6732-4471-a2b4-f71e471bbed8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 28 15:16:49.920603 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:49.920569 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9441c18-6732-4471-a2b4-f71e471bbed8-kube-api-access-42r8f" (OuterVolumeSpecName: "kube-api-access-42r8f") pod "f9441c18-6732-4471-a2b4-f71e471bbed8" (UID: "f9441c18-6732-4471-a2b4-f71e471bbed8"). InnerVolumeSpecName "kube-api-access-42r8f". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 15:16:49.923033 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:49.922941 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-6sw42"] May 28 15:16:49.924526 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:16:49.924470 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod264d077b_757b_470a_85e5_fe4257145180.slice/crio-e56c53edcd1762f182c570b776796ac1db4f25d2d65cea165ee4c32c60de4cdd WatchSource:0}: Error finding container e56c53edcd1762f182c570b776796ac1db4f25d2d65cea165ee4c32c60de4cdd: Status 404 returned error can't find the container with id e56c53edcd1762f182c570b776796ac1db4f25d2d65cea165ee4c32c60de4cdd May 28 15:16:50.018944 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:50.018913 2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9441c18-6732-4471-a2b4-f71e471bbed8-extensions-socket-volume\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:16:50.018944 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:50.018939 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-42r8f\" (UniqueName: \"kubernetes.io/projected/f9441c18-6732-4471-a2b4-f71e471bbed8-kube-api-access-42r8f\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:16:50.538635 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:50.538598 2559 generic.go:358] "Generic (PLEG): container finished" podID="f9441c18-6732-4471-a2b4-f71e471bbed8" containerID="3c51ca767548d8ba7754b0041c0e1d64923ed979f8f5c29b9a04dde2fe9ccbed" exitCode=0 May 28 15:16:50.538815 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:50.538652 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-l22fb" May 28 15:16:50.538815 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:50.538709 2559 scope.go:117] "RemoveContainer" containerID="3c51ca767548d8ba7754b0041c0e1d64923ed979f8f5c29b9a04dde2fe9ccbed" May 28 15:16:50.540227 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:50.540197 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-946np" event={"ID":"7b734797-e0f8-4bef-8537-d564d53d545e","Type":"ContainerStarted","Data":"2eb0c5ac1b6c38a937c9db5abbcf04b134be1859fe7a9cb2a5874fbb27d324da"} May 28 15:16:50.541152 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:50.541121 2559 status_manager.go:895] "Failed to get status for pod" podUID="f9441c18-6732-4471-a2b4-f71e471bbed8" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-l22fb" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-l22fb\" is forbidden: User \"system:node:ip-10-0-140-141.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-141.ec2.internal' and this object" May 28 15:16:50.541882 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:50.541828 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-6sw42" event={"ID":"264d077b-757b-470a-85e5-fe4257145180","Type":"ContainerStarted","Data":"cd6dbcf24954c6cdbca3ad8c394096f50568ceedbb37ebe8f38bce06ec29adac"} May 28 15:16:50.541882 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:50.541862 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-6sw42" event={"ID":"264d077b-757b-470a-85e5-fe4257145180","Type":"ContainerStarted","Data":"e56c53edcd1762f182c570b776796ac1db4f25d2d65cea165ee4c32c60de4cdd"} May 28 15:16:50.542052 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:50.541971 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-6sw42" May 28 15:16:50.543471 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:50.543443 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ld65f" event={"ID":"fe94dbc4-ee3d-4f71-b68d-99823bd4f594","Type":"ContainerStarted","Data":"50aeeb345baf0f5bae553718c020342a4cb1a9d3697437dc88e65c740b55813e"} May 28 15:16:50.543567 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:50.543500 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ld65f" event={"ID":"fe94dbc4-ee3d-4f71-b68d-99823bd4f594","Type":"ContainerStarted","Data":"605ccf6ef1f38651466a2b68ded759d73ecdc73beaf5dffb44e59e08ac77f3c4"} May 28 15:16:50.543610 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:50.543589 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ld65f" May 28 15:16:50.548811 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:50.548797 2559 scope.go:117] "RemoveContainer" containerID="3c51ca767548d8ba7754b0041c0e1d64923ed979f8f5c29b9a04dde2fe9ccbed" May 28 15:16:50.549040 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:16:50.549024 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c51ca767548d8ba7754b0041c0e1d64923ed979f8f5c29b9a04dde2fe9ccbed\": container with ID starting with 3c51ca767548d8ba7754b0041c0e1d64923ed979f8f5c29b9a04dde2fe9ccbed not found: ID does not exist" containerID="3c51ca767548d8ba7754b0041c0e1d64923ed979f8f5c29b9a04dde2fe9ccbed" May 28 15:16:50.549096 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:50.549047 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c51ca767548d8ba7754b0041c0e1d64923ed979f8f5c29b9a04dde2fe9ccbed"} err="failed to get container status \"3c51ca767548d8ba7754b0041c0e1d64923ed979f8f5c29b9a04dde2fe9ccbed\": rpc error: code = NotFound desc = could not find container \"3c51ca767548d8ba7754b0041c0e1d64923ed979f8f5c29b9a04dde2fe9ccbed\": container with ID starting with 3c51ca767548d8ba7754b0041c0e1d64923ed979f8f5c29b9a04dde2fe9ccbed not found: ID does not exist" May 28 15:16:50.558164 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:50.558139 2559 status_manager.go:895] "Failed to get status for pod" podUID="f9441c18-6732-4471-a2b4-f71e471bbed8" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-l22fb" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-l22fb\" is forbidden: User \"system:node:ip-10-0-140-141.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-141.ec2.internal' and this object" May 28 15:16:50.558878 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:50.558842 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-946np" podStartSLOduration=1.345352903 podStartE2EDuration="23.55883256s" podCreationTimestamp="2026-05-28 15:16:27 +0000 UTC" firstStartedPulling="2026-05-28 15:16:27.578699241 +0000 UTC m=+538.467686303" lastFinishedPulling="2026-05-28 15:16:49.792178884 +0000 UTC m=+560.681165960" observedRunningTime="2026-05-28 15:16:50.55589428 +0000 UTC m=+561.444881364" watchObservedRunningTime="2026-05-28 15:16:50.55883256 +0000 UTC m=+561.447819645" May 28 15:16:50.560076 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:50.560050 2559 status_manager.go:895] "Failed to get status for pod" podUID="f9441c18-6732-4471-a2b4-f71e471bbed8" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-l22fb" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-l22fb\" is forbidden: User \"system:node:ip-10-0-140-141.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-141.ec2.internal' and this object" May 28 15:16:50.577972 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:50.577933 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ld65f" podStartSLOduration=13.577922512 podStartE2EDuration="13.577922512s" podCreationTimestamp="2026-05-28 15:16:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 15:16:50.576805955 +0000 UTC m=+561.465793040" watchObservedRunningTime="2026-05-28 15:16:50.577922512 +0000 UTC m=+561.466909596" May 28 15:16:50.597449 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:50.597399 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-6sw42" podStartSLOduration=13.597381159 podStartE2EDuration="13.597381159s" podCreationTimestamp="2026-05-28 15:16:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 15:16:50.594799985 +0000 UTC m=+561.483787070" watchObservedRunningTime="2026-05-28 15:16:50.597381159 +0000 UTC m=+561.486368244" May 28 15:16:51.713083 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:51.713053 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9441c18-6732-4471-a2b4-f71e471bbed8" path="/var/lib/kubelet/pods/f9441c18-6732-4471-a2b4-f71e471bbed8/volumes" May 28 15:16:52.285127 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:16:52.285096 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83d18a45_5298_49fa_a8e9_64565cb293e1.slice/crio-3ef8fe21f4bd99f1a069adfbcef3a4ac74d7383ed1ad6860f76746886524410d WatchSource:0}: Error finding container 3ef8fe21f4bd99f1a069adfbcef3a4ac74d7383ed1ad6860f76746886524410d: Status 404 returned error can't find the container with id 3ef8fe21f4bd99f1a069adfbcef3a4ac74d7383ed1ad6860f76746886524410d May 28 15:16:52.553391 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:52.553328 2559 generic.go:358] "Generic (PLEG): container finished" podID="83d18a45-5298-49fa-a8e9-64565cb293e1" containerID="78c79249bffeaace41fa5dda8b230202e8a49d51e80814560c327619f430141a" exitCode=1 May 28 15:16:52.556008 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:52.555979 2559 status_manager.go:895] "Failed to get status for pod" podUID="83d18a45-5298-49fa-a8e9-64565cb293e1" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-47mbq" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-47mbq\" is forbidden: User \"system:node:ip-10-0-140-141.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-141.ec2.internal' and this object" May 28 15:16:52.584061 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:52.584042 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-47mbq" May 28 15:16:52.586248 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:52.586225 2559 status_manager.go:895] "Failed to get status for pod" podUID="83d18a45-5298-49fa-a8e9-64565cb293e1" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-47mbq" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-47mbq\" is forbidden: User \"system:node:ip-10-0-140-141.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-141.ec2.internal' and this object" May 28 15:16:52.642136 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:52.642105 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/83d18a45-5298-49fa-a8e9-64565cb293e1-extensions-socket-volume\") pod \"83d18a45-5298-49fa-a8e9-64565cb293e1\" (UID: \"83d18a45-5298-49fa-a8e9-64565cb293e1\") " May 28 15:16:52.642231 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:52.642150 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tlcxx\" (UniqueName: \"kubernetes.io/projected/83d18a45-5298-49fa-a8e9-64565cb293e1-kube-api-access-tlcxx\") pod \"83d18a45-5298-49fa-a8e9-64565cb293e1\" (UID: \"83d18a45-5298-49fa-a8e9-64565cb293e1\") " May 28 15:16:52.642393 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:52.642368 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83d18a45-5298-49fa-a8e9-64565cb293e1-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "83d18a45-5298-49fa-a8e9-64565cb293e1" (UID: "83d18a45-5298-49fa-a8e9-64565cb293e1"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 28 15:16:52.644040 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:52.644020 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83d18a45-5298-49fa-a8e9-64565cb293e1-kube-api-access-tlcxx" (OuterVolumeSpecName: "kube-api-access-tlcxx") pod "83d18a45-5298-49fa-a8e9-64565cb293e1" (UID: "83d18a45-5298-49fa-a8e9-64565cb293e1"). InnerVolumeSpecName "kube-api-access-tlcxx". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 15:16:52.743506 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:52.743461 2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/83d18a45-5298-49fa-a8e9-64565cb293e1-extensions-socket-volume\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:16:52.743506 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:52.743503 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlcxx\" (UniqueName: \"kubernetes.io/projected/83d18a45-5298-49fa-a8e9-64565cb293e1-kube-api-access-tlcxx\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:16:53.558329 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:53.558267 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-47mbq" May 28 15:16:53.558329 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:53.558287 2559 scope.go:117] "RemoveContainer" containerID="78c79249bffeaace41fa5dda8b230202e8a49d51e80814560c327619f430141a" May 28 15:16:53.560503 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:53.560463 2559 status_manager.go:895] "Failed to get status for pod" podUID="83d18a45-5298-49fa-a8e9-64565cb293e1" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-47mbq" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-47mbq\" is forbidden: User \"system:node:ip-10-0-140-141.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-141.ec2.internal' and this object" May 28 15:16:53.568454 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:53.568430 2559 status_manager.go:895] "Failed to get status for pod" podUID="83d18a45-5298-49fa-a8e9-64565cb293e1" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-47mbq" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-47mbq\" is forbidden: User \"system:node:ip-10-0-140-141.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-141.ec2.internal' and this object" May 28 15:16:53.712595 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:16:53.712572 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83d18a45-5298-49fa-a8e9-64565cb293e1" path="/var/lib/kubelet/pods/83d18a45-5298-49fa-a8e9-64565cb293e1/volumes" May 28 15:17:01.550967 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:01.550934 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ld65f" May 28 15:17:01.551555 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:01.550995 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-6sw42" May 28 15:17:01.613415 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:01.613387 2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ld65f"] May 28 15:17:01.613629 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:01.613601 2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ld65f" podUID="fe94dbc4-ee3d-4f71-b68d-99823bd4f594" containerName="manager" containerID="cri-o://50aeeb345baf0f5bae553718c020342a4cb1a9d3697437dc88e65c740b55813e" gracePeriod=10 May 28 15:17:01.803545 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:01.800109 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-z647b"] May 28 15:17:01.803545 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:01.801063 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83d18a45-5298-49fa-a8e9-64565cb293e1" containerName="manager" May 28 15:17:01.803545 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:01.801084 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="83d18a45-5298-49fa-a8e9-64565cb293e1" containerName="manager" May 28 15:17:01.803545 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:01.801302 2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="83d18a45-5298-49fa-a8e9-64565cb293e1" containerName="manager" May 28 15:17:01.806453 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:01.806178 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-z647b" May 28 15:17:01.812090 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:01.812060 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-z647b"] May 28 15:17:01.867134 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:01.867113 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ld65f" May 28 15:17:01.918648 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:01.918624 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f21feb1-f535-41f2-846a-bf6b6e8c1cb8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-z647b\" (UID: \"0f21feb1-f535-41f2-846a-bf6b6e8c1cb8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-z647b" May 28 15:17:01.918759 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:01.918680 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4k49\" (UniqueName: \"kubernetes.io/projected/0f21feb1-f535-41f2-846a-bf6b6e8c1cb8-kube-api-access-x4k49\") pod \"kuadrant-operator-controller-manager-55c7f4c975-z647b\" (UID: \"0f21feb1-f535-41f2-846a-bf6b6e8c1cb8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-z647b" May 28 15:17:02.019341 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:02.019315 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fe94dbc4-ee3d-4f71-b68d-99823bd4f594-extensions-socket-volume\") pod \"fe94dbc4-ee3d-4f71-b68d-99823bd4f594\" (UID: \"fe94dbc4-ee3d-4f71-b68d-99823bd4f594\") " May 28 15:17:02.019451 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:02.019374 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7zczw\" (UniqueName: \"kubernetes.io/projected/fe94dbc4-ee3d-4f71-b68d-99823bd4f594-kube-api-access-7zczw\") pod \"fe94dbc4-ee3d-4f71-b68d-99823bd4f594\" (UID: \"fe94dbc4-ee3d-4f71-b68d-99823bd4f594\") " May 28 15:17:02.019547 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:02.019528 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f21feb1-f535-41f2-846a-bf6b6e8c1cb8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-z647b\" (UID: \"0f21feb1-f535-41f2-846a-bf6b6e8c1cb8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-z647b" May 28 15:17:02.019602 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:02.019591 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-x4k49\" (UniqueName: \"kubernetes.io/projected/0f21feb1-f535-41f2-846a-bf6b6e8c1cb8-kube-api-access-x4k49\") pod \"kuadrant-operator-controller-manager-55c7f4c975-z647b\" (UID: \"0f21feb1-f535-41f2-846a-bf6b6e8c1cb8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-z647b" May 28 15:17:02.019705 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:02.019679 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe94dbc4-ee3d-4f71-b68d-99823bd4f594-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "fe94dbc4-ee3d-4f71-b68d-99823bd4f594" (UID: "fe94dbc4-ee3d-4f71-b68d-99823bd4f594"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 28 15:17:02.019893 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:02.019873 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f21feb1-f535-41f2-846a-bf6b6e8c1cb8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-z647b\" (UID: \"0f21feb1-f535-41f2-846a-bf6b6e8c1cb8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-z647b" May 28 15:17:02.021426 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:02.021403 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe94dbc4-ee3d-4f71-b68d-99823bd4f594-kube-api-access-7zczw" (OuterVolumeSpecName: "kube-api-access-7zczw") pod "fe94dbc4-ee3d-4f71-b68d-99823bd4f594" (UID: "fe94dbc4-ee3d-4f71-b68d-99823bd4f594"). InnerVolumeSpecName "kube-api-access-7zczw". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 15:17:02.027355 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:02.027334 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4k49\" (UniqueName: \"kubernetes.io/projected/0f21feb1-f535-41f2-846a-bf6b6e8c1cb8-kube-api-access-x4k49\") pod \"kuadrant-operator-controller-manager-55c7f4c975-z647b\" (UID: \"0f21feb1-f535-41f2-846a-bf6b6e8c1cb8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-z647b" May 28 15:17:02.119640 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:02.119582 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-z647b" May 28 15:17:02.120269 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:02.120173 2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fe94dbc4-ee3d-4f71-b68d-99823bd4f594-extensions-socket-volume\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:17:02.120269 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:02.120191 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7zczw\" (UniqueName: \"kubernetes.io/projected/fe94dbc4-ee3d-4f71-b68d-99823bd4f594-kube-api-access-7zczw\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:17:02.249288 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:02.249266 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-z647b"] May 28 15:17:02.251306 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:17:02.251284 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f21feb1_f535_41f2_846a_bf6b6e8c1cb8.slice/crio-030bbb8f6af3c73a2c850e48ed009b1fd32bdb9b05c292b3f40d5673689ce63d WatchSource:0}: Error finding container 030bbb8f6af3c73a2c850e48ed009b1fd32bdb9b05c292b3f40d5673689ce63d: Status 404 returned error can't find the container with id 030bbb8f6af3c73a2c850e48ed009b1fd32bdb9b05c292b3f40d5673689ce63d May 28 15:17:02.594878 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:02.594845 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-z647b" event={"ID":"0f21feb1-f535-41f2-846a-bf6b6e8c1cb8","Type":"ContainerStarted","Data":"fb76fb33a4461b2144a0a9eb2dac96ec14b78675919a7ae5744a815de609a11e"} May 28 15:17:02.594878 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:02.594882 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-z647b" event={"ID":"0f21feb1-f535-41f2-846a-bf6b6e8c1cb8","Type":"ContainerStarted","Data":"030bbb8f6af3c73a2c850e48ed009b1fd32bdb9b05c292b3f40d5673689ce63d"} May 28 15:17:02.595343 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:02.594911 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-z647b" May 28 15:17:02.596114 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:02.596094 2559 generic.go:358] "Generic (PLEG): container finished" podID="fe94dbc4-ee3d-4f71-b68d-99823bd4f594" containerID="50aeeb345baf0f5bae553718c020342a4cb1a9d3697437dc88e65c740b55813e" exitCode=0 May 28 15:17:02.596210 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:02.596121 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ld65f" event={"ID":"fe94dbc4-ee3d-4f71-b68d-99823bd4f594","Type":"ContainerDied","Data":"50aeeb345baf0f5bae553718c020342a4cb1a9d3697437dc88e65c740b55813e"} May 28 15:17:02.596210 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:02.596137 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ld65f" event={"ID":"fe94dbc4-ee3d-4f71-b68d-99823bd4f594","Type":"ContainerDied","Data":"605ccf6ef1f38651466a2b68ded759d73ecdc73beaf5dffb44e59e08ac77f3c4"} May 28 15:17:02.596210 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:02.596147 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ld65f" May 28 15:17:02.596356 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:02.596152 2559 scope.go:117] "RemoveContainer" containerID="50aeeb345baf0f5bae553718c020342a4cb1a9d3697437dc88e65c740b55813e" May 28 15:17:02.604849 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:02.604827 2559 scope.go:117] "RemoveContainer" containerID="50aeeb345baf0f5bae553718c020342a4cb1a9d3697437dc88e65c740b55813e" May 28 15:17:02.605087 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:17:02.605068 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50aeeb345baf0f5bae553718c020342a4cb1a9d3697437dc88e65c740b55813e\": container with ID starting with 50aeeb345baf0f5bae553718c020342a4cb1a9d3697437dc88e65c740b55813e not found: ID does not exist" containerID="50aeeb345baf0f5bae553718c020342a4cb1a9d3697437dc88e65c740b55813e" May 28 15:17:02.605153 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:02.605093 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50aeeb345baf0f5bae553718c020342a4cb1a9d3697437dc88e65c740b55813e"} err="failed to get container status \"50aeeb345baf0f5bae553718c020342a4cb1a9d3697437dc88e65c740b55813e\": rpc error: code = NotFound desc = could not find container \"50aeeb345baf0f5bae553718c020342a4cb1a9d3697437dc88e65c740b55813e\": container with ID starting with 50aeeb345baf0f5bae553718c020342a4cb1a9d3697437dc88e65c740b55813e not found: ID does not exist" May 28 15:17:02.619876 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:02.619836 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-z647b" podStartSLOduration=1.619824575 podStartE2EDuration="1.619824575s" podCreationTimestamp="2026-05-28 15:17:01 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 15:17:02.614921158 +0000 UTC m=+573.503908242" watchObservedRunningTime="2026-05-28 15:17:02.619824575 +0000 UTC m=+573.508811660" May 28 15:17:02.634068 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:02.634048 2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ld65f"] May 28 15:17:02.645512 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:02.645459 2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-ld65f"] May 28 15:17:03.712784 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:03.712750 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe94dbc4-ee3d-4f71-b68d-99823bd4f594" path="/var/lib/kubelet/pods/fe94dbc4-ee3d-4f71-b68d-99823bd4f594/volumes" May 28 15:17:13.603842 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:13.603810 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-z647b" May 28 15:17:13.670986 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:13.670954 2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-6sw42"] May 28 15:17:13.671300 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:13.671260 2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-6sw42" podUID="264d077b-757b-470a-85e5-fe4257145180" containerName="manager" containerID="cri-o://cd6dbcf24954c6cdbca3ad8c394096f50568ceedbb37ebe8f38bce06ec29adac" gracePeriod=10 May 28 15:17:14.490860 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:14.490839 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-6sw42" May 28 15:17:14.627945 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:14.627879 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/264d077b-757b-470a-85e5-fe4257145180-extensions-socket-volume\") pod \"264d077b-757b-470a-85e5-fe4257145180\" (UID: \"264d077b-757b-470a-85e5-fe4257145180\") " May 28 15:17:14.628265 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:14.627965 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-47q2h\" (UniqueName: \"kubernetes.io/projected/264d077b-757b-470a-85e5-fe4257145180-kube-api-access-47q2h\") pod \"264d077b-757b-470a-85e5-fe4257145180\" (UID: \"264d077b-757b-470a-85e5-fe4257145180\") " May 28 15:17:14.628304 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:14.628271 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/264d077b-757b-470a-85e5-fe4257145180-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "264d077b-757b-470a-85e5-fe4257145180" (UID: "264d077b-757b-470a-85e5-fe4257145180"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 28 15:17:14.629942 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:14.629916 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/264d077b-757b-470a-85e5-fe4257145180-kube-api-access-47q2h" (OuterVolumeSpecName: "kube-api-access-47q2h") pod "264d077b-757b-470a-85e5-fe4257145180" (UID: "264d077b-757b-470a-85e5-fe4257145180"). InnerVolumeSpecName "kube-api-access-47q2h". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 15:17:14.649367 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:14.649339 2559 generic.go:358] "Generic (PLEG): container finished" podID="264d077b-757b-470a-85e5-fe4257145180" containerID="cd6dbcf24954c6cdbca3ad8c394096f50568ceedbb37ebe8f38bce06ec29adac" exitCode=0 May 28 15:17:14.649502 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:14.649410 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-6sw42" May 28 15:17:14.649502 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:14.649423 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-6sw42" event={"ID":"264d077b-757b-470a-85e5-fe4257145180","Type":"ContainerDied","Data":"cd6dbcf24954c6cdbca3ad8c394096f50568ceedbb37ebe8f38bce06ec29adac"} May 28 15:17:14.649502 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:14.649468 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-6sw42" event={"ID":"264d077b-757b-470a-85e5-fe4257145180","Type":"ContainerDied","Data":"e56c53edcd1762f182c570b776796ac1db4f25d2d65cea165ee4c32c60de4cdd"} May 28 15:17:14.649673 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:14.649502 2559 scope.go:117] "RemoveContainer" containerID="cd6dbcf24954c6cdbca3ad8c394096f50568ceedbb37ebe8f38bce06ec29adac" May 28 15:17:14.658649 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:14.658634 2559 scope.go:117] "RemoveContainer" containerID="cd6dbcf24954c6cdbca3ad8c394096f50568ceedbb37ebe8f38bce06ec29adac" May 28 15:17:14.658887 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:17:14.658869 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd6dbcf24954c6cdbca3ad8c394096f50568ceedbb37ebe8f38bce06ec29adac\": container with ID starting with cd6dbcf24954c6cdbca3ad8c394096f50568ceedbb37ebe8f38bce06ec29adac not found: ID does not exist" containerID="cd6dbcf24954c6cdbca3ad8c394096f50568ceedbb37ebe8f38bce06ec29adac" May 28 15:17:14.658953 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:14.658895 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd6dbcf24954c6cdbca3ad8c394096f50568ceedbb37ebe8f38bce06ec29adac"} err="failed to get container status \"cd6dbcf24954c6cdbca3ad8c394096f50568ceedbb37ebe8f38bce06ec29adac\": rpc error: code = NotFound desc = could not find container \"cd6dbcf24954c6cdbca3ad8c394096f50568ceedbb37ebe8f38bce06ec29adac\": container with ID starting with cd6dbcf24954c6cdbca3ad8c394096f50568ceedbb37ebe8f38bce06ec29adac not found: ID does not exist" May 28 15:17:14.682004 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:14.681982 2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-6sw42"] May 28 15:17:14.687779 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:14.687760 2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-6sw42"] May 28 15:17:14.729048 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:14.729028 2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/264d077b-757b-470a-85e5-fe4257145180-extensions-socket-volume\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:17:14.729048 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:14.729048 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47q2h\" (UniqueName: \"kubernetes.io/projected/264d077b-757b-470a-85e5-fe4257145180-kube-api-access-47q2h\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:17:15.713411 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:15.713375 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="264d077b-757b-470a-85e5-fe4257145180" path="/var/lib/kubelet/pods/264d077b-757b-470a-85e5-fe4257145180/volumes" May 28 15:17:17.871734 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:17.871698 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-79zsb"] May 28 15:17:17.872140 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:17.872099 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe94dbc4-ee3d-4f71-b68d-99823bd4f594" containerName="manager" May 28 15:17:17.872140 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:17.872114 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe94dbc4-ee3d-4f71-b68d-99823bd4f594" containerName="manager" May 28 15:17:17.872140 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:17.872126 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="264d077b-757b-470a-85e5-fe4257145180" containerName="manager" May 28 15:17:17.872140 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:17.872132 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="264d077b-757b-470a-85e5-fe4257145180" containerName="manager" May 28 15:17:17.872291 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:17.872190 2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="264d077b-757b-470a-85e5-fe4257145180" containerName="manager" May 28 15:17:17.872291 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:17.872202 2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="fe94dbc4-ee3d-4f71-b68d-99823bd4f594" containerName="manager" May 28 15:17:18.035466 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:18.035435 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-79zsb"] May 28 15:17:18.035623 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:18.035593 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-79zsb" May 28 15:17:18.038201 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:18.038182 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-xfptd\"" May 28 15:17:18.051675 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:18.051652 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c18ad486-95ca-4632-a07b-461e1c7ee15d-istio-data\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-79zsb\" (UID: \"c18ad486-95ca-4632-a07b-461e1c7ee15d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-79zsb" May 28 15:17:18.051790 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:18.051688 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c18ad486-95ca-4632-a07b-461e1c7ee15d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-79zsb\" (UID: \"c18ad486-95ca-4632-a07b-461e1c7ee15d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-79zsb" May 28 15:17:18.051790 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:18.051707 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c18ad486-95ca-4632-a07b-461e1c7ee15d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-79zsb\" (UID: \"c18ad486-95ca-4632-a07b-461e1c7ee15d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-79zsb" May 28 15:17:18.051907 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:18.051820 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c18ad486-95ca-4632-a07b-461e1c7ee15d-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-79zsb\" (UID: \"c18ad486-95ca-4632-a07b-461e1c7ee15d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-79zsb" May 28 15:17:18.051907 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:18.051853 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqzx7\" (UniqueName: \"kubernetes.io/projected/c18ad486-95ca-4632-a07b-461e1c7ee15d-kube-api-access-bqzx7\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-79zsb\" (UID: \"c18ad486-95ca-4632-a07b-461e1c7ee15d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-79zsb" May 28 15:17:18.051907 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:18.051887 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c18ad486-95ca-4632-a07b-461e1c7ee15d-credential-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-79zsb\" (UID: \"c18ad486-95ca-4632-a07b-461e1c7ee15d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-79zsb" May 28 15:17:18.051907 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:18.051905 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c18ad486-95ca-4632-a07b-461e1c7ee15d-workload-certs\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-79zsb\" (UID: \"c18ad486-95ca-4632-a07b-461e1c7ee15d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-79zsb" May 28 15:17:18.052101 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:18.051943 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c18ad486-95ca-4632-a07b-461e1c7ee15d-istio-token\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-79zsb\" (UID: \"c18ad486-95ca-4632-a07b-461e1c7ee15d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-79zsb" May 28 15:17:18.052101 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:18.051961 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c18ad486-95ca-4632-a07b-461e1c7ee15d-workload-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-79zsb\" (UID: \"c18ad486-95ca-4632-a07b-461e1c7ee15d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-79zsb" May 28 15:17:18.153235 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:18.153168 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c18ad486-95ca-4632-a07b-461e1c7ee15d-istio-token\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-79zsb\" (UID: \"c18ad486-95ca-4632-a07b-461e1c7ee15d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-79zsb" May 28 15:17:18.153235 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:18.153202 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c18ad486-95ca-4632-a07b-461e1c7ee15d-workload-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-79zsb\" (UID: \"c18ad486-95ca-4632-a07b-461e1c7ee15d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-79zsb" May 28 15:17:18.153410 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:18.153237 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c18ad486-95ca-4632-a07b-461e1c7ee15d-istio-data\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-79zsb\" (UID: \"c18ad486-95ca-4632-a07b-461e1c7ee15d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-79zsb" May 28 15:17:18.153410 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:18.153359 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c18ad486-95ca-4632-a07b-461e1c7ee15d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-79zsb\" (UID: \"c18ad486-95ca-4632-a07b-461e1c7ee15d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-79zsb" May 28 15:17:18.153410 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:18.153399 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c18ad486-95ca-4632-a07b-461e1c7ee15d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-79zsb\" (UID: \"c18ad486-95ca-4632-a07b-461e1c7ee15d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-79zsb" May 28 15:17:18.153589 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:18.153468 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c18ad486-95ca-4632-a07b-461e1c7ee15d-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-79zsb\" (UID: \"c18ad486-95ca-4632-a07b-461e1c7ee15d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-79zsb" May 28 15:17:18.153589 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:18.153518 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bqzx7\" (UniqueName: \"kubernetes.io/projected/c18ad486-95ca-4632-a07b-461e1c7ee15d-kube-api-access-bqzx7\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-79zsb\" (UID: \"c18ad486-95ca-4632-a07b-461e1c7ee15d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-79zsb" May 28 15:17:18.153589 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:18.153568 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c18ad486-95ca-4632-a07b-461e1c7ee15d-credential-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-79zsb\" (UID: \"c18ad486-95ca-4632-a07b-461e1c7ee15d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-79zsb" May 28 15:17:18.153747 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:18.153590 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c18ad486-95ca-4632-a07b-461e1c7ee15d-workload-certs\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-79zsb\" (UID: \"c18ad486-95ca-4632-a07b-461e1c7ee15d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-79zsb" May 28 15:17:18.153747 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:18.153591 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c18ad486-95ca-4632-a07b-461e1c7ee15d-istio-data\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-79zsb\" (UID: \"c18ad486-95ca-4632-a07b-461e1c7ee15d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-79zsb" May 28 15:17:18.153747 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:18.153679 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c18ad486-95ca-4632-a07b-461e1c7ee15d-workload-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-79zsb\" (UID: \"c18ad486-95ca-4632-a07b-461e1c7ee15d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-79zsb" May 28 15:17:18.154048 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:18.154026 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c18ad486-95ca-4632-a07b-461e1c7ee15d-credential-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-79zsb\" (UID: \"c18ad486-95ca-4632-a07b-461e1c7ee15d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-79zsb" May 28 15:17:18.154113 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:18.154035 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c18ad486-95ca-4632-a07b-461e1c7ee15d-workload-certs\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-79zsb\" (UID: \"c18ad486-95ca-4632-a07b-461e1c7ee15d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-79zsb" May 28 15:17:18.154177 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:18.154160 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c18ad486-95ca-4632-a07b-461e1c7ee15d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-79zsb\" (UID: \"c18ad486-95ca-4632-a07b-461e1c7ee15d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-79zsb" May 28 15:17:18.156146 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:18.156121 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c18ad486-95ca-4632-a07b-461e1c7ee15d-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-79zsb\" (UID: \"c18ad486-95ca-4632-a07b-461e1c7ee15d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-79zsb" May 28 15:17:18.156296 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:18.156278 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c18ad486-95ca-4632-a07b-461e1c7ee15d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-79zsb\" (UID: \"c18ad486-95ca-4632-a07b-461e1c7ee15d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-79zsb" May 28 15:17:18.162188 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:18.162162 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c18ad486-95ca-4632-a07b-461e1c7ee15d-istio-token\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-79zsb\" (UID: \"c18ad486-95ca-4632-a07b-461e1c7ee15d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-79zsb" May 28 15:17:18.162464 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:18.162444 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqzx7\" (UniqueName: \"kubernetes.io/projected/c18ad486-95ca-4632-a07b-461e1c7ee15d-kube-api-access-bqzx7\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-79zsb\" (UID: \"c18ad486-95ca-4632-a07b-461e1c7ee15d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-79zsb" May 28 15:17:18.345958 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:18.345934 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-79zsb" May 28 15:17:18.481813 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:18.481779 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-79zsb"] May 28 15:17:18.483695 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:17:18.483666 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc18ad486_95ca_4632_a07b_461e1c7ee15d.slice/crio-59016ea1e2fc3bb379a55ba53e570d200e8055870a7464ac46543a3403541e9a WatchSource:0}: Error finding container 59016ea1e2fc3bb379a55ba53e570d200e8055870a7464ac46543a3403541e9a: Status 404 returned error can't find the container with id 59016ea1e2fc3bb379a55ba53e570d200e8055870a7464ac46543a3403541e9a May 28 15:17:18.485952 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:18.485917 2559 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} May 28 15:17:18.486033 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:18.486005 2559 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} May 28 15:17:18.486071 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:18.486051 2559 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} May 28 15:17:18.667099 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:18.667065 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-79zsb" event={"ID":"c18ad486-95ca-4632-a07b-461e1c7ee15d","Type":"ContainerStarted","Data":"f3aec570e08fb8b77cf65d59aed7768646e7eccbfd2bef3ebc2e2eae695f1b78"} May 28 15:17:18.667099 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:18.667102 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-79zsb" event={"ID":"c18ad486-95ca-4632-a07b-461e1c7ee15d","Type":"ContainerStarted","Data":"59016ea1e2fc3bb379a55ba53e570d200e8055870a7464ac46543a3403541e9a"} May 28 15:17:18.687554 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:18.687450 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-79zsb" podStartSLOduration=1.687437753 podStartE2EDuration="1.687437753s" podCreationTimestamp="2026-05-28 15:17:17 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 15:17:18.686835154 +0000 UTC m=+589.575822239" watchObservedRunningTime="2026-05-28 15:17:18.687437753 +0000 UTC m=+589.576424837" May 28 15:17:19.346310 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:19.346278 2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-79zsb" May 28 15:17:19.351238 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:19.351214 2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-79zsb" May 28 15:17:19.674162 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:19.674082 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-79zsb" May 28 15:17:19.675090 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:19.675069 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-79zsb" May 28 15:17:31.431556 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:31.431521 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:17:31.433784 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:31.433768 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-78c99df468-kvjvq" May 28 15:17:31.436189 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:31.436169 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" May 28 15:17:31.444923 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:31.444893 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:17:31.457380 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:31.457355 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0c1b7268-aef5-47b4-ad20-fd2fa9785f47-config-file\") pod \"limitador-limitador-78c99df468-kvjvq\" (UID: \"0c1b7268-aef5-47b4-ad20-fd2fa9785f47\") " pod="kuadrant-system/limitador-limitador-78c99df468-kvjvq" May 28 15:17:31.457465 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:31.457408 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z28cl\" (UniqueName: \"kubernetes.io/projected/0c1b7268-aef5-47b4-ad20-fd2fa9785f47-kube-api-access-z28cl\") pod \"limitador-limitador-78c99df468-kvjvq\" (UID: \"0c1b7268-aef5-47b4-ad20-fd2fa9785f47\") " pod="kuadrant-system/limitador-limitador-78c99df468-kvjvq" May 28 15:17:31.472771 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:31.472741 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:17:31.558266 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:31.558246 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0c1b7268-aef5-47b4-ad20-fd2fa9785f47-config-file\") pod \"limitador-limitador-78c99df468-kvjvq\" (UID: \"0c1b7268-aef5-47b4-ad20-fd2fa9785f47\") " pod="kuadrant-system/limitador-limitador-78c99df468-kvjvq" May 28 15:17:31.558360 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:31.558308 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z28cl\" (UniqueName: \"kubernetes.io/projected/0c1b7268-aef5-47b4-ad20-fd2fa9785f47-kube-api-access-z28cl\") pod \"limitador-limitador-78c99df468-kvjvq\" (UID: \"0c1b7268-aef5-47b4-ad20-fd2fa9785f47\") " pod="kuadrant-system/limitador-limitador-78c99df468-kvjvq" May 28 15:17:31.559020 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:31.558996 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0c1b7268-aef5-47b4-ad20-fd2fa9785f47-config-file\") pod \"limitador-limitador-78c99df468-kvjvq\" (UID: \"0c1b7268-aef5-47b4-ad20-fd2fa9785f47\") " pod="kuadrant-system/limitador-limitador-78c99df468-kvjvq" May 28 15:17:31.566075 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:31.566054 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z28cl\" (UniqueName: \"kubernetes.io/projected/0c1b7268-aef5-47b4-ad20-fd2fa9785f47-kube-api-access-z28cl\") pod \"limitador-limitador-78c99df468-kvjvq\" (UID: \"0c1b7268-aef5-47b4-ad20-fd2fa9785f47\") " pod="kuadrant-system/limitador-limitador-78c99df468-kvjvq" May 28 15:17:31.743949 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:31.743907 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-78c99df468-kvjvq" May 28 15:17:31.869930 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:31.869857 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:17:31.871252 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:17:31.871227 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c1b7268_aef5_47b4_ad20_fd2fa9785f47.slice/crio-be06af8eca6eee52652bcb826066ea6736dd02ec055248af32957e88f1719320 WatchSource:0}: Error finding container be06af8eca6eee52652bcb826066ea6736dd02ec055248af32957e88f1719320: Status 404 returned error can't find the container with id be06af8eca6eee52652bcb826066ea6736dd02ec055248af32957e88f1719320 May 28 15:17:31.996754 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:31.996701 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7498df8756-7pwbt"] May 28 15:17:31.999825 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:31.999810 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-7pwbt" May 28 15:17:32.002183 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:32.002163 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-lm6l6\"" May 28 15:17:32.005683 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:32.005663 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7498df8756-7pwbt"] May 28 15:17:32.062101 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:32.062073 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2gxn\" (UniqueName: \"kubernetes.io/projected/0ef6f20a-1d1c-4ab8-b7a5-db800a8e410f-kube-api-access-j2gxn\") pod \"authorino-7498df8756-7pwbt\" (UID: \"0ef6f20a-1d1c-4ab8-b7a5-db800a8e410f\") " pod="kuadrant-system/authorino-7498df8756-7pwbt" May 28 15:17:32.162826 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:32.162790 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j2gxn\" (UniqueName: \"kubernetes.io/projected/0ef6f20a-1d1c-4ab8-b7a5-db800a8e410f-kube-api-access-j2gxn\") pod \"authorino-7498df8756-7pwbt\" (UID: \"0ef6f20a-1d1c-4ab8-b7a5-db800a8e410f\") " pod="kuadrant-system/authorino-7498df8756-7pwbt" May 28 15:17:32.170582 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:32.170561 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2gxn\" (UniqueName: \"kubernetes.io/projected/0ef6f20a-1d1c-4ab8-b7a5-db800a8e410f-kube-api-access-j2gxn\") pod \"authorino-7498df8756-7pwbt\" (UID: \"0ef6f20a-1d1c-4ab8-b7a5-db800a8e410f\") " pod="kuadrant-system/authorino-7498df8756-7pwbt" May 28 15:17:32.309665 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:32.309594 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-7pwbt" May 28 15:17:32.479495 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:32.479406 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7498df8756-7pwbt"] May 28 15:17:32.482671 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:17:32.482639 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0ef6f20a_1d1c_4ab8_b7a5_db800a8e410f.slice/crio-688eed78358f3a4f3efbe0f3827c2280f21b183bca75b5d1a286baee116a7632 WatchSource:0}: Error finding container 688eed78358f3a4f3efbe0f3827c2280f21b183bca75b5d1a286baee116a7632: Status 404 returned error can't find the container with id 688eed78358f3a4f3efbe0f3827c2280f21b183bca75b5d1a286baee116a7632 May 28 15:17:32.732424 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:32.732338 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-7pwbt" event={"ID":"0ef6f20a-1d1c-4ab8-b7a5-db800a8e410f","Type":"ContainerStarted","Data":"688eed78358f3a4f3efbe0f3827c2280f21b183bca75b5d1a286baee116a7632"} May 28 15:17:32.733431 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:32.733400 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-78c99df468-kvjvq" event={"ID":"0c1b7268-aef5-47b4-ad20-fd2fa9785f47","Type":"ContainerStarted","Data":"be06af8eca6eee52652bcb826066ea6736dd02ec055248af32957e88f1719320"} May 28 15:17:36.753832 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:36.753795 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-7pwbt" event={"ID":"0ef6f20a-1d1c-4ab8-b7a5-db800a8e410f","Type":"ContainerStarted","Data":"a53bd549f5e37f14a1c23d7cc75ca194dfdddc6bd77260de283a0af9dcc0bbec"} May 28 15:17:36.755286 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:36.755256 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-78c99df468-kvjvq" event={"ID":"0c1b7268-aef5-47b4-ad20-fd2fa9785f47","Type":"ContainerStarted","Data":"0b8c9ad50a03cedee65ef013877c5c55e85dde9bbb941e61940ee46fc49065f8"} May 28 15:17:36.755405 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:36.755332 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-78c99df468-kvjvq" May 28 15:17:36.767129 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:36.767084 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-7498df8756-7pwbt" podStartSLOduration=2.197170572 podStartE2EDuration="5.767072691s" podCreationTimestamp="2026-05-28 15:17:31 +0000 UTC" firstStartedPulling="2026-05-28 15:17:32.486033065 +0000 UTC m=+603.375020131" lastFinishedPulling="2026-05-28 15:17:36.055935178 +0000 UTC m=+606.944922250" observedRunningTime="2026-05-28 15:17:36.766444512 +0000 UTC m=+607.655431596" watchObservedRunningTime="2026-05-28 15:17:36.767072691 +0000 UTC m=+607.656059773" May 28 15:17:36.782879 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:36.782834 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-78c99df468-kvjvq" podStartSLOduration=1.605110062 podStartE2EDuration="5.782822132s" podCreationTimestamp="2026-05-28 15:17:31 +0000 UTC" firstStartedPulling="2026-05-28 15:17:31.872927606 +0000 UTC m=+602.761914671" lastFinishedPulling="2026-05-28 15:17:36.050639666 +0000 UTC m=+606.939626741" observedRunningTime="2026-05-28 15:17:36.780728616 +0000 UTC m=+607.669715718" watchObservedRunningTime="2026-05-28 15:17:36.782822132 +0000 UTC m=+607.671809238" May 28 15:17:47.759897 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:17:47.759867 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-78c99df468-kvjvq" May 28 15:18:06.524927 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:18:06.524892 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-74bfbd878-slkb4"] May 28 15:18:06.585885 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:18:06.585854 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-74bfbd878-slkb4"] May 28 15:18:06.586028 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:18:06.585945 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-74bfbd878-slkb4" May 28 15:18:06.588509 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:18:06.588474 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" May 28 15:18:06.745657 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:18:06.745637 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9rcn\" (UniqueName: \"kubernetes.io/projected/33c2e433-d007-4c4f-ba6c-d2038fbc8ff2-kube-api-access-w9rcn\") pod \"authorino-74bfbd878-slkb4\" (UID: \"33c2e433-d007-4c4f-ba6c-d2038fbc8ff2\") " pod="kuadrant-system/authorino-74bfbd878-slkb4" May 28 15:18:06.745761 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:18:06.745684 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/33c2e433-d007-4c4f-ba6c-d2038fbc8ff2-tls-cert\") pod \"authorino-74bfbd878-slkb4\" (UID: \"33c2e433-d007-4c4f-ba6c-d2038fbc8ff2\") " pod="kuadrant-system/authorino-74bfbd878-slkb4" May 28 15:18:06.846692 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:18:06.846640 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w9rcn\" (UniqueName: \"kubernetes.io/projected/33c2e433-d007-4c4f-ba6c-d2038fbc8ff2-kube-api-access-w9rcn\") pod \"authorino-74bfbd878-slkb4\" (UID: \"33c2e433-d007-4c4f-ba6c-d2038fbc8ff2\") " pod="kuadrant-system/authorino-74bfbd878-slkb4" May 28 15:18:06.846692 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:18:06.846686 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/33c2e433-d007-4c4f-ba6c-d2038fbc8ff2-tls-cert\") pod \"authorino-74bfbd878-slkb4\" (UID: \"33c2e433-d007-4c4f-ba6c-d2038fbc8ff2\") " pod="kuadrant-system/authorino-74bfbd878-slkb4" May 28 15:18:06.849061 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:18:06.849039 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/33c2e433-d007-4c4f-ba6c-d2038fbc8ff2-tls-cert\") pod \"authorino-74bfbd878-slkb4\" (UID: \"33c2e433-d007-4c4f-ba6c-d2038fbc8ff2\") " pod="kuadrant-system/authorino-74bfbd878-slkb4" May 28 15:18:06.856778 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:18:06.856753 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9rcn\" (UniqueName: \"kubernetes.io/projected/33c2e433-d007-4c4f-ba6c-d2038fbc8ff2-kube-api-access-w9rcn\") pod \"authorino-74bfbd878-slkb4\" (UID: \"33c2e433-d007-4c4f-ba6c-d2038fbc8ff2\") " pod="kuadrant-system/authorino-74bfbd878-slkb4" May 28 15:18:06.894734 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:18:06.894711 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-74bfbd878-slkb4" May 28 15:18:07.018271 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:18:07.018246 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-74bfbd878-slkb4"] May 28 15:18:07.019929 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:18:07.019907 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod33c2e433_d007_4c4f_ba6c_d2038fbc8ff2.slice/crio-a33b9436551a8239e0477bb8d6140f0f8c8ab0a9f476d79f43e419a53f2d47de WatchSource:0}: Error finding container a33b9436551a8239e0477bb8d6140f0f8c8ab0a9f476d79f43e419a53f2d47de: Status 404 returned error can't find the container with id a33b9436551a8239e0477bb8d6140f0f8c8ab0a9f476d79f43e419a53f2d47de May 28 15:18:07.021583 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:18:07.021563 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 28 15:18:07.863202 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:18:07.863122 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-74bfbd878-slkb4" event={"ID":"33c2e433-d007-4c4f-ba6c-d2038fbc8ff2","Type":"ContainerStarted","Data":"bdb61a13080ceb087d9709a61abf22eb690f56ec7f43ec3ee2ae65acb9991b46"} May 28 15:18:07.863202 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:18:07.863162 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-74bfbd878-slkb4" event={"ID":"33c2e433-d007-4c4f-ba6c-d2038fbc8ff2","Type":"ContainerStarted","Data":"a33b9436551a8239e0477bb8d6140f0f8c8ab0a9f476d79f43e419a53f2d47de"} May 28 15:18:07.878958 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:18:07.878912 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-74bfbd878-slkb4" podStartSLOduration=1.30121295 podStartE2EDuration="1.878899356s" podCreationTimestamp="2026-05-28 15:18:06 +0000 UTC" firstStartedPulling="2026-05-28 15:18:07.021684778 +0000 UTC m=+637.910671839" lastFinishedPulling="2026-05-28 15:18:07.599371169 +0000 UTC m=+638.488358245" observedRunningTime="2026-05-28 15:18:07.877610504 +0000 UTC m=+638.766597588" watchObservedRunningTime="2026-05-28 15:18:07.878899356 +0000 UTC m=+638.767886471" May 28 15:18:07.903987 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:18:07.903962 2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7498df8756-7pwbt"] May 28 15:18:07.904146 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:18:07.904129 2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-7498df8756-7pwbt" podUID="0ef6f20a-1d1c-4ab8-b7a5-db800a8e410f" containerName="authorino" containerID="cri-o://a53bd549f5e37f14a1c23d7cc75ca194dfdddc6bd77260de283a0af9dcc0bbec" gracePeriod=30 May 28 15:18:08.154903 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:18:08.154883 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-7pwbt" May 28 15:18:08.258746 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:18:08.258718 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2gxn\" (UniqueName: \"kubernetes.io/projected/0ef6f20a-1d1c-4ab8-b7a5-db800a8e410f-kube-api-access-j2gxn\") pod \"0ef6f20a-1d1c-4ab8-b7a5-db800a8e410f\" (UID: \"0ef6f20a-1d1c-4ab8-b7a5-db800a8e410f\") " May 28 15:18:08.260657 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:18:08.260630 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ef6f20a-1d1c-4ab8-b7a5-db800a8e410f-kube-api-access-j2gxn" (OuterVolumeSpecName: "kube-api-access-j2gxn") pod "0ef6f20a-1d1c-4ab8-b7a5-db800a8e410f" (UID: "0ef6f20a-1d1c-4ab8-b7a5-db800a8e410f"). InnerVolumeSpecName "kube-api-access-j2gxn". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 15:18:08.360317 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:18:08.360296 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2gxn\" (UniqueName: \"kubernetes.io/projected/0ef6f20a-1d1c-4ab8-b7a5-db800a8e410f-kube-api-access-j2gxn\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:18:08.867899 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:18:08.867865 2559 generic.go:358] "Generic (PLEG): container finished" podID="0ef6f20a-1d1c-4ab8-b7a5-db800a8e410f" containerID="a53bd549f5e37f14a1c23d7cc75ca194dfdddc6bd77260de283a0af9dcc0bbec" exitCode=0 May 28 15:18:08.868301 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:18:08.867915 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-7pwbt" May 28 15:18:08.868301 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:18:08.867943 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-7pwbt" event={"ID":"0ef6f20a-1d1c-4ab8-b7a5-db800a8e410f","Type":"ContainerDied","Data":"a53bd549f5e37f14a1c23d7cc75ca194dfdddc6bd77260de283a0af9dcc0bbec"} May 28 15:18:08.868301 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:18:08.867975 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-7pwbt" event={"ID":"0ef6f20a-1d1c-4ab8-b7a5-db800a8e410f","Type":"ContainerDied","Data":"688eed78358f3a4f3efbe0f3827c2280f21b183bca75b5d1a286baee116a7632"} May 28 15:18:08.868301 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:18:08.867991 2559 scope.go:117] "RemoveContainer" containerID="a53bd549f5e37f14a1c23d7cc75ca194dfdddc6bd77260de283a0af9dcc0bbec" May 28 15:18:08.876622 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:18:08.876606 2559 scope.go:117] "RemoveContainer" containerID="a53bd549f5e37f14a1c23d7cc75ca194dfdddc6bd77260de283a0af9dcc0bbec" May 28 15:18:08.876861 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:18:08.876844 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a53bd549f5e37f14a1c23d7cc75ca194dfdddc6bd77260de283a0af9dcc0bbec\": container with ID starting with a53bd549f5e37f14a1c23d7cc75ca194dfdddc6bd77260de283a0af9dcc0bbec not found: ID does not exist" containerID="a53bd549f5e37f14a1c23d7cc75ca194dfdddc6bd77260de283a0af9dcc0bbec" May 28 15:18:08.876917 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:18:08.876869 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a53bd549f5e37f14a1c23d7cc75ca194dfdddc6bd77260de283a0af9dcc0bbec"} err="failed to get container status \"a53bd549f5e37f14a1c23d7cc75ca194dfdddc6bd77260de283a0af9dcc0bbec\": rpc error: code = NotFound desc = could not find container \"a53bd549f5e37f14a1c23d7cc75ca194dfdddc6bd77260de283a0af9dcc0bbec\": container with ID starting with a53bd549f5e37f14a1c23d7cc75ca194dfdddc6bd77260de283a0af9dcc0bbec not found: ID does not exist" May 28 15:18:08.889426 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:18:08.889404 2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7498df8756-7pwbt"] May 28 15:18:08.893135 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:18:08.893115 2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7498df8756-7pwbt"] May 28 15:18:09.713728 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:18:09.713699 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ef6f20a-1d1c-4ab8-b7a5-db800a8e410f" path="/var/lib/kubelet/pods/0ef6f20a-1d1c-4ab8-b7a5-db800a8e410f/volumes" May 28 15:18:45.740902 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:18:45.740869 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:19:00.042056 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:19:00.042020 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:19:12.536883 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:19:12.536846 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:19:15.739929 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:19:15.739894 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:19:20.538951 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:19:20.538837 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:19:32.142987 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:19:32.142958 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:20:27.224553 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:20:27.224518 2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-74bfbd878-slkb4"] May 28 15:20:27.225852 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:20:27.225368 2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-74bfbd878-slkb4" podUID="33c2e433-d007-4c4f-ba6c-d2038fbc8ff2" containerName="authorino" containerID="cri-o://bdb61a13080ceb087d9709a61abf22eb690f56ec7f43ec3ee2ae65acb9991b46" gracePeriod=30 May 28 15:20:27.386075 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:20:27.386041 2559 generic.go:358] "Generic (PLEG): container finished" podID="33c2e433-d007-4c4f-ba6c-d2038fbc8ff2" containerID="bdb61a13080ceb087d9709a61abf22eb690f56ec7f43ec3ee2ae65acb9991b46" exitCode=0 May 28 15:20:27.386229 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:20:27.386115 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-74bfbd878-slkb4" event={"ID":"33c2e433-d007-4c4f-ba6c-d2038fbc8ff2","Type":"ContainerDied","Data":"bdb61a13080ceb087d9709a61abf22eb690f56ec7f43ec3ee2ae65acb9991b46"} May 28 15:20:27.476671 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:20:27.476616 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-74bfbd878-slkb4" May 28 15:20:27.499541 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:20:27.499509 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/33c2e433-d007-4c4f-ba6c-d2038fbc8ff2-tls-cert\") pod \"33c2e433-d007-4c4f-ba6c-d2038fbc8ff2\" (UID: \"33c2e433-d007-4c4f-ba6c-d2038fbc8ff2\") " May 28 15:20:27.499696 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:20:27.499554 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w9rcn\" (UniqueName: \"kubernetes.io/projected/33c2e433-d007-4c4f-ba6c-d2038fbc8ff2-kube-api-access-w9rcn\") pod \"33c2e433-d007-4c4f-ba6c-d2038fbc8ff2\" (UID: \"33c2e433-d007-4c4f-ba6c-d2038fbc8ff2\") " May 28 15:20:27.501515 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:20:27.501475 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33c2e433-d007-4c4f-ba6c-d2038fbc8ff2-kube-api-access-w9rcn" (OuterVolumeSpecName: "kube-api-access-w9rcn") pod "33c2e433-d007-4c4f-ba6c-d2038fbc8ff2" (UID: "33c2e433-d007-4c4f-ba6c-d2038fbc8ff2"). InnerVolumeSpecName "kube-api-access-w9rcn". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 15:20:27.512280 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:20:27.512253 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33c2e433-d007-4c4f-ba6c-d2038fbc8ff2-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "33c2e433-d007-4c4f-ba6c-d2038fbc8ff2" (UID: "33c2e433-d007-4c4f-ba6c-d2038fbc8ff2"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 28 15:20:27.601159 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:20:27.601128 2559 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/33c2e433-d007-4c4f-ba6c-d2038fbc8ff2-tls-cert\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:20:27.601159 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:20:27.601155 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w9rcn\" (UniqueName: \"kubernetes.io/projected/33c2e433-d007-4c4f-ba6c-d2038fbc8ff2-kube-api-access-w9rcn\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:20:28.391084 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:20:28.391053 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-74bfbd878-slkb4" May 28 15:20:28.391563 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:20:28.391100 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-74bfbd878-slkb4" event={"ID":"33c2e433-d007-4c4f-ba6c-d2038fbc8ff2","Type":"ContainerDied","Data":"a33b9436551a8239e0477bb8d6140f0f8c8ab0a9f476d79f43e419a53f2d47de"} May 28 15:20:28.391563 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:20:28.391155 2559 scope.go:117] "RemoveContainer" containerID="bdb61a13080ceb087d9709a61abf22eb690f56ec7f43ec3ee2ae65acb9991b46" May 28 15:20:28.407222 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:20:28.407198 2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-74bfbd878-slkb4"] May 28 15:20:28.415424 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:20:28.415403 2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-74bfbd878-slkb4"] May 28 15:20:29.714473 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:20:29.714439 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33c2e433-d007-4c4f-ba6c-d2038fbc8ff2" path="/var/lib/kubelet/pods/33c2e433-d007-4c4f-ba6c-d2038fbc8ff2/volumes" May 28 15:20:55.155211 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:20:55.155172 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:21:05.243295 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:21:05.243264 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:21:14.141758 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:21:14.141721 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:21:24.441716 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:21:24.441678 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:21:33.338453 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:21:33.338415 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:21:43.851088 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:21:43.851043 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:22:56.544164 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:22:56.544126 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:23:11.843679 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:23:11.843620 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:23:50.237226 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:23:50.237143 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:24:06.729868 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:24:06.729832 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:24:20.935420 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:24:20.935376 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:24:37.336800 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:24:37.336760 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:25:30.647831 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:25:30.647789 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:25:40.535800 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:25:40.535752 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:25:56.234711 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:25:56.234669 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:26:05.935501 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:26:05.935448 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:26:22.534659 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:26:22.534619 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:26:31.032794 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:26:31.032755 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:27:03.140365 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:27:03.140328 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:27:11.637303 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:27:11.637261 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:27:20.828621 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:27:20.828585 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:27:28.035637 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:27:28.035602 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:27:36.528723 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:27:36.528681 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:27:53.534541 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:27:53.534501 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:28:04.839387 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:28:04.839355 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:28:51.535587 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:28:51.535515 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:28:56.034296 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:28:56.034259 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:29:05.031992 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:29:05.031954 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:29:13.535863 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:29:13.535832 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:29:22.436671 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:29:22.436632 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:29:30.135623 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:29:30.135586 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:29:40.436282 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:29:40.436242 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:29:47.737304 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:29:47.737269 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:29:57.134713 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:29:57.134670 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:30:00.140053 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:30:00.140018 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29666370-6mjk6"] May 28 15:30:00.140420 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:30:00.140410 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ef6f20a-1d1c-4ab8-b7a5-db800a8e410f" containerName="authorino" May 28 15:30:00.140467 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:30:00.140423 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ef6f20a-1d1c-4ab8-b7a5-db800a8e410f" containerName="authorino" May 28 15:30:00.140467 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:30:00.140447 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33c2e433-d007-4c4f-ba6c-d2038fbc8ff2" containerName="authorino" May 28 15:30:00.140467 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:30:00.140452 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="33c2e433-d007-4c4f-ba6c-d2038fbc8ff2" containerName="authorino" May 28 15:30:00.140579 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:30:00.140529 2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="33c2e433-d007-4c4f-ba6c-d2038fbc8ff2" containerName="authorino" May 28 15:30:00.140579 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:30:00.140541 2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="0ef6f20a-1d1c-4ab8-b7a5-db800a8e410f" containerName="authorino" May 28 15:30:00.143450 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:30:00.143433 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29666370-6mjk6" May 28 15:30:00.145595 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:30:00.145569 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-74ftg\"" May 28 15:30:00.156929 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:30:00.156903 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29666370-6mjk6"] May 28 15:30:00.239235 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:30:00.239189 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d84xt\" (UniqueName: \"kubernetes.io/projected/f5125335-b0b9-4082-a11f-7cd274d42d0b-kube-api-access-d84xt\") pod \"maas-api-key-cleanup-29666370-6mjk6\" (UID: \"f5125335-b0b9-4082-a11f-7cd274d42d0b\") " pod="opendatahub/maas-api-key-cleanup-29666370-6mjk6" May 28 15:30:00.339886 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:30:00.339849 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d84xt\" (UniqueName: \"kubernetes.io/projected/f5125335-b0b9-4082-a11f-7cd274d42d0b-kube-api-access-d84xt\") pod \"maas-api-key-cleanup-29666370-6mjk6\" (UID: \"f5125335-b0b9-4082-a11f-7cd274d42d0b\") " pod="opendatahub/maas-api-key-cleanup-29666370-6mjk6" May 28 15:30:00.347448 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:30:00.347418 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d84xt\" (UniqueName: \"kubernetes.io/projected/f5125335-b0b9-4082-a11f-7cd274d42d0b-kube-api-access-d84xt\") pod \"maas-api-key-cleanup-29666370-6mjk6\" (UID: \"f5125335-b0b9-4082-a11f-7cd274d42d0b\") " pod="opendatahub/maas-api-key-cleanup-29666370-6mjk6" May 28 15:30:00.453428 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:30:00.453338 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29666370-6mjk6" May 28 15:30:00.583365 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:30:00.583337 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29666370-6mjk6"] May 28 15:30:00.585270 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:30:00.585236 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5125335_b0b9_4082_a11f_7cd274d42d0b.slice/crio-b09b673f9ee5322139bf2d9eaff52f491896ab730fd990652caa27d42b17e6fc WatchSource:0}: Error finding container b09b673f9ee5322139bf2d9eaff52f491896ab730fd990652caa27d42b17e6fc: Status 404 returned error can't find the container with id b09b673f9ee5322139bf2d9eaff52f491896ab730fd990652caa27d42b17e6fc May 28 15:30:00.586991 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:30:00.586976 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 28 15:30:01.535853 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:30:01.535812 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29666370-6mjk6" event={"ID":"f5125335-b0b9-4082-a11f-7cd274d42d0b","Type":"ContainerStarted","Data":"b09b673f9ee5322139bf2d9eaff52f491896ab730fd990652caa27d42b17e6fc"} May 28 15:30:03.544669 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:30:03.544629 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29666370-6mjk6" event={"ID":"f5125335-b0b9-4082-a11f-7cd274d42d0b","Type":"ContainerStarted","Data":"6726437a57ef866e09ae5582d84d392c157b3c9d08d5c21af43471e96af37fd9"} May 28 15:30:03.560851 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:30:03.560797 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29666370-6mjk6" podStartSLOduration=1.015156588 podStartE2EDuration="3.560782077s" podCreationTimestamp="2026-05-28 15:30:00 +0000 UTC" firstStartedPulling="2026-05-28 15:30:00.587102703 +0000 UTC m=+1351.476089765" lastFinishedPulling="2026-05-28 15:30:03.132728191 +0000 UTC m=+1354.021715254" observedRunningTime="2026-05-28 15:30:03.558300098 +0000 UTC m=+1354.447287235" watchObservedRunningTime="2026-05-28 15:30:03.560782077 +0000 UTC m=+1354.449769161" May 28 15:30:04.549708 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:30:04.549673 2559 generic.go:358] "Generic (PLEG): container finished" podID="f5125335-b0b9-4082-a11f-7cd274d42d0b" containerID="6726437a57ef866e09ae5582d84d392c157b3c9d08d5c21af43471e96af37fd9" exitCode=0 May 28 15:30:04.550084 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:30:04.549757 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29666370-6mjk6" event={"ID":"f5125335-b0b9-4082-a11f-7cd274d42d0b","Type":"ContainerDied","Data":"6726437a57ef866e09ae5582d84d392c157b3c9d08d5c21af43471e96af37fd9"} May 28 15:30:05.689067 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:30:05.689044 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29666370-6mjk6" May 28 15:30:05.745627 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:30:05.745592 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:30:05.792883 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:30:05.792856 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d84xt\" (UniqueName: \"kubernetes.io/projected/f5125335-b0b9-4082-a11f-7cd274d42d0b-kube-api-access-d84xt\") pod \"f5125335-b0b9-4082-a11f-7cd274d42d0b\" (UID: \"f5125335-b0b9-4082-a11f-7cd274d42d0b\") " May 28 15:30:05.795018 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:30:05.794990 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5125335-b0b9-4082-a11f-7cd274d42d0b-kube-api-access-d84xt" (OuterVolumeSpecName: "kube-api-access-d84xt") pod "f5125335-b0b9-4082-a11f-7cd274d42d0b" (UID: "f5125335-b0b9-4082-a11f-7cd274d42d0b"). InnerVolumeSpecName "kube-api-access-d84xt". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 15:30:05.893891 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:30:05.893805 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d84xt\" (UniqueName: \"kubernetes.io/projected/f5125335-b0b9-4082-a11f-7cd274d42d0b-kube-api-access-d84xt\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:30:06.559211 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:30:06.559179 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29666370-6mjk6" May 28 15:30:06.559211 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:30:06.559201 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29666370-6mjk6" event={"ID":"f5125335-b0b9-4082-a11f-7cd274d42d0b","Type":"ContainerDied","Data":"b09b673f9ee5322139bf2d9eaff52f491896ab730fd990652caa27d42b17e6fc"} May 28 15:30:06.559470 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:30:06.559236 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b09b673f9ee5322139bf2d9eaff52f491896ab730fd990652caa27d42b17e6fc" May 28 15:30:14.435429 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:30:14.435391 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:30:23.130970 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:30:23.130933 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:30:32.039634 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:30:32.039597 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:30:40.430557 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:30:40.430508 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:30:49.339405 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:30:49.339359 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:30:58.441377 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:30:58.441342 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:31:06.735866 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:31:06.735823 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:31:15.435647 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:31:15.435604 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:32:07.029366 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:32:07.029323 2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-z647b"] May 28 15:32:07.029965 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:32:07.029622 2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-z647b" podUID="0f21feb1-f535-41f2-846a-bf6b6e8c1cb8" containerName="manager" containerID="cri-o://fb76fb33a4461b2144a0a9eb2dac96ec14b78675919a7ae5744a815de609a11e" gracePeriod=10 May 28 15:32:07.386174 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:32:07.386141 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-z647b" May 28 15:32:07.562224 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:32:07.562138 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f21feb1-f535-41f2-846a-bf6b6e8c1cb8-extensions-socket-volume\") pod \"0f21feb1-f535-41f2-846a-bf6b6e8c1cb8\" (UID: \"0f21feb1-f535-41f2-846a-bf6b6e8c1cb8\") " May 28 15:32:07.562224 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:32:07.562210 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x4k49\" (UniqueName: \"kubernetes.io/projected/0f21feb1-f535-41f2-846a-bf6b6e8c1cb8-kube-api-access-x4k49\") pod \"0f21feb1-f535-41f2-846a-bf6b6e8c1cb8\" (UID: \"0f21feb1-f535-41f2-846a-bf6b6e8c1cb8\") " May 28 15:32:07.562555 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:32:07.562530 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f21feb1-f535-41f2-846a-bf6b6e8c1cb8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0f21feb1-f535-41f2-846a-bf6b6e8c1cb8" (UID: "0f21feb1-f535-41f2-846a-bf6b6e8c1cb8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 28 15:32:07.564177 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:32:07.564158 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f21feb1-f535-41f2-846a-bf6b6e8c1cb8-kube-api-access-x4k49" (OuterVolumeSpecName: "kube-api-access-x4k49") pod "0f21feb1-f535-41f2-846a-bf6b6e8c1cb8" (UID: "0f21feb1-f535-41f2-846a-bf6b6e8c1cb8"). InnerVolumeSpecName "kube-api-access-x4k49". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 15:32:07.663702 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:32:07.663657 2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f21feb1-f535-41f2-846a-bf6b6e8c1cb8-extensions-socket-volume\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:32:07.663702 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:32:07.663696 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4k49\" (UniqueName: \"kubernetes.io/projected/0f21feb1-f535-41f2-846a-bf6b6e8c1cb8-kube-api-access-x4k49\") on node \"ip-10-0-140-141.ec2.internal\" DevicePath \"\"" May 28 15:32:08.037448 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:32:08.037415 2559 generic.go:358] "Generic (PLEG): container finished" podID="0f21feb1-f535-41f2-846a-bf6b6e8c1cb8" containerID="fb76fb33a4461b2144a0a9eb2dac96ec14b78675919a7ae5744a815de609a11e" exitCode=0 May 28 15:32:08.037920 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:32:08.037517 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-z647b" event={"ID":"0f21feb1-f535-41f2-846a-bf6b6e8c1cb8","Type":"ContainerDied","Data":"fb76fb33a4461b2144a0a9eb2dac96ec14b78675919a7ae5744a815de609a11e"} May 28 15:32:08.037920 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:32:08.037535 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-z647b" May 28 15:32:08.037920 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:32:08.037556 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-z647b" event={"ID":"0f21feb1-f535-41f2-846a-bf6b6e8c1cb8","Type":"ContainerDied","Data":"030bbb8f6af3c73a2c850e48ed009b1fd32bdb9b05c292b3f40d5673689ce63d"} May 28 15:32:08.037920 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:32:08.037572 2559 scope.go:117] "RemoveContainer" containerID="fb76fb33a4461b2144a0a9eb2dac96ec14b78675919a7ae5744a815de609a11e" May 28 15:32:08.046220 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:32:08.046207 2559 scope.go:117] "RemoveContainer" containerID="fb76fb33a4461b2144a0a9eb2dac96ec14b78675919a7ae5744a815de609a11e" May 28 15:32:08.046472 ip-10-0-140-141 kubenswrapper[2559]: E0528 15:32:08.046450 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb76fb33a4461b2144a0a9eb2dac96ec14b78675919a7ae5744a815de609a11e\": container with ID starting with fb76fb33a4461b2144a0a9eb2dac96ec14b78675919a7ae5744a815de609a11e not found: ID does not exist" containerID="fb76fb33a4461b2144a0a9eb2dac96ec14b78675919a7ae5744a815de609a11e" May 28 15:32:08.046562 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:32:08.046496 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb76fb33a4461b2144a0a9eb2dac96ec14b78675919a7ae5744a815de609a11e"} err="failed to get container status \"fb76fb33a4461b2144a0a9eb2dac96ec14b78675919a7ae5744a815de609a11e\": rpc error: code = NotFound desc = could not find container \"fb76fb33a4461b2144a0a9eb2dac96ec14b78675919a7ae5744a815de609a11e\": container with ID starting with fb76fb33a4461b2144a0a9eb2dac96ec14b78675919a7ae5744a815de609a11e not found: ID does not exist" May 28 15:32:08.055944 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:32:08.055884 2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-z647b"] May 28 15:32:08.058515 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:32:08.058474 2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-z647b"] May 28 15:32:09.714107 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:32:09.714070 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f21feb1-f535-41f2-846a-bf6b6e8c1cb8" path="/var/lib/kubelet/pods/0f21feb1-f535-41f2-846a-bf6b6e8c1cb8/volumes" May 28 15:33:13.129543 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:33:13.129504 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-48r9z"] May 28 15:33:13.131831 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:33:13.129969 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f21feb1-f535-41f2-846a-bf6b6e8c1cb8" containerName="manager" May 28 15:33:13.131831 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:33:13.129985 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f21feb1-f535-41f2-846a-bf6b6e8c1cb8" containerName="manager" May 28 15:33:13.131831 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:33:13.129997 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5125335-b0b9-4082-a11f-7cd274d42d0b" containerName="cleanup" May 28 15:33:13.131831 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:33:13.130002 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5125335-b0b9-4082-a11f-7cd274d42d0b" containerName="cleanup" May 28 15:33:13.131831 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:33:13.130064 2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="0f21feb1-f535-41f2-846a-bf6b6e8c1cb8" containerName="manager" May 28 15:33:13.131831 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:33:13.130076 2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="f5125335-b0b9-4082-a11f-7cd274d42d0b" containerName="cleanup" May 28 15:33:13.132879 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:33:13.132864 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-48r9z" May 28 15:33:13.135503 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:33:13.135468 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-lgmkq\"" May 28 15:33:13.142823 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:33:13.142804 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-48r9z"] May 28 15:33:13.203709 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:33:13.203677 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jmw2\" (UniqueName: \"kubernetes.io/projected/033849c4-a13e-47a9-a0b1-309e9f86c941-kube-api-access-6jmw2\") pod \"kuadrant-operator-controller-manager-55c7f4c975-48r9z\" (UID: \"033849c4-a13e-47a9-a0b1-309e9f86c941\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-48r9z" May 28 15:33:13.203841 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:33:13.203800 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/033849c4-a13e-47a9-a0b1-309e9f86c941-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-48r9z\" (UID: \"033849c4-a13e-47a9-a0b1-309e9f86c941\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-48r9z" May 28 15:33:13.304500 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:33:13.304461 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/033849c4-a13e-47a9-a0b1-309e9f86c941-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-48r9z\" (UID: \"033849c4-a13e-47a9-a0b1-309e9f86c941\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-48r9z" May 28 15:33:13.304644 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:33:13.304546 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6jmw2\" (UniqueName: \"kubernetes.io/projected/033849c4-a13e-47a9-a0b1-309e9f86c941-kube-api-access-6jmw2\") pod \"kuadrant-operator-controller-manager-55c7f4c975-48r9z\" (UID: \"033849c4-a13e-47a9-a0b1-309e9f86c941\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-48r9z" May 28 15:33:13.304858 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:33:13.304841 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/033849c4-a13e-47a9-a0b1-309e9f86c941-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-48r9z\" (UID: \"033849c4-a13e-47a9-a0b1-309e9f86c941\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-48r9z" May 28 15:33:13.313818 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:33:13.313768 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jmw2\" (UniqueName: \"kubernetes.io/projected/033849c4-a13e-47a9-a0b1-309e9f86c941-kube-api-access-6jmw2\") pod \"kuadrant-operator-controller-manager-55c7f4c975-48r9z\" (UID: \"033849c4-a13e-47a9-a0b1-309e9f86c941\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-48r9z" May 28 15:33:13.443533 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:33:13.443461 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-48r9z" May 28 15:33:13.583011 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:33:13.582978 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-48r9z"] May 28 15:33:13.584779 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:33:13.584755 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod033849c4_a13e_47a9_a0b1_309e9f86c941.slice/crio-bf587456acab405c1dfee7cdbbd0bcfde2e62f6de05e1493d219d08d04ba81d7 WatchSource:0}: Error finding container bf587456acab405c1dfee7cdbbd0bcfde2e62f6de05e1493d219d08d04ba81d7: Status 404 returned error can't find the container with id bf587456acab405c1dfee7cdbbd0bcfde2e62f6de05e1493d219d08d04ba81d7 May 28 15:33:14.295801 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:33:14.295761 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-48r9z" event={"ID":"033849c4-a13e-47a9-a0b1-309e9f86c941","Type":"ContainerStarted","Data":"7a304c81c562bd720c2a450f63fb9fa81e132f385acd502eb5d23f288492ba1a"} May 28 15:33:14.295801 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:33:14.295803 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-48r9z" event={"ID":"033849c4-a13e-47a9-a0b1-309e9f86c941","Type":"ContainerStarted","Data":"bf587456acab405c1dfee7cdbbd0bcfde2e62f6de05e1493d219d08d04ba81d7"} May 28 15:33:14.296217 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:33:14.295834 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-48r9z" May 28 15:33:14.316784 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:33:14.316738 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-48r9z" podStartSLOduration=1.316724854 podStartE2EDuration="1.316724854s" podCreationTimestamp="2026-05-28 15:33:13 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 15:33:14.313591937 +0000 UTC m=+1545.202579021" watchObservedRunningTime="2026-05-28 15:33:14.316724854 +0000 UTC m=+1545.205711937" May 28 15:33:25.301364 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:33:25.301328 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-48r9z" May 28 15:33:33.240447 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:33:33.240410 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:33:38.034205 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:33:38.034168 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:34:03.542814 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:34:03.542776 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:34:08.334808 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:34:08.334771 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:34:17.436624 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:34:17.436582 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:34:27.846006 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:34:27.845968 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:34:37.843984 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:34:37.843946 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:34:48.338697 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:34:48.338660 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:34:57.139254 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:34:57.139218 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:35:06.639049 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:35:06.639015 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:35:16.135630 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:35:16.135598 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:35:26.333906 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:35:26.333873 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:35:36.041882 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:35:36.041826 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:36:10.072797 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:36:10.072760 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:36:52.936012 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:36:52.935976 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:37:02.141591 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:37:02.141559 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:37:11.435058 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:37:11.435022 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:37:19.435180 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:37:19.435102 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:37:28.534548 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:37:28.534515 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:37:38.440915 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:37:38.440877 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:37:47.243429 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:37:47.243395 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:37:55.043142 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:37:55.043107 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:38:04.238165 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:38:04.238128 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:38:12.038626 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:38:12.038584 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:38:20.738782 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:38:20.738742 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:38:30.939709 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:38:30.939672 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:38:48.532854 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:38:48.532814 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:38:57.433736 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:38:57.433702 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:39:05.438888 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:39:05.438855 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:39:14.328712 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:39:14.328677 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:39:31.641418 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:39:31.641383 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:39:39.331498 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:39:39.331460 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:39:48.134846 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:39:48.134813 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:39:56.931458 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:39:56.931422 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:40:05.642466 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:40:05.642431 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:40:13.930259 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:40:13.930223 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:40:24.533593 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:40:24.533511 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:40:34.134804 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:40:34.134765 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:40:43.840354 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:40:43.840320 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:40:56.137992 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:40:56.137959 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:41:05.739115 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:41:05.739080 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:41:13.345354 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:41:13.345324 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:41:21.840589 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:41:21.840558 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:41:29.836362 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:41:29.836326 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:41:47.034764 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:41:47.034731 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:41:55.246018 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:41:55.245973 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:42:03.639202 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:42:03.639169 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:42:11.536630 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:42:11.536598 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:42:35.032820 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:42:35.032788 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:42:48.740682 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:42:48.740646 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kvjvq"] May 28 15:42:56.915516 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:42:56.915474 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29666370-6mjk6_f5125335-b0b9-4082-a11f-7cd274d42d0b/cleanup/0.log" May 28 15:42:57.270180 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:42:57.270152 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57c44ff6d8-b2zj5_0dbad4af-3d6a-4836-bee5-0c9b0e0a2013/manager/0.log" May 28 15:42:58.375053 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:42:58.375020 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759btbj4_1f2fed77-e2ed-4149-86f4-767f909314fc/util/0.log" May 28 15:42:58.380957 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:42:58.380932 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759btbj4_1f2fed77-e2ed-4149-86f4-767f909314fc/pull/0.log" May 28 15:42:58.386743 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:42:58.386708 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759btbj4_1f2fed77-e2ed-4149-86f4-767f909314fc/extract/0.log" May 28 15:42:58.487742 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:42:58.487718 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bz2jl_1cd08bac-f55e-4be0-b887-a50543cbba4b/extract/0.log" May 28 15:42:58.492809 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:42:58.492783 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bz2jl_1cd08bac-f55e-4be0-b887-a50543cbba4b/util/0.log" May 28 15:42:58.498148 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:42:58.498131 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bz2jl_1cd08bac-f55e-4be0-b887-a50543cbba4b/pull/0.log" May 28 15:42:58.613175 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:42:58.613148 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732246x_bf2518bc-3be2-4f08-9567-2a4e4f6aa833/util/0.log" May 28 15:42:58.619587 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:42:58.619560 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732246x_bf2518bc-3be2-4f08-9567-2a4e4f6aa833/pull/0.log" May 28 15:42:58.626281 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:42:58.626222 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732246x_bf2518bc-3be2-4f08-9567-2a4e4f6aa833/extract/0.log" May 28 15:42:59.087540 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:42:59.087515 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-648d5c98bc-mrdzf_f2253641-dc2e-4536-9ac0-9999e647ea53/manager/0.log" May 28 15:42:59.196971 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:42:59.196947 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-6cb54b5c86-946np_7b734797-e0f8-4bef-8537-d564d53d545e/kuadrant-console-plugin/0.log" May 28 15:42:59.314905 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:42:59.314884 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-k7g4d_01b6eda7-dbb7-4d91-bb5b-0a78899d6596/registry-server/0.log" May 28 15:42:59.438002 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:42:59.437939 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-55c7f4c975-48r9z_033849c4-a13e-47a9-a0b1-309e9f86c941/manager/0.log" May 28 15:42:59.543722 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:42:59.543695 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-78c99df468-kvjvq_0c1b7268-aef5-47b4-ad20-fd2fa9785f47/limitador/0.log" May 28 15:42:59.989439 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:42:59.989405 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-686dffbf4b45p4l_46389ad5-3b8d-4802-8fee-d069c467ab50/istio-proxy/0.log" May 28 15:43:00.458412 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:00.458384 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d6f447cf5-79zsb_c18ad486-95ca-4632-a07b-461e1c7ee15d/istio-proxy/0.log" May 28 15:43:00.681473 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:00.681447 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-69c468644b-ff99j_3889dc6c-fe09-49a2-b52a-cb4de8f769de/router/0.log" May 28 15:43:05.844602 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:05.844573 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xkl9v/must-gather-kczhc"] May 28 15:43:05.846943 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:05.846923 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xkl9v/must-gather-kczhc" May 28 15:43:05.849595 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:05.849577 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xkl9v\"/\"kube-root-ca.crt\"" May 28 15:43:05.850668 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:05.850648 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xkl9v\"/\"openshift-service-ca.crt\"" May 28 15:43:05.850777 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:05.850648 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-xkl9v\"/\"default-dockercfg-vtnc5\"" May 28 15:43:05.863355 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:05.863335 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xkl9v/must-gather-kczhc"] May 28 15:43:05.985024 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:05.984999 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8ff187d2-48d3-404b-bd79-d04d48b35905-must-gather-output\") pod \"must-gather-kczhc\" (UID: \"8ff187d2-48d3-404b-bd79-d04d48b35905\") " pod="openshift-must-gather-xkl9v/must-gather-kczhc" May 28 15:43:05.985123 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:05.985031 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmrb4\" (UniqueName: \"kubernetes.io/projected/8ff187d2-48d3-404b-bd79-d04d48b35905-kube-api-access-rmrb4\") pod \"must-gather-kczhc\" (UID: \"8ff187d2-48d3-404b-bd79-d04d48b35905\") " pod="openshift-must-gather-xkl9v/must-gather-kczhc" May 28 15:43:06.085585 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:06.085562 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8ff187d2-48d3-404b-bd79-d04d48b35905-must-gather-output\") pod \"must-gather-kczhc\" (UID: \"8ff187d2-48d3-404b-bd79-d04d48b35905\") " pod="openshift-must-gather-xkl9v/must-gather-kczhc" May 28 15:43:06.085685 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:06.085594 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rmrb4\" (UniqueName: \"kubernetes.io/projected/8ff187d2-48d3-404b-bd79-d04d48b35905-kube-api-access-rmrb4\") pod \"must-gather-kczhc\" (UID: \"8ff187d2-48d3-404b-bd79-d04d48b35905\") " pod="openshift-must-gather-xkl9v/must-gather-kczhc" May 28 15:43:06.085885 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:06.085867 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8ff187d2-48d3-404b-bd79-d04d48b35905-must-gather-output\") pod \"must-gather-kczhc\" (UID: \"8ff187d2-48d3-404b-bd79-d04d48b35905\") " pod="openshift-must-gather-xkl9v/must-gather-kczhc" May 28 15:43:06.095203 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:06.095154 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmrb4\" (UniqueName: \"kubernetes.io/projected/8ff187d2-48d3-404b-bd79-d04d48b35905-kube-api-access-rmrb4\") pod \"must-gather-kczhc\" (UID: \"8ff187d2-48d3-404b-bd79-d04d48b35905\") " pod="openshift-must-gather-xkl9v/must-gather-kczhc" May 28 15:43:06.155248 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:06.155228 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xkl9v/must-gather-kczhc" May 28 15:43:06.286306 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:06.286281 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xkl9v/must-gather-kczhc"] May 28 15:43:06.288180 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:43:06.288145 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8ff187d2_48d3_404b_bd79_d04d48b35905.slice/crio-3816ba8f6b2ebe2becd8c39e6a2a6947c35f14ce37ad44d97e39afdf27737680 WatchSource:0}: Error finding container 3816ba8f6b2ebe2becd8c39e6a2a6947c35f14ce37ad44d97e39afdf27737680: Status 404 returned error can't find the container with id 3816ba8f6b2ebe2becd8c39e6a2a6947c35f14ce37ad44d97e39afdf27737680 May 28 15:43:06.289976 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:06.289961 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 28 15:43:06.498262 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:06.498229 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xkl9v/must-gather-kczhc" event={"ID":"8ff187d2-48d3-404b-bd79-d04d48b35905","Type":"ContainerStarted","Data":"3816ba8f6b2ebe2becd8c39e6a2a6947c35f14ce37ad44d97e39afdf27737680"} May 28 15:43:07.505959 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:07.505922 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xkl9v/must-gather-kczhc" event={"ID":"8ff187d2-48d3-404b-bd79-d04d48b35905","Type":"ContainerStarted","Data":"530d8b934a6b5dc602a9516963bad30cd1077f787b4de38cd35cf111ae9ac1f3"} May 28 15:43:07.506510 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:07.506463 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xkl9v/must-gather-kczhc" event={"ID":"8ff187d2-48d3-404b-bd79-d04d48b35905","Type":"ContainerStarted","Data":"14d3e869992ff36903153009b6ddb7f0befbed35a4e43716b6c6972ab2a93c90"} May 28 15:43:07.523231 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:07.523172 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-xkl9v/must-gather-kczhc" podStartSLOduration=1.782918081 podStartE2EDuration="2.523154729s" podCreationTimestamp="2026-05-28 15:43:05 +0000 UTC" firstStartedPulling="2026-05-28 15:43:06.290078005 +0000 UTC m=+2137.179065066" lastFinishedPulling="2026-05-28 15:43:07.030314639 +0000 UTC m=+2137.919301714" observedRunningTime="2026-05-28 15:43:07.51966309 +0000 UTC m=+2138.408650174" watchObservedRunningTime="2026-05-28 15:43:07.523154729 +0000 UTC m=+2138.412141813" May 28 15:43:08.503308 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:08.503275 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-4jdp2_b64275b4-68e5-42e4-ac36-ac46f6d55524/global-pull-secret-syncer/0.log" May 28 15:43:08.629822 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:08.629789 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-bjm2p_c83b6a1e-4582-4530-8993-9b3ab169873d/konnectivity-agent/0.log" May 28 15:43:08.748762 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:08.748720 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-141.ec2.internal_f46265c427d4cbfcaecf493db63aaa49/haproxy/0.log" May 28 15:43:12.121627 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:12.121601 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759btbj4_1f2fed77-e2ed-4149-86f4-767f909314fc/extract/0.log" May 28 15:43:12.148038 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:12.148013 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759btbj4_1f2fed77-e2ed-4149-86f4-767f909314fc/util/0.log" May 28 15:43:12.186676 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:12.186642 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759btbj4_1f2fed77-e2ed-4149-86f4-767f909314fc/pull/0.log" May 28 15:43:12.230458 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:12.230419 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bz2jl_1cd08bac-f55e-4be0-b887-a50543cbba4b/extract/0.log" May 28 15:43:12.256214 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:12.256190 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bz2jl_1cd08bac-f55e-4be0-b887-a50543cbba4b/util/0.log" May 28 15:43:12.287503 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:12.287461 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bz2jl_1cd08bac-f55e-4be0-b887-a50543cbba4b/pull/0.log" May 28 15:43:12.321795 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:12.321762 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732246x_bf2518bc-3be2-4f08-9567-2a4e4f6aa833/extract/0.log" May 28 15:43:12.348181 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:12.348158 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732246x_bf2518bc-3be2-4f08-9567-2a4e4f6aa833/util/0.log" May 28 15:43:12.369865 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:12.369833 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732246x_bf2518bc-3be2-4f08-9567-2a4e4f6aa833/pull/0.log" May 28 15:43:12.690960 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:12.690930 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-648d5c98bc-mrdzf_f2253641-dc2e-4536-9ac0-9999e647ea53/manager/0.log" May 28 15:43:12.718734 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:12.718704 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-6cb54b5c86-946np_7b734797-e0f8-4bef-8537-d564d53d545e/kuadrant-console-plugin/0.log" May 28 15:43:12.768200 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:12.768167 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-k7g4d_01b6eda7-dbb7-4d91-bb5b-0a78899d6596/registry-server/0.log" May 28 15:43:12.833655 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:12.833622 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-55c7f4c975-48r9z_033849c4-a13e-47a9-a0b1-309e9f86c941/manager/0.log" May 28 15:43:12.867336 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:12.867304 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-78c99df468-kvjvq_0c1b7268-aef5-47b4-ad20-fd2fa9785f47/limitador/0.log" May 28 15:43:14.306438 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:14.306409 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6c75959455-7zpd4_b6714928-0dd7-4b82-b82c-9c83f3967f52/cluster-monitoring-operator/0.log" May 28 15:43:14.439824 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:14.439794 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7844b88b7-gjcqf_6db25964-8104-4046-bfe7-a17f56d28a66/metrics-server/0.log" May 28 15:43:14.535615 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:14.535583 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-455m8_1f665373-fcd1-43e4-a2a8-65c4a204d15b/node-exporter/0.log" May 28 15:43:14.562172 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:14.562138 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-455m8_1f665373-fcd1-43e4-a2a8-65c4a204d15b/kube-rbac-proxy/0.log" May 28 15:43:14.586080 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:14.586055 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-455m8_1f665373-fcd1-43e4-a2a8-65c4a204d15b/init-textfile/0.log" May 28 15:43:15.009605 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:15.009569 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7779bbdd78-9f8ct_1b3396aa-17d4-49e3-9220-ec3bb17cf2a8/prometheus-operator/0.log" May 28 15:43:15.028347 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:15.028309 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7779bbdd78-9f8ct_1b3396aa-17d4-49e3-9220-ec3bb17cf2a8/kube-rbac-proxy/0.log" May 28 15:43:15.052027 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:15.051994 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6c4bbc67b7-2v4zh_b906c71c-5fbb-4b0a-917c-8fc778742437/prometheus-operator-admission-webhook/0.log" May 28 15:43:15.078519 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:15.078473 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-9f5b64bf-mnlzs_59716931-45d5-4ad2-8fea-fc240f8155ac/telemeter-client/0.log" May 28 15:43:15.098206 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:15.098177 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-9f5b64bf-mnlzs_59716931-45d5-4ad2-8fea-fc240f8155ac/reload/0.log" May 28 15:43:15.119330 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:15.119286 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-9f5b64bf-mnlzs_59716931-45d5-4ad2-8fea-fc240f8155ac/kube-rbac-proxy/0.log" May 28 15:43:15.149439 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:15.149404 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5479d49877-hxbf6_0c588793-be04-492c-b10a-391b5bd3686b/thanos-query/0.log" May 28 15:43:15.173493 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:15.173403 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5479d49877-hxbf6_0c588793-be04-492c-b10a-391b5bd3686b/kube-rbac-proxy-web/0.log" May 28 15:43:15.192524 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:15.192472 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5479d49877-hxbf6_0c588793-be04-492c-b10a-391b5bd3686b/kube-rbac-proxy/0.log" May 28 15:43:15.216571 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:15.216541 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5479d49877-hxbf6_0c588793-be04-492c-b10a-391b5bd3686b/prom-label-proxy/0.log" May 28 15:43:15.239622 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:15.239598 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5479d49877-hxbf6_0c588793-be04-492c-b10a-391b5bd3686b/kube-rbac-proxy-rules/0.log" May 28 15:43:15.262165 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:15.262082 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5479d49877-hxbf6_0c588793-be04-492c-b10a-391b5bd3686b/kube-rbac-proxy-metrics/0.log" May 28 15:43:16.485194 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:16.485167 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-d4d8f94f9-9s6tj_cf3cad76-3b1f-4cce-bc1b-e3c710fd6a58/networking-console-plugin/0.log" May 28 15:43:16.915308 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:16.915269 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xkl9v/perf-node-gather-daemonset-npkmb"] May 28 15:43:16.920175 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:16.920155 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-npkmb" May 28 15:43:16.927129 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:16.927093 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xkl9v/perf-node-gather-daemonset-npkmb"] May 28 15:43:17.009234 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:17.009193 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a4b8303b-574c-432d-9bb4-f481df830ab8-lib-modules\") pod \"perf-node-gather-daemonset-npkmb\" (UID: \"a4b8303b-574c-432d-9bb4-f481df830ab8\") " pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-npkmb" May 28 15:43:17.009398 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:17.009265 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a4b8303b-574c-432d-9bb4-f481df830ab8-sys\") pod \"perf-node-gather-daemonset-npkmb\" (UID: \"a4b8303b-574c-432d-9bb4-f481df830ab8\") " pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-npkmb" May 28 15:43:17.009398 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:17.009367 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a4b8303b-574c-432d-9bb4-f481df830ab8-podres\") pod \"perf-node-gather-daemonset-npkmb\" (UID: \"a4b8303b-574c-432d-9bb4-f481df830ab8\") " pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-npkmb" May 28 15:43:17.009475 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:17.009424 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z85fm\" (UniqueName: \"kubernetes.io/projected/a4b8303b-574c-432d-9bb4-f481df830ab8-kube-api-access-z85fm\") pod \"perf-node-gather-daemonset-npkmb\" (UID: \"a4b8303b-574c-432d-9bb4-f481df830ab8\") " pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-npkmb" May 28 15:43:17.009537 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:17.009474 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a4b8303b-574c-432d-9bb4-f481df830ab8-proc\") pod \"perf-node-gather-daemonset-npkmb\" (UID: \"a4b8303b-574c-432d-9bb4-f481df830ab8\") " pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-npkmb" May 28 15:43:17.110339 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:17.110293 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a4b8303b-574c-432d-9bb4-f481df830ab8-sys\") pod \"perf-node-gather-daemonset-npkmb\" (UID: \"a4b8303b-574c-432d-9bb4-f481df830ab8\") " pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-npkmb" May 28 15:43:17.110553 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:17.110395 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a4b8303b-574c-432d-9bb4-f481df830ab8-podres\") pod \"perf-node-gather-daemonset-npkmb\" (UID: \"a4b8303b-574c-432d-9bb4-f481df830ab8\") " pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-npkmb" May 28 15:43:17.110553 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:17.110438 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z85fm\" (UniqueName: \"kubernetes.io/projected/a4b8303b-574c-432d-9bb4-f481df830ab8-kube-api-access-z85fm\") pod \"perf-node-gather-daemonset-npkmb\" (UID: \"a4b8303b-574c-432d-9bb4-f481df830ab8\") " pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-npkmb" May 28 15:43:17.110553 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:17.110498 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a4b8303b-574c-432d-9bb4-f481df830ab8-proc\") pod \"perf-node-gather-daemonset-npkmb\" (UID: \"a4b8303b-574c-432d-9bb4-f481df830ab8\") " pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-npkmb" May 28 15:43:17.110553 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:17.110547 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a4b8303b-574c-432d-9bb4-f481df830ab8-lib-modules\") pod \"perf-node-gather-daemonset-npkmb\" (UID: \"a4b8303b-574c-432d-9bb4-f481df830ab8\") " pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-npkmb" May 28 15:43:17.110793 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:17.110737 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a4b8303b-574c-432d-9bb4-f481df830ab8-lib-modules\") pod \"perf-node-gather-daemonset-npkmb\" (UID: \"a4b8303b-574c-432d-9bb4-f481df830ab8\") " pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-npkmb" May 28 15:43:17.110857 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:17.110798 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a4b8303b-574c-432d-9bb4-f481df830ab8-sys\") pod \"perf-node-gather-daemonset-npkmb\" (UID: \"a4b8303b-574c-432d-9bb4-f481df830ab8\") " pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-npkmb" May 28 15:43:17.110914 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:17.110866 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a4b8303b-574c-432d-9bb4-f481df830ab8-podres\") pod \"perf-node-gather-daemonset-npkmb\" (UID: \"a4b8303b-574c-432d-9bb4-f481df830ab8\") " pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-npkmb" May 28 15:43:17.111217 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:17.111195 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a4b8303b-574c-432d-9bb4-f481df830ab8-proc\") pod \"perf-node-gather-daemonset-npkmb\" (UID: \"a4b8303b-574c-432d-9bb4-f481df830ab8\") " pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-npkmb" May 28 15:43:17.127294 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:17.127265 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z85fm\" (UniqueName: \"kubernetes.io/projected/a4b8303b-574c-432d-9bb4-f481df830ab8-kube-api-access-z85fm\") pod \"perf-node-gather-daemonset-npkmb\" (UID: \"a4b8303b-574c-432d-9bb4-f481df830ab8\") " pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-npkmb" May 28 15:43:17.231622 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:17.231542 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-npkmb" May 28 15:43:17.389458 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:17.386793 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xkl9v/perf-node-gather-daemonset-npkmb"] May 28 15:43:17.391787 ip-10-0-140-141 kubenswrapper[2559]: W0528 15:43:17.391753 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda4b8303b_574c_432d_9bb4_f481df830ab8.slice/crio-e4c925396a859aabb368f11f8873efff4e6591c33ae1e4aaa0c40f8f3e11aa63 WatchSource:0}: Error finding container e4c925396a859aabb368f11f8873efff4e6591c33ae1e4aaa0c40f8f3e11aa63: Status 404 returned error can't find the container with id e4c925396a859aabb368f11f8873efff4e6591c33ae1e4aaa0c40f8f3e11aa63 May 28 15:43:17.523042 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:17.523017 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-84855f4f96-qp4rb_bcb3263b-a6f4-46a2-9da8-2d7b09ed29c3/console/0.log" May 28 15:43:17.557113 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:17.557072 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-npkmb" event={"ID":"a4b8303b-574c-432d-9bb4-f481df830ab8","Type":"ContainerStarted","Data":"f26881f429992f27efa4373437d8bbfabb343fb669a0592416a241f786bd4629"} May 28 15:43:17.557113 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:17.557116 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-npkmb" event={"ID":"a4b8303b-574c-432d-9bb4-f481df830ab8","Type":"ContainerStarted","Data":"e4c925396a859aabb368f11f8873efff4e6591c33ae1e4aaa0c40f8f3e11aa63"} May 28 15:43:17.557352 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:17.557235 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-npkmb" May 28 15:43:17.559138 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:17.559114 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-57bd96b888-76mzv_0951dff9-48da-4274-8d7e-b47651895056/download-server/0.log" May 28 15:43:17.574032 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:17.573976 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-npkmb" podStartSLOduration=1.573959579 podStartE2EDuration="1.573959579s" podCreationTimestamp="2026-05-28 15:43:16 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 15:43:17.570725018 +0000 UTC m=+2148.459712125" watchObservedRunningTime="2026-05-28 15:43:17.573959579 +0000 UTC m=+2148.462946663" May 28 15:43:18.817436 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:18.817405 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nvshr_59c5be81-b339-4a6d-9c00-f66c662406a3/dns/0.log" May 28 15:43:18.835940 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:18.835917 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nvshr_59c5be81-b339-4a6d-9c00-f66c662406a3/kube-rbac-proxy/0.log" May 28 15:43:18.895234 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:18.895214 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kr9wd_297e38b8-9a81-4112-811d-a97cfa5341ff/dns-node-resolver/0.log" May 28 15:43:19.397603 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:19.397573 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bpxxm_a2246888-2a0d-48b8-8fe6-12e283e47eb0/node-ca/0.log" May 28 15:43:20.186472 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:20.186430 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-686dffbf4b45p4l_46389ad5-3b8d-4802-8fee-d069c467ab50/istio-proxy/0.log" May 28 15:43:20.407736 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:20.407704 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d6f447cf5-79zsb_c18ad486-95ca-4632-a07b-461e1c7ee15d/istio-proxy/0.log" May 28 15:43:20.482846 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:20.482765 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-69c468644b-ff99j_3889dc6c-fe09-49a2-b52a-cb4de8f769de/router/0.log" May 28 15:43:20.957282 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:20.957256 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9h2xl_8fa7b229-d678-41cd-b578-a31774414667/serve-healthcheck-canary/0.log" May 28 15:43:21.536195 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:21.536164 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-prjnr_e20212be-1864-4edd-8ecb-90312a889a89/kube-rbac-proxy/0.log" May 28 15:43:21.554403 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:21.554382 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-prjnr_e20212be-1864-4edd-8ecb-90312a889a89/exporter/0.log" May 28 15:43:21.573678 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:21.573645 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-prjnr_e20212be-1864-4edd-8ecb-90312a889a89/extractor/0.log" May 28 15:43:23.370212 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:23.370185 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29666370-6mjk6_f5125335-b0b9-4082-a11f-7cd274d42d0b/cleanup/0.log" May 28 15:43:23.536020 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:23.535988 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57c44ff6d8-b2zj5_0dbad4af-3d6a-4836-bee5-0c9b0e0a2013/manager/0.log" May 28 15:43:23.571226 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:23.571201 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-npkmb" May 28 15:43:24.630098 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:24.630072 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5657b98b9f-gxd6s_33db757d-c1e2-4459-b3e9-c7847fbea04b/manager/0.log" May 28 15:43:24.674525 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:24.674476 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-bfc7f696d-b8g2p_4ef17cd1-d142-4170-9a5f-9509ea6c20dc/openshift-lws-operator/0.log" May 28 15:43:29.167643 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:29.167606 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-565644f49-7zwzb_d75362ae-8357-492a-a275-c3ace0009a19/kube-storage-version-migrator-operator/1.log" May 28 15:43:29.169432 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:29.169408 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-565644f49-7zwzb_d75362ae-8357-492a-a275-c3ace0009a19/kube-storage-version-migrator-operator/0.log" May 28 15:43:30.227503 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:30.227459 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jc4zf_615d5292-c021-4a20-be77-56f9230db424/kube-multus-additional-cni-plugins/0.log" May 28 15:43:30.245592 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:30.245570 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jc4zf_615d5292-c021-4a20-be77-56f9230db424/egress-router-binary-copy/0.log" May 28 15:43:30.263891 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:30.263867 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jc4zf_615d5292-c021-4a20-be77-56f9230db424/cni-plugins/0.log" May 28 15:43:30.283152 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:30.283133 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jc4zf_615d5292-c021-4a20-be77-56f9230db424/bond-cni-plugin/0.log" May 28 15:43:30.302931 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:30.302911 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jc4zf_615d5292-c021-4a20-be77-56f9230db424/routeoverride-cni/0.log" May 28 15:43:30.321442 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:30.321421 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jc4zf_615d5292-c021-4a20-be77-56f9230db424/whereabouts-cni-bincopy/0.log" May 28 15:43:30.339584 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:30.339564 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jc4zf_615d5292-c021-4a20-be77-56f9230db424/whereabouts-cni/0.log" May 28 15:43:30.523564 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:30.523547 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-l7mvw_10ca4edd-091a-4f34-912a-0a579c845f52/kube-multus/0.log" May 28 15:43:30.623775 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:30.623751 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-l9p7b_e3a78917-de69-42d0-8fbe-85f346276bf9/network-metrics-daemon/0.log" May 28 15:43:30.640221 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:30.640200 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-l9p7b_e3a78917-de69-42d0-8fbe-85f346276bf9/kube-rbac-proxy/0.log" May 28 15:43:31.696615 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:31.696586 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xcg2z_0f511937-01b1-4615-bc4c-12832012e6ce/ovn-controller/0.log" May 28 15:43:31.729554 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:31.729515 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xcg2z_0f511937-01b1-4615-bc4c-12832012e6ce/ovn-acl-logging/0.log" May 28 15:43:31.749280 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:31.749259 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xcg2z_0f511937-01b1-4615-bc4c-12832012e6ce/kube-rbac-proxy-node/0.log" May 28 15:43:31.770352 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:31.770327 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xcg2z_0f511937-01b1-4615-bc4c-12832012e6ce/kube-rbac-proxy-ovn-metrics/0.log" May 28 15:43:31.785935 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:31.785910 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xcg2z_0f511937-01b1-4615-bc4c-12832012e6ce/northd/0.log" May 28 15:43:31.805662 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:31.805621 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xcg2z_0f511937-01b1-4615-bc4c-12832012e6ce/nbdb/0.log" May 28 15:43:31.824668 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:31.824642 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xcg2z_0f511937-01b1-4615-bc4c-12832012e6ce/sbdb/0.log" May 28 15:43:31.998642 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:31.998612 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xcg2z_0f511937-01b1-4615-bc4c-12832012e6ce/ovnkube-controller/0.log" May 28 15:43:33.268257 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:33.268219 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-7f44b6dd87-rjwvn_82f2a321-d05a-4b4d-9666-8539c5b0b7a4/check-endpoints/0.log" May 28 15:43:33.331711 ip-10-0-140-141 kubenswrapper[2559]: I0528 15:43:33.331684 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-rpkl2_8306acd6-5d60-42dd-a8e4-d21a859b6525/network-check-target-container/0.log"