May 29 16:54:23.826714 ip-10-0-141-197 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory May 29 16:54:23.826729 ip-10-0-141-197 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory May 29 16:54:23.826739 ip-10-0-141-197 systemd[1]: kubelet.service: Failed with result 'resources'. May 29 16:54:23.827061 ip-10-0-141-197 systemd[1]: Failed to start Kubernetes Kubelet. May 29 16:54:33.210536 ip-10-0-141-197 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. May 29 16:54:33.210553 ip-10-0-141-197 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot fbcfcb3ea6ec4f92a9e8f20510509615 -- May 29 16:56:49.376132 ip-10-0-141-197 systemd[1]: Starting Kubernetes Kubelet... May 29 16:56:49.842645 ip-10-0-141-197 kubenswrapper[2549]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 29 16:56:49.842645 ip-10-0-141-197 kubenswrapper[2549]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. May 29 16:56:49.842645 ip-10-0-141-197 kubenswrapper[2549]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 29 16:56:49.842645 ip-10-0-141-197 kubenswrapper[2549]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. May 29 16:56:49.842645 ip-10-0-141-197 kubenswrapper[2549]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 29 16:56:49.844622 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.844484 2549 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" May 29 16:56:49.849029 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849014 2549 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 29 16:56:49.849029 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849028 2549 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 29 16:56:49.849108 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849032 2549 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 29 16:56:49.849108 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849036 2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 29 16:56:49.849108 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849039 2549 feature_gate.go:328] unrecognized feature gate: Example2 May 29 16:56:49.849108 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849043 2549 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 29 16:56:49.849108 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849046 2549 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 29 16:56:49.849108 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849049 2549 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 29 16:56:49.849108 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849051 2549 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 29 16:56:49.849108 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849054 2549 feature_gate.go:328] unrecognized feature gate: SignatureStores May 29 16:56:49.849108 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849057 2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 29 16:56:49.849108 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849060 2549 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 29 16:56:49.849108 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849063 2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 29 16:56:49.849108 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849065 2549 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 29 16:56:49.849108 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849068 2549 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 29 16:56:49.849108 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849070 2549 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 29 16:56:49.849108 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849073 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 29 16:56:49.849108 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849076 2549 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 29 16:56:49.849108 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849078 2549 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 29 16:56:49.849108 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849081 2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 29 16:56:49.849108 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849083 2549 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 29 16:56:49.849108 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849086 2549 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 29 16:56:49.849580 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849093 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 29 16:56:49.849580 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849096 2549 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 29 16:56:49.849580 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849098 2549 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 29 16:56:49.849580 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849100 2549 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 29 16:56:49.849580 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849103 2549 feature_gate.go:328] unrecognized feature gate: DualReplica May 29 16:56:49.849580 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849106 2549 feature_gate.go:328] unrecognized feature gate: OVNObservability May 29 16:56:49.849580 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849109 2549 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 29 16:56:49.849580 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849111 2549 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 29 16:56:49.849580 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849114 2549 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 29 16:56:49.849580 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849117 2549 feature_gate.go:328] unrecognized feature gate: PinnedImages May 29 16:56:49.849580 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849119 2549 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 29 16:56:49.849580 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849122 2549 feature_gate.go:328] unrecognized feature gate: Example May 29 16:56:49.849580 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849125 2549 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 29 16:56:49.849580 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849127 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 29 16:56:49.849580 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849130 2549 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 29 16:56:49.849580 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849134 2549 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 29 16:56:49.849580 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849136 2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 29 16:56:49.849580 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849139 2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 29 16:56:49.849580 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849141 2549 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 29 16:56:49.849580 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849144 2549 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 29 16:56:49.850123 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849146 2549 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 29 16:56:49.850123 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849149 2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 29 16:56:49.850123 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849151 2549 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 29 16:56:49.850123 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849153 2549 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 29 16:56:49.850123 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849156 2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 29 16:56:49.850123 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849159 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 29 16:56:49.850123 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849161 2549 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 29 16:56:49.850123 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849164 2549 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 29 16:56:49.850123 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849166 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 29 16:56:49.850123 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849169 2549 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 29 16:56:49.850123 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849172 2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 29 16:56:49.850123 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849174 2549 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 29 16:56:49.850123 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849179 2549 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 29 16:56:49.850123 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849184 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 29 16:56:49.850123 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849187 2549 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 29 16:56:49.850123 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849190 2549 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 29 16:56:49.850123 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849193 2549 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 29 16:56:49.850123 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849196 2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 29 16:56:49.850123 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849199 2549 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 29 16:56:49.850582 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849201 2549 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 29 16:56:49.850582 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849204 2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 29 16:56:49.850582 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849207 2549 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 29 16:56:49.850582 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849210 2549 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 29 16:56:49.850582 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849212 2549 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 29 16:56:49.850582 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849215 2549 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 29 16:56:49.850582 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849217 2549 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 29 16:56:49.850582 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849222 2549 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 29 16:56:49.850582 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849226 2549 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 29 16:56:49.850582 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849228 2549 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 29 16:56:49.850582 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849231 2549 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 29 16:56:49.850582 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849233 2549 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 29 16:56:49.850582 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849236 2549 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 29 16:56:49.850582 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849238 2549 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 29 16:56:49.850582 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849242 2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 29 16:56:49.850582 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849244 2549 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 29 16:56:49.850582 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849247 2549 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 29 16:56:49.850582 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849249 2549 feature_gate.go:328] unrecognized feature gate: NewOLM May 29 16:56:49.850582 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849252 2549 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 29 16:56:49.851058 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849254 2549 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 29 16:56:49.851058 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849257 2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 29 16:56:49.851058 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849260 2549 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 29 16:56:49.851058 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849262 2549 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 29 16:56:49.851058 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849265 2549 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 29 16:56:49.851058 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849267 2549 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 29 16:56:49.851058 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849632 2549 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 29 16:56:49.851058 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849637 2549 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 29 16:56:49.851058 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849640 2549 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 29 16:56:49.851058 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849643 2549 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 29 16:56:49.851058 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849646 2549 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 29 16:56:49.851058 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849649 2549 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 29 16:56:49.851058 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849652 2549 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 29 16:56:49.851058 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849654 2549 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 29 16:56:49.851058 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849657 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 29 16:56:49.851058 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849659 2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 29 16:56:49.851058 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849662 2549 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 29 16:56:49.851058 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849665 2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 29 16:56:49.851058 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849667 2549 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 29 16:56:49.851058 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849670 2549 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 29 16:56:49.851551 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849672 2549 feature_gate.go:328] unrecognized feature gate: Example2 May 29 16:56:49.851551 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849675 2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 29 16:56:49.851551 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849677 2549 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 29 16:56:49.851551 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849680 2549 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 29 16:56:49.851551 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849682 2549 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 29 16:56:49.851551 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849685 2549 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 29 16:56:49.851551 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849687 2549 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 29 16:56:49.851551 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849690 2549 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 29 16:56:49.851551 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849692 2549 feature_gate.go:328] unrecognized feature gate: DualReplica May 29 16:56:49.851551 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849695 2549 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 29 16:56:49.851551 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849697 2549 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 29 16:56:49.851551 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849700 2549 feature_gate.go:328] unrecognized feature gate: OVNObservability May 29 16:56:49.851551 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849702 2549 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 29 16:56:49.851551 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849705 2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 29 16:56:49.851551 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849709 2549 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 29 16:56:49.851551 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849713 2549 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 29 16:56:49.851551 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849716 2549 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 29 16:56:49.851551 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849718 2549 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 29 16:56:49.851551 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849721 2549 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 29 16:56:49.851551 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849724 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 29 16:56:49.852055 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849726 2549 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 29 16:56:49.852055 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849729 2549 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 29 16:56:49.852055 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849731 2549 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 29 16:56:49.852055 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849734 2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 29 16:56:49.852055 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849737 2549 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 29 16:56:49.852055 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849739 2549 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 29 16:56:49.852055 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849742 2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 29 16:56:49.852055 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849744 2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 29 16:56:49.852055 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849746 2549 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 29 16:56:49.852055 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849749 2549 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 29 16:56:49.852055 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849751 2549 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 29 16:56:49.852055 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849754 2549 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 29 16:56:49.852055 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849756 2549 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 29 16:56:49.852055 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849759 2549 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 29 16:56:49.852055 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849761 2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 29 16:56:49.852055 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849764 2549 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 29 16:56:49.852055 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849769 2549 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 29 16:56:49.852055 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849772 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 29 16:56:49.852055 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849775 2549 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 29 16:56:49.852523 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849778 2549 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 29 16:56:49.852523 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849781 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 29 16:56:49.852523 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849784 2549 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 29 16:56:49.852523 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849787 2549 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 29 16:56:49.852523 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849789 2549 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 29 16:56:49.852523 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849792 2549 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 29 16:56:49.852523 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849795 2549 feature_gate.go:328] unrecognized feature gate: PinnedImages May 29 16:56:49.852523 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849798 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 29 16:56:49.852523 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849800 2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 29 16:56:49.852523 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849803 2549 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 29 16:56:49.852523 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849805 2549 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 29 16:56:49.852523 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849808 2549 feature_gate.go:328] unrecognized feature gate: Example May 29 16:56:49.852523 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849811 2549 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 29 16:56:49.852523 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849814 2549 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 29 16:56:49.852523 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849816 2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 29 16:56:49.852523 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849819 2549 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 29 16:56:49.852523 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849821 2549 feature_gate.go:328] unrecognized feature gate: NewOLM May 29 16:56:49.852523 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849824 2549 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 29 16:56:49.852523 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849826 2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 29 16:56:49.852523 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849829 2549 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 29 16:56:49.853025 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849831 2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 29 16:56:49.853025 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849834 2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 29 16:56:49.853025 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849836 2549 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 29 16:56:49.853025 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849853 2549 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 29 16:56:49.853025 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849856 2549 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 29 16:56:49.853025 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849859 2549 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 29 16:56:49.853025 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849861 2549 feature_gate.go:328] unrecognized feature gate: SignatureStores May 29 16:56:49.853025 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849864 2549 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 29 16:56:49.853025 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849867 2549 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 29 16:56:49.853025 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849872 2549 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 29 16:56:49.853025 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849875 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 29 16:56:49.853025 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849877 2549 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 29 16:56:49.853025 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.849880 2549 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 29 16:56:49.853025 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.849949 2549 flags.go:64] FLAG: --address="0.0.0.0" May 29 16:56:49.853025 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.849956 2549 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" May 29 16:56:49.853025 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.849965 2549 flags.go:64] FLAG: --anonymous-auth="true" May 29 16:56:49.853025 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.849969 2549 flags.go:64] FLAG: --application-metrics-count-limit="100" May 29 16:56:49.853025 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.849974 2549 flags.go:64] FLAG: --authentication-token-webhook="false" May 29 16:56:49.853025 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.849977 2549 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" May 29 16:56:49.853025 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.849981 2549 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" May 29 16:56:49.853025 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.849986 2549 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" May 29 16:56:49.853547 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.849989 2549 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" May 29 16:56:49.853547 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.849992 2549 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" May 29 16:56:49.853547 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.849995 2549 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" May 29 16:56:49.853547 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.849999 2549 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" May 29 16:56:49.853547 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850002 2549 flags.go:64] FLAG: --cgroup-driver="cgroupfs" May 29 16:56:49.853547 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850005 2549 flags.go:64] FLAG: --cgroup-root="" May 29 16:56:49.853547 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850007 2549 flags.go:64] FLAG: --cgroups-per-qos="true" May 29 16:56:49.853547 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850010 2549 flags.go:64] FLAG: --client-ca-file="" May 29 16:56:49.853547 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850013 2549 flags.go:64] FLAG: --cloud-config="" May 29 16:56:49.853547 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850016 2549 flags.go:64] FLAG: --cloud-provider="external" May 29 16:56:49.853547 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850019 2549 flags.go:64] FLAG: --cluster-dns="[]" May 29 16:56:49.853547 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850023 2549 flags.go:64] FLAG: --cluster-domain="" May 29 16:56:49.853547 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850026 2549 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" May 29 16:56:49.853547 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850029 2549 flags.go:64] FLAG: --config-dir="" May 29 16:56:49.853547 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850032 2549 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" May 29 16:56:49.853547 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850035 2549 flags.go:64] FLAG: --container-log-max-files="5" May 29 16:56:49.853547 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850039 2549 flags.go:64] FLAG: --container-log-max-size="10Mi" May 29 16:56:49.853547 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850041 2549 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" May 29 16:56:49.853547 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850048 2549 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" May 29 16:56:49.853547 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850051 2549 flags.go:64] FLAG: --containerd-namespace="k8s.io" May 29 16:56:49.853547 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850055 2549 flags.go:64] FLAG: --contention-profiling="false" May 29 16:56:49.853547 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850058 2549 flags.go:64] FLAG: --cpu-cfs-quota="true" May 29 16:56:49.853547 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850061 2549 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" May 29 16:56:49.853547 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850064 2549 flags.go:64] FLAG: --cpu-manager-policy="none" May 29 16:56:49.853547 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850067 2549 flags.go:64] FLAG: --cpu-manager-policy-options="" May 29 16:56:49.854304 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850071 2549 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" May 29 16:56:49.854304 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850075 2549 flags.go:64] FLAG: --enable-controller-attach-detach="true" May 29 16:56:49.854304 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850078 2549 flags.go:64] FLAG: --enable-debugging-handlers="true" May 29 16:56:49.854304 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850080 2549 flags.go:64] FLAG: --enable-load-reader="false" May 29 16:56:49.854304 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850084 2549 flags.go:64] FLAG: --enable-server="true" May 29 16:56:49.854304 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850086 2549 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" May 29 16:56:49.854304 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850091 2549 flags.go:64] FLAG: --event-burst="100" May 29 16:56:49.854304 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850094 2549 flags.go:64] FLAG: --event-qps="50" May 29 16:56:49.854304 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850096 2549 flags.go:64] FLAG: --event-storage-age-limit="default=0" May 29 16:56:49.854304 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850100 2549 flags.go:64] FLAG: --event-storage-event-limit="default=0" May 29 16:56:49.854304 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850102 2549 flags.go:64] FLAG: --eviction-hard="" May 29 16:56:49.854304 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850106 2549 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" May 29 16:56:49.854304 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850109 2549 flags.go:64] FLAG: --eviction-minimum-reclaim="" May 29 16:56:49.854304 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850112 2549 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" May 29 16:56:49.854304 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850114 2549 flags.go:64] FLAG: --eviction-soft="" May 29 16:56:49.854304 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850117 2549 flags.go:64] FLAG: --eviction-soft-grace-period="" May 29 16:56:49.854304 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850120 2549 flags.go:64] FLAG: --exit-on-lock-contention="false" May 29 16:56:49.854304 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850123 2549 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" May 29 16:56:49.854304 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850126 2549 flags.go:64] FLAG: --experimental-mounter-path="" May 29 16:56:49.854304 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850129 2549 flags.go:64] FLAG: --fail-cgroupv1="false" May 29 16:56:49.854304 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850132 2549 flags.go:64] FLAG: --fail-swap-on="true" May 29 16:56:49.854304 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850135 2549 flags.go:64] FLAG: --feature-gates="" May 29 16:56:49.854304 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850139 2549 flags.go:64] FLAG: --file-check-frequency="20s" May 29 16:56:49.854304 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850142 2549 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" May 29 16:56:49.854304 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850145 2549 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" May 29 16:56:49.854936 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850150 2549 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" May 29 16:56:49.854936 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850152 2549 flags.go:64] FLAG: --healthz-port="10248" May 29 16:56:49.854936 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850156 2549 flags.go:64] FLAG: --help="false" May 29 16:56:49.854936 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850159 2549 flags.go:64] FLAG: --hostname-override="ip-10-0-141-197.ec2.internal" May 29 16:56:49.854936 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850162 2549 flags.go:64] FLAG: --housekeeping-interval="10s" May 29 16:56:49.854936 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850165 2549 flags.go:64] FLAG: --http-check-frequency="20s" May 29 16:56:49.854936 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850167 2549 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" May 29 16:56:49.854936 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850171 2549 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" May 29 16:56:49.854936 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850174 2549 flags.go:64] FLAG: --image-gc-high-threshold="85" May 29 16:56:49.854936 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850177 2549 flags.go:64] FLAG: --image-gc-low-threshold="80" May 29 16:56:49.854936 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850180 2549 flags.go:64] FLAG: --image-service-endpoint="" May 29 16:56:49.854936 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850183 2549 flags.go:64] FLAG: --kernel-memcg-notification="false" May 29 16:56:49.854936 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850186 2549 flags.go:64] FLAG: --kube-api-burst="100" May 29 16:56:49.854936 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850189 2549 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" May 29 16:56:49.854936 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850192 2549 flags.go:64] FLAG: --kube-api-qps="50" May 29 16:56:49.854936 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850195 2549 flags.go:64] FLAG: --kube-reserved="" May 29 16:56:49.854936 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850197 2549 flags.go:64] FLAG: --kube-reserved-cgroup="" May 29 16:56:49.854936 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850200 2549 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" May 29 16:56:49.854936 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850203 2549 flags.go:64] FLAG: --kubelet-cgroups="" May 29 16:56:49.854936 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850206 2549 flags.go:64] FLAG: --local-storage-capacity-isolation="true" May 29 16:56:49.854936 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850209 2549 flags.go:64] FLAG: --lock-file="" May 29 16:56:49.854936 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850212 2549 flags.go:64] FLAG: --log-cadvisor-usage="false" May 29 16:56:49.854936 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850215 2549 flags.go:64] FLAG: --log-flush-frequency="5s" May 29 16:56:49.854936 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850218 2549 flags.go:64] FLAG: --log-json-info-buffer-size="0" May 29 16:56:49.855551 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850223 2549 flags.go:64] FLAG: --log-json-split-stream="false" May 29 16:56:49.855551 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850225 2549 flags.go:64] FLAG: --log-text-info-buffer-size="0" May 29 16:56:49.855551 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850228 2549 flags.go:64] FLAG: --log-text-split-stream="false" May 29 16:56:49.855551 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850231 2549 flags.go:64] FLAG: --logging-format="text" May 29 16:56:49.855551 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850234 2549 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" May 29 16:56:49.855551 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850237 2549 flags.go:64] FLAG: --make-iptables-util-chains="true" May 29 16:56:49.855551 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850239 2549 flags.go:64] FLAG: --manifest-url="" May 29 16:56:49.855551 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850242 2549 flags.go:64] FLAG: --manifest-url-header="" May 29 16:56:49.855551 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850247 2549 flags.go:64] FLAG: --max-housekeeping-interval="15s" May 29 16:56:49.855551 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850250 2549 flags.go:64] FLAG: --max-open-files="1000000" May 29 16:56:49.855551 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850255 2549 flags.go:64] FLAG: --max-pods="110" May 29 16:56:49.855551 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850258 2549 flags.go:64] FLAG: --maximum-dead-containers="-1" May 29 16:56:49.855551 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850261 2549 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" May 29 16:56:49.855551 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850263 2549 flags.go:64] FLAG: --memory-manager-policy="None" May 29 16:56:49.855551 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850266 2549 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" May 29 16:56:49.855551 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850269 2549 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" May 29 16:56:49.855551 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850272 2549 flags.go:64] FLAG: --node-ip="0.0.0.0" May 29 16:56:49.855551 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850275 2549 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" May 29 16:56:49.855551 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850283 2549 flags.go:64] FLAG: --node-status-max-images="50" May 29 16:56:49.855551 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850286 2549 flags.go:64] FLAG: --node-status-update-frequency="10s" May 29 16:56:49.855551 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850288 2549 flags.go:64] FLAG: --oom-score-adj="-999" May 29 16:56:49.855551 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850292 2549 flags.go:64] FLAG: --pod-cidr="" May 29 16:56:49.855551 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850295 2549 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f1083dc6abd9805f79797fa11c9accacc4a38caa073c94cb99097faffce05983" May 29 16:56:49.856179 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850300 2549 flags.go:64] FLAG: --pod-manifest-path="" May 29 16:56:49.856179 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850303 2549 flags.go:64] FLAG: --pod-max-pids="-1" May 29 16:56:49.856179 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850306 2549 flags.go:64] FLAG: --pods-per-core="0" May 29 16:56:49.856179 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850309 2549 flags.go:64] FLAG: --port="10250" May 29 16:56:49.856179 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850312 2549 flags.go:64] FLAG: --protect-kernel-defaults="false" May 29 16:56:49.856179 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850315 2549 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0991e3e532f9814f3" May 29 16:56:49.856179 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850318 2549 flags.go:64] FLAG: --qos-reserved="" May 29 16:56:49.856179 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850321 2549 flags.go:64] FLAG: --read-only-port="10255" May 29 16:56:49.856179 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850324 2549 flags.go:64] FLAG: --register-node="true" May 29 16:56:49.856179 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850327 2549 flags.go:64] FLAG: --register-schedulable="true" May 29 16:56:49.856179 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850330 2549 flags.go:64] FLAG: --register-with-taints="" May 29 16:56:49.856179 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850333 2549 flags.go:64] FLAG: --registry-burst="10" May 29 16:56:49.856179 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850336 2549 flags.go:64] FLAG: --registry-qps="5" May 29 16:56:49.856179 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850339 2549 flags.go:64] FLAG: --reserved-cpus="" May 29 16:56:49.856179 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850341 2549 flags.go:64] FLAG: --reserved-memory="" May 29 16:56:49.856179 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850345 2549 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" May 29 16:56:49.856179 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850348 2549 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" May 29 16:56:49.856179 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850351 2549 flags.go:64] FLAG: --rotate-certificates="false" May 29 16:56:49.856179 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850356 2549 flags.go:64] FLAG: --rotate-server-certificates="false" May 29 16:56:49.856179 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850359 2549 flags.go:64] FLAG: --runonce="false" May 29 16:56:49.856179 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850362 2549 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" May 29 16:56:49.856179 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850365 2549 flags.go:64] FLAG: --runtime-request-timeout="2m0s" May 29 16:56:49.856179 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850368 2549 flags.go:64] FLAG: --seccomp-default="false" May 29 16:56:49.856179 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850370 2549 flags.go:64] FLAG: --serialize-image-pulls="true" May 29 16:56:49.856179 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850373 2549 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" May 29 16:56:49.856179 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850376 2549 flags.go:64] FLAG: --storage-driver-db="cadvisor" May 29 16:56:49.856826 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850379 2549 flags.go:64] FLAG: --storage-driver-host="localhost:8086" May 29 16:56:49.856826 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850382 2549 flags.go:64] FLAG: --storage-driver-password="root" May 29 16:56:49.856826 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850385 2549 flags.go:64] FLAG: --storage-driver-secure="false" May 29 16:56:49.856826 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850388 2549 flags.go:64] FLAG: --storage-driver-table="stats" May 29 16:56:49.856826 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850391 2549 flags.go:64] FLAG: --storage-driver-user="root" May 29 16:56:49.856826 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850394 2549 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" May 29 16:56:49.856826 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850397 2549 flags.go:64] FLAG: --sync-frequency="1m0s" May 29 16:56:49.856826 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850399 2549 flags.go:64] FLAG: --system-cgroups="" May 29 16:56:49.856826 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850402 2549 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" May 29 16:56:49.856826 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850408 2549 flags.go:64] FLAG: --system-reserved-cgroup="" May 29 16:56:49.856826 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850411 2549 flags.go:64] FLAG: --tls-cert-file="" May 29 16:56:49.856826 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850413 2549 flags.go:64] FLAG: --tls-cipher-suites="[]" May 29 16:56:49.856826 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850417 2549 flags.go:64] FLAG: --tls-min-version="" May 29 16:56:49.856826 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850420 2549 flags.go:64] FLAG: --tls-private-key-file="" May 29 16:56:49.856826 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850422 2549 flags.go:64] FLAG: --topology-manager-policy="none" May 29 16:56:49.856826 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850425 2549 flags.go:64] FLAG: --topology-manager-policy-options="" May 29 16:56:49.856826 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850428 2549 flags.go:64] FLAG: --topology-manager-scope="container" May 29 16:56:49.856826 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850431 2549 flags.go:64] FLAG: --v="2" May 29 16:56:49.856826 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850435 2549 flags.go:64] FLAG: --version="false" May 29 16:56:49.856826 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850438 2549 flags.go:64] FLAG: --vmodule="" May 29 16:56:49.856826 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850442 2549 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" May 29 16:56:49.856826 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.850446 2549 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" May 29 16:56:49.856826 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850529 2549 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 29 16:56:49.856826 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850532 2549 feature_gate.go:328] unrecognized feature gate: PinnedImages May 29 16:56:49.857457 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850536 2549 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 29 16:56:49.857457 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850539 2549 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 29 16:56:49.857457 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850542 2549 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 29 16:56:49.857457 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850545 2549 feature_gate.go:328] unrecognized feature gate: NewOLM May 29 16:56:49.857457 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850547 2549 feature_gate.go:328] unrecognized feature gate: SignatureStores May 29 16:56:49.857457 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850550 2549 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 29 16:56:49.857457 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850552 2549 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 29 16:56:49.857457 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850555 2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 29 16:56:49.857457 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850557 2549 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 29 16:56:49.857457 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850560 2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 29 16:56:49.857457 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850562 2549 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 29 16:56:49.857457 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850565 2549 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 29 16:56:49.857457 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850570 2549 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 29 16:56:49.857457 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850573 2549 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 29 16:56:49.857457 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850576 2549 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 29 16:56:49.857457 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850579 2549 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 29 16:56:49.857457 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850581 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 29 16:56:49.857457 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850584 2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 29 16:56:49.857457 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850587 2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 29 16:56:49.857948 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850590 2549 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 29 16:56:49.857948 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850593 2549 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 29 16:56:49.857948 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850597 2549 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 29 16:56:49.857948 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850600 2549 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 29 16:56:49.857948 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850602 2549 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 29 16:56:49.857948 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850605 2549 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 29 16:56:49.857948 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850607 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 29 16:56:49.857948 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850610 2549 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 29 16:56:49.857948 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850612 2549 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 29 16:56:49.857948 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850615 2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 29 16:56:49.857948 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850617 2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 29 16:56:49.857948 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850620 2549 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 29 16:56:49.857948 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850622 2549 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 29 16:56:49.857948 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850626 2549 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 29 16:56:49.857948 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850630 2549 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 29 16:56:49.857948 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850632 2549 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 29 16:56:49.857948 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850635 2549 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 29 16:56:49.857948 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850638 2549 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 29 16:56:49.857948 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850640 2549 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 29 16:56:49.857948 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850643 2549 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 29 16:56:49.858455 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850645 2549 feature_gate.go:328] unrecognized feature gate: DualReplica May 29 16:56:49.858455 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850648 2549 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 29 16:56:49.858455 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850650 2549 feature_gate.go:328] unrecognized feature gate: Example2 May 29 16:56:49.858455 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850652 2549 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 29 16:56:49.858455 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850655 2549 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 29 16:56:49.858455 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850658 2549 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 29 16:56:49.858455 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850660 2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 29 16:56:49.858455 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850663 2549 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 29 16:56:49.858455 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850665 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 29 16:56:49.858455 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850668 2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 29 16:56:49.858455 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850671 2549 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 29 16:56:49.858455 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850673 2549 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 29 16:56:49.858455 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850676 2549 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 29 16:56:49.858455 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850678 2549 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 29 16:56:49.858455 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850681 2549 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 29 16:56:49.858455 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850684 2549 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 29 16:56:49.858455 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850686 2549 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 29 16:56:49.858455 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850689 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 29 16:56:49.858455 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850691 2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 29 16:56:49.858455 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850693 2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 29 16:56:49.858950 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850696 2549 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 29 16:56:49.858950 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850698 2549 feature_gate.go:328] unrecognized feature gate: OVNObservability May 29 16:56:49.858950 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850701 2549 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 29 16:56:49.858950 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850703 2549 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 29 16:56:49.858950 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850705 2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 29 16:56:49.858950 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850710 2549 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 29 16:56:49.858950 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850714 2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 29 16:56:49.858950 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850716 2549 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 29 16:56:49.858950 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850719 2549 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 29 16:56:49.858950 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850721 2549 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 29 16:56:49.858950 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850724 2549 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 29 16:56:49.858950 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850726 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 29 16:56:49.858950 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850729 2549 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 29 16:56:49.858950 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850731 2549 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 29 16:56:49.858950 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850734 2549 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 29 16:56:49.858950 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850736 2549 feature_gate.go:328] unrecognized feature gate: Example May 29 16:56:49.858950 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850739 2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 29 16:56:49.858950 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850741 2549 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 29 16:56:49.858950 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850744 2549 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 29 16:56:49.858950 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850747 2549 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 29 16:56:49.859456 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850749 2549 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 29 16:56:49.859456 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850752 2549 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 29 16:56:49.859456 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850754 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 29 16:56:49.859456 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850757 2549 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 29 16:56:49.859456 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.850759 2549 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 29 16:56:49.859456 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.851445 2549 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 29 16:56:49.859456 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.858120 2549 server.go:530] "Kubelet version" kubeletVersion="v1.33.12" May 29 16:56:49.859456 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.858135 2549 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" May 29 16:56:49.859456 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858189 2549 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 29 16:56:49.859456 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858194 2549 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 29 16:56:49.859456 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858198 2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 29 16:56:49.859456 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858203 2549 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 29 16:56:49.859456 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858207 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 29 16:56:49.859456 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858210 2549 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 29 16:56:49.859456 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858214 2549 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 29 16:56:49.859833 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858216 2549 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 29 16:56:49.859833 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858220 2549 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 29 16:56:49.859833 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858222 2549 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 29 16:56:49.859833 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858225 2549 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 29 16:56:49.859833 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858228 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 29 16:56:49.859833 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858232 2549 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 29 16:56:49.859833 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858236 2549 feature_gate.go:328] unrecognized feature gate: Example May 29 16:56:49.859833 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858240 2549 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 29 16:56:49.859833 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858244 2549 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 29 16:56:49.859833 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858247 2549 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 29 16:56:49.859833 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858250 2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 29 16:56:49.859833 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858253 2549 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 29 16:56:49.859833 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858255 2549 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 29 16:56:49.859833 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858258 2549 feature_gate.go:328] unrecognized feature gate: SignatureStores May 29 16:56:49.859833 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858260 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 29 16:56:49.859833 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858264 2549 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 29 16:56:49.859833 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858267 2549 feature_gate.go:328] unrecognized feature gate: DualReplica May 29 16:56:49.859833 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858269 2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 29 16:56:49.859833 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858272 2549 feature_gate.go:328] unrecognized feature gate: PinnedImages May 29 16:56:49.859833 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858274 2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 29 16:56:49.860340 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858277 2549 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 29 16:56:49.860340 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858281 2549 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 29 16:56:49.860340 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858283 2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 29 16:56:49.860340 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858286 2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 29 16:56:49.860340 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858288 2549 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 29 16:56:49.860340 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858291 2549 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 29 16:56:49.860340 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858294 2549 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 29 16:56:49.860340 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858297 2549 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 29 16:56:49.860340 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858299 2549 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 29 16:56:49.860340 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858302 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 29 16:56:49.860340 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858305 2549 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 29 16:56:49.860340 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858307 2549 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 29 16:56:49.860340 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858312 2549 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 29 16:56:49.860340 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858316 2549 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 29 16:56:49.860340 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858320 2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 29 16:56:49.860340 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858323 2549 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 29 16:56:49.860340 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858326 2549 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 29 16:56:49.860340 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858329 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 29 16:56:49.860340 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858332 2549 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 29 16:56:49.860798 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858334 2549 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 29 16:56:49.860798 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858337 2549 feature_gate.go:328] unrecognized feature gate: NewOLM May 29 16:56:49.860798 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858340 2549 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 29 16:56:49.860798 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858342 2549 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 29 16:56:49.860798 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858345 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 29 16:56:49.860798 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858347 2549 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 29 16:56:49.860798 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858350 2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 29 16:56:49.860798 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858353 2549 feature_gate.go:328] unrecognized feature gate: Example2 May 29 16:56:49.860798 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858355 2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 29 16:56:49.860798 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858358 2549 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 29 16:56:49.860798 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858361 2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 29 16:56:49.860798 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858364 2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 29 16:56:49.860798 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858366 2549 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 29 16:56:49.860798 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858369 2549 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 29 16:56:49.860798 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858372 2549 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 29 16:56:49.860798 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858374 2549 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 29 16:56:49.860798 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858377 2549 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 29 16:56:49.860798 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858381 2549 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 29 16:56:49.860798 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858386 2549 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 29 16:56:49.860798 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858390 2549 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 29 16:56:49.861385 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858394 2549 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 29 16:56:49.861385 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858398 2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 29 16:56:49.861385 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858401 2549 feature_gate.go:328] unrecognized feature gate: OVNObservability May 29 16:56:49.861385 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858404 2549 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 29 16:56:49.861385 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858406 2549 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 29 16:56:49.861385 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858410 2549 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 29 16:56:49.861385 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858413 2549 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 29 16:56:49.861385 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858415 2549 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 29 16:56:49.861385 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858418 2549 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 29 16:56:49.861385 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858420 2549 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 29 16:56:49.861385 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858423 2549 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 29 16:56:49.861385 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858426 2549 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 29 16:56:49.861385 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858428 2549 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 29 16:56:49.861385 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858431 2549 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 29 16:56:49.861385 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858434 2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 29 16:56:49.861385 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858436 2549 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 29 16:56:49.861385 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858439 2549 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 29 16:56:49.861385 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858441 2549 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 29 16:56:49.861385 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858444 2549 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 29 16:56:49.861871 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858446 2549 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 29 16:56:49.861871 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.858451 2549 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 29 16:56:49.861871 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858556 2549 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 29 16:56:49.861871 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858561 2549 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 29 16:56:49.861871 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858564 2549 feature_gate.go:328] unrecognized feature gate: SignatureStores May 29 16:56:49.861871 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858567 2549 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 29 16:56:49.861871 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858570 2549 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 29 16:56:49.861871 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858573 2549 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 29 16:56:49.861871 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858576 2549 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 29 16:56:49.861871 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858578 2549 feature_gate.go:328] unrecognized feature gate: Example May 29 16:56:49.861871 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858581 2549 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 29 16:56:49.861871 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858583 2549 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 29 16:56:49.861871 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858586 2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 29 16:56:49.861871 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858588 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 29 16:56:49.861871 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858591 2549 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 29 16:56:49.862244 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858593 2549 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 29 16:56:49.862244 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858596 2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 29 16:56:49.862244 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858598 2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 29 16:56:49.862244 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858601 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 29 16:56:49.862244 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858603 2549 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 29 16:56:49.862244 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858606 2549 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 29 16:56:49.862244 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858608 2549 feature_gate.go:328] unrecognized feature gate: Example2 May 29 16:56:49.862244 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858610 2549 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 29 16:56:49.862244 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858613 2549 feature_gate.go:328] unrecognized feature gate: NewOLM May 29 16:56:49.862244 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858616 2549 feature_gate.go:328] unrecognized feature gate: OVNObservability May 29 16:56:49.862244 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858620 2549 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 29 16:56:49.862244 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858624 2549 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 29 16:56:49.862244 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858629 2549 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 29 16:56:49.862244 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858632 2549 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 29 16:56:49.862244 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858634 2549 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 29 16:56:49.862244 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858637 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 29 16:56:49.862244 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858639 2549 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 29 16:56:49.862244 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858643 2549 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 29 16:56:49.862244 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858646 2549 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 29 16:56:49.862244 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858648 2549 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 29 16:56:49.862729 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858651 2549 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 29 16:56:49.862729 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858654 2549 feature_gate.go:328] unrecognized feature gate: PinnedImages May 29 16:56:49.862729 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858657 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 29 16:56:49.862729 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858659 2549 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 29 16:56:49.862729 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858662 2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 29 16:56:49.862729 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858664 2549 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 29 16:56:49.862729 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858667 2549 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 29 16:56:49.862729 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858669 2549 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 29 16:56:49.862729 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858671 2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 29 16:56:49.862729 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858674 2549 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 29 16:56:49.862729 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858676 2549 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 29 16:56:49.862729 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858679 2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 29 16:56:49.862729 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858681 2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 29 16:56:49.862729 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858684 2549 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 29 16:56:49.862729 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858688 2549 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 29 16:56:49.862729 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858691 2549 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 29 16:56:49.862729 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858695 2549 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 29 16:56:49.862729 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858699 2549 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 29 16:56:49.862729 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858704 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 29 16:56:49.863209 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858709 2549 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 29 16:56:49.863209 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858711 2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 29 16:56:49.863209 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858714 2549 feature_gate.go:328] unrecognized feature gate: DualReplica May 29 16:56:49.863209 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858717 2549 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 29 16:56:49.863209 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858719 2549 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 29 16:56:49.863209 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858722 2549 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 29 16:56:49.863209 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858725 2549 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 29 16:56:49.863209 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858727 2549 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 29 16:56:49.863209 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858730 2549 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 29 16:56:49.863209 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858732 2549 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 29 16:56:49.863209 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858735 2549 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 29 16:56:49.863209 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858737 2549 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 29 16:56:49.863209 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858740 2549 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 29 16:56:49.863209 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858742 2549 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 29 16:56:49.863209 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858745 2549 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 29 16:56:49.863209 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858748 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 29 16:56:49.863209 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858750 2549 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 29 16:56:49.863209 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858753 2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 29 16:56:49.863209 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858756 2549 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 29 16:56:49.863752 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858758 2549 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 29 16:56:49.863752 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858761 2549 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 29 16:56:49.863752 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858763 2549 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 29 16:56:49.863752 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858766 2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 29 16:56:49.863752 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858768 2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 29 16:56:49.863752 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858772 2549 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 29 16:56:49.863752 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858776 2549 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 29 16:56:49.863752 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858780 2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 29 16:56:49.863752 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858783 2549 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 29 16:56:49.863752 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858786 2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 29 16:56:49.863752 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858788 2549 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 29 16:56:49.863752 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858790 2549 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 29 16:56:49.863752 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858793 2549 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 29 16:56:49.863752 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858795 2549 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 29 16:56:49.863752 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:49.858798 2549 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 29 16:56:49.864297 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.858802 2549 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 29 16:56:49.864297 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.859564 2549 server.go:962] "Client rotation is on, will bootstrap in background" May 29 16:56:49.864297 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.863492 2549 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" May 29 16:56:49.864524 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.864511 2549 server.go:1019] "Starting client certificate rotation" May 29 16:56:49.864621 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.864604 2549 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" May 29 16:56:49.864660 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.864640 2549 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" May 29 16:56:49.886728 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.886705 2549 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" May 29 16:56:49.888880 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.888859 2549 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" May 29 16:56:49.907127 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.907104 2549 log.go:25] "Validated CRI v1 runtime API" May 29 16:56:49.916216 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.916199 2549 log.go:25] "Validated CRI v1 image API" May 29 16:56:49.918226 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.918206 2549 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" May 29 16:56:49.921190 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.921174 2549 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" May 29 16:56:49.925884 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.925859 2549 fs.go:135] Filesystem UUIDs: map[48bec062-a209-47fe-8c1b-9a55055866ee:/dev/nvme0n1p4 4ff33c31-19e6-4431-a928-cad1ce06b75f:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] May 29 16:56:49.925943 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.925882 2549 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] May 29 16:56:49.930767 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.930654 2549 manager.go:217] Machine: {Timestamp:2026-05-29 16:56:49.930156038 +0000 UTC m=+0.416585243 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099901 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2916b2cd1a9cd0668e1be4810fa252 SystemUUID:ec2916b2-cd1a-9cd0-668e-1be4810fa252 BootID:fbcfcb3e-a6ec-4f92-a9e8-f20510509615 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:98:5c:51:57:9d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:98:5c:51:57:9d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:26:37:e2:a1:d8:40 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} May 29 16:56:49.932104 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.932092 2549 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. May 29 16:56:49.932185 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.932173 2549 manager.go:233] Version: {KernelVersion:5.14.0-570.117.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260521-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} May 29 16:56:49.934546 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.934523 2549 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] May 29 16:56:49.934646 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.934530 2549 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nf5ts" May 29 16:56:49.934693 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.934552 2549 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-197.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} May 29 16:56:49.934742 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.934702 2549 topology_manager.go:138] "Creating topology manager with none policy" May 29 16:56:49.934742 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.934711 2549 container_manager_linux.go:306] "Creating device plugin manager" May 29 16:56:49.934742 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.934718 2549 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" May 29 16:56:49.935482 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.935471 2549 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" May 29 16:56:49.937144 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.937134 2549 state_mem.go:36] "Initialized new in-memory state store" May 29 16:56:49.937246 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.937237 2549 server.go:1267] "Using root directory" path="/var/lib/kubelet" May 29 16:56:49.939761 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.939748 2549 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nf5ts" May 29 16:56:49.941756 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.941745 2549 kubelet.go:491] "Attempting to sync node with API server" May 29 16:56:49.941797 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.941761 2549 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" May 29 16:56:49.941797 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.941773 2549 file.go:69] "Watching path" path="/etc/kubernetes/manifests" May 29 16:56:49.941797 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.941782 2549 kubelet.go:397] "Adding apiserver pod source" May 29 16:56:49.941797 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.941791 2549 apiserver.go:42] "Waiting for node sync before watching apiserver pods" May 29 16:56:49.942803 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.942790 2549 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" May 29 16:56:49.942856 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.942819 2549 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" May 29 16:56:49.946360 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.946339 2549 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.12-3.rhaos4.20.git64c6a00.el9" apiVersion="v1" May 29 16:56:49.948253 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.948240 2549 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" May 29 16:56:49.950329 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.950317 2549 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" May 29 16:56:49.950370 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.950335 2549 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" May 29 16:56:49.950370 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.950341 2549 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" May 29 16:56:49.950370 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.950347 2549 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" May 29 16:56:49.950370 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.950356 2549 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" May 29 16:56:49.950370 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.950362 2549 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" May 29 16:56:49.950370 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.950368 2549 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" May 29 16:56:49.950575 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.950373 2549 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" May 29 16:56:49.950575 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.950382 2549 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" May 29 16:56:49.950575 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.950388 2549 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" May 29 16:56:49.950575 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.950396 2549 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" May 29 16:56:49.950575 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.950402 2549 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" May 29 16:56:49.951217 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.951203 2549 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" May 29 16:56:49.951217 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.951219 2549 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" May 29 16:56:49.954701 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.954688 2549 watchdog_linux.go:99] "Systemd watchdog is not enabled" May 29 16:56:49.954742 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.954724 2549 server.go:1295] "Started kubelet" May 29 16:56:49.954835 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.954797 2549 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 May 29 16:56:49.957800 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.954835 2549 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 May 29 16:56:49.957959 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.957938 2549 server_v1.go:47] "podresources" method="list" useActivePods=true May 29 16:56:49.958330 ip-10-0-141-197 systemd[1]: Started Kubernetes Kubelet. May 29 16:56:49.959614 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.959398 2549 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" May 29 16:56:49.959712 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.959696 2549 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" May 29 16:56:49.961936 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.961920 2549 server.go:317] "Adding debug handlers to kubelet server" May 29 16:56:49.963398 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.963382 2549 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" May 29 16:56:49.964110 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.964095 2549 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-141-197.ec2.internal" not found May 29 16:56:49.966913 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.966894 2549 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" May 29 16:56:49.966913 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.966900 2549 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" May 29 16:56:49.967067 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.966978 2549 volume_manager.go:295] "The desired_state_of_world populator starts" May 29 16:56:49.967067 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.966990 2549 volume_manager.go:297] "Starting Kubelet Volume Manager" May 29 16:56:49.967183 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.967101 2549 desired_state_of_world_populator.go:150] "Desired state populator starts to run" May 29 16:56:49.967183 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.967180 2549 reconstruct.go:97] "Volume reconstruction finished" May 29 16:56:49.967282 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.967188 2549 reconciler.go:26] "Reconciler: start to sync state" May 29 16:56:49.967725 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:49.967694 2549 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-197.ec2.internal\" not found" May 29 16:56:49.967909 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.967888 2549 factory.go:55] Registering systemd factory May 29 16:56:49.967909 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.967910 2549 factory.go:223] Registration of the systemd container factory successfully May 29 16:56:49.968244 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.968228 2549 factory.go:153] Registering CRI-O factory May 29 16:56:49.968244 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.968246 2549 factory.go:223] Registration of the crio container factory successfully May 29 16:56:49.968367 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.968309 2549 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory May 29 16:56:49.968367 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.968328 2549 factory.go:103] Registering Raw factory May 29 16:56:49.968367 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.968346 2549 manager.go:1196] Started watching for new ooms in manager May 29 16:56:49.968973 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.968958 2549 manager.go:319] Starting recovery of all containers May 29 16:56:49.969978 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.969957 2549 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" May 29 16:56:49.970172 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:49.970151 2549 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" May 29 16:56:49.972899 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:49.972880 2549 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-141-197.ec2.internal\" not found" node="ip-10-0-141-197.ec2.internal" May 29 16:56:49.979568 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.979418 2549 manager.go:324] Recovery completed May 29 16:56:49.980097 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.980080 2549 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-141-197.ec2.internal" not found May 29 16:56:49.985298 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.985283 2549 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 29 16:56:49.987045 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.987029 2549 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-197.ec2.internal" event="NodeHasSufficientMemory" May 29 16:56:49.987123 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.987055 2549 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-197.ec2.internal" event="NodeHasNoDiskPressure" May 29 16:56:49.987123 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.987065 2549 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-197.ec2.internal" event="NodeHasSufficientPID" May 29 16:56:49.987481 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.987468 2549 cpu_manager.go:222] "Starting CPU manager" policy="none" May 29 16:56:49.987528 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.987481 2549 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" May 29 16:56:49.987528 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.987501 2549 state_mem.go:36] "Initialized new in-memory state store" May 29 16:56:49.989908 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.989896 2549 policy_none.go:49] "None policy: Start" May 29 16:56:49.989947 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.989913 2549 memory_manager.go:186] "Starting memorymanager" policy="None" May 29 16:56:49.989947 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:49.989924 2549 state_mem.go:35] "Initializing new in-memory state store" May 29 16:56:50.036391 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.036377 2549 manager.go:341] "Starting Device Plugin manager" May 29 16:56:50.071351 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:50.036406 2549 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" May 29 16:56:50.071351 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.036416 2549 server.go:85] "Starting device plugin registration server" May 29 16:56:50.071351 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.036671 2549 eviction_manager.go:189] "Eviction manager: starting control loop" May 29 16:56:50.071351 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.036681 2549 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" May 29 16:56:50.071351 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.036782 2549 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" May 29 16:56:50.071351 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.036887 2549 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" May 29 16:56:50.071351 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.036897 2549 plugin_manager.go:118] "Starting Kubelet Plugin Manager" May 29 16:56:50.071351 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:50.037401 2549 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" May 29 16:56:50.071351 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:50.037440 2549 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-197.ec2.internal\" not found" May 29 16:56:50.071351 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.039783 2549 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-141-197.ec2.internal" not found May 29 16:56:50.078413 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.078390 2549 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" May 29 16:56:50.079596 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.079578 2549 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" May 29 16:56:50.079679 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.079604 2549 status_manager.go:230] "Starting to sync pod status with apiserver" May 29 16:56:50.079679 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.079623 2549 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." May 29 16:56:50.079679 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.079629 2549 kubelet.go:2451] "Starting kubelet main sync loop" May 29 16:56:50.079679 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:50.079668 2549 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" May 29 16:56:50.083462 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.083441 2549 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" May 29 16:56:50.137676 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.137616 2549 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 29 16:56:50.138471 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.138457 2549 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-197.ec2.internal" event="NodeHasSufficientMemory" May 29 16:56:50.138529 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.138484 2549 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-197.ec2.internal" event="NodeHasNoDiskPressure" May 29 16:56:50.138529 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.138494 2549 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-197.ec2.internal" event="NodeHasSufficientPID" May 29 16:56:50.138529 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.138516 2549 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-197.ec2.internal" May 29 16:56:50.147648 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.147614 2549 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-197.ec2.internal" May 29 16:56:50.180687 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.180666 2549 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-197.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-197.ec2.internal"] May 29 16:56:50.183085 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.183070 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-197.ec2.internal" May 29 16:56:50.183175 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.183071 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-197.ec2.internal" May 29 16:56:50.208418 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.208395 2549 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-197.ec2.internal" May 29 16:56:50.212677 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.212664 2549 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-197.ec2.internal" May 29 16:56:50.220680 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.220666 2549 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" May 29 16:56:50.222225 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.222206 2549 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" May 29 16:56:50.269377 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.269355 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/dfaf61f08c1074bed45e475649697dac-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-197.ec2.internal\" (UID: \"dfaf61f08c1074bed45e475649697dac\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-197.ec2.internal" May 29 16:56:50.269461 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.269380 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dfaf61f08c1074bed45e475649697dac-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-197.ec2.internal\" (UID: \"dfaf61f08c1074bed45e475649697dac\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-197.ec2.internal" May 29 16:56:50.269461 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.269398 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/62bc43ff86f9d08efd1447f9d66d955f-config\") pod \"kube-apiserver-proxy-ip-10-0-141-197.ec2.internal\" (UID: \"62bc43ff86f9d08efd1447f9d66d955f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-197.ec2.internal" May 29 16:56:50.369706 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.369679 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/dfaf61f08c1074bed45e475649697dac-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-197.ec2.internal\" (UID: \"dfaf61f08c1074bed45e475649697dac\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-197.ec2.internal" May 29 16:56:50.369706 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.369706 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dfaf61f08c1074bed45e475649697dac-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-197.ec2.internal\" (UID: \"dfaf61f08c1074bed45e475649697dac\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-197.ec2.internal" May 29 16:56:50.369906 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.369722 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/62bc43ff86f9d08efd1447f9d66d955f-config\") pod \"kube-apiserver-proxy-ip-10-0-141-197.ec2.internal\" (UID: \"62bc43ff86f9d08efd1447f9d66d955f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-197.ec2.internal" May 29 16:56:50.369906 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.369773 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/62bc43ff86f9d08efd1447f9d66d955f-config\") pod \"kube-apiserver-proxy-ip-10-0-141-197.ec2.internal\" (UID: \"62bc43ff86f9d08efd1447f9d66d955f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-197.ec2.internal" May 29 16:56:50.369906 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.369805 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dfaf61f08c1074bed45e475649697dac-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-197.ec2.internal\" (UID: \"dfaf61f08c1074bed45e475649697dac\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-197.ec2.internal" May 29 16:56:50.369906 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.369773 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/dfaf61f08c1074bed45e475649697dac-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-197.ec2.internal\" (UID: \"dfaf61f08c1074bed45e475649697dac\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-197.ec2.internal" May 29 16:56:50.524313 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.524216 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-197.ec2.internal" May 29 16:56:50.524313 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.524251 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-197.ec2.internal" May 29 16:56:50.864798 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.864771 2549 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" May 29 16:56:50.865377 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.864923 2549 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" May 29 16:56:50.865377 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.864952 2549 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" May 29 16:56:50.865377 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.864974 2549 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" May 29 16:56:50.941519 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.941462 2549 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-28 16:51:49 +0000 UTC" deadline="2027-11-25 00:36:03.9666076 +0000 UTC" May 29 16:56:50.941519 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.941515 2549 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13063h39m13.025095619s" May 29 16:56:50.942556 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.942536 2549 apiserver.go:52] "Watching apiserver" May 29 16:56:50.947231 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.947212 2549 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" May 29 16:56:50.947593 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.947568 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-nfv6s","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2m2f6","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-197.ec2.internal","openshift-multus/network-metrics-daemon-ps265","openshift-network-diagnostics/network-check-target-crvwv","openshift-network-operator/iptables-alerter-wt655","kube-system/konnectivity-agent-w9jdd","kube-system/kube-apiserver-proxy-ip-10-0-141-197.ec2.internal","openshift-cluster-node-tuning-operator/tuned-9nn7d","openshift-image-registry/node-ca-qdvbn","openshift-multus/multus-4fwqn","openshift-multus/multus-additional-cni-plugins-52zq6"] May 29 16:56:50.948954 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.948932 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:50.950168 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.950151 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2m2f6" May 29 16:56:50.951142 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.951104 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" May 29 16:56:50.951346 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.951327 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" May 29 16:56:50.951432 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.951332 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" May 29 16:56:50.951489 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.951465 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" May 29 16:56:50.952066 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.951647 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" May 29 16:56:50.952066 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.951812 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-2kt7n\"" May 29 16:56:50.952066 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.951826 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" May 29 16:56:50.952480 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.952453 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-jftcr\"" May 29 16:56:50.952603 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.952585 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" May 29 16:56:50.952785 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.952727 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" May 29 16:56:50.953000 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.952825 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" May 29 16:56:50.954192 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.954160 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ps265" May 29 16:56:50.954281 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.954198 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-crvwv" May 29 16:56:50.954331 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:50.954291 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-crvwv" podUID="be92fdf2-3d2b-4075-8919-8697586347c6" May 29 16:56:50.954383 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:50.954327 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ps265" podUID="7d457970-4377-4157-9d77-81db07916925" May 29 16:56:50.955416 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.955398 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wt655" May 29 16:56:50.956526 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.956509 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-w9jdd" May 29 16:56:50.956964 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.956950 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" May 29 16:56:50.957025 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.957004 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" May 29 16:56:50.957084 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.957057 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-6q7wv\"" May 29 16:56:50.957124 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.957063 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" May 29 16:56:50.957726 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.957714 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:50.958004 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.957988 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" May 29 16:56:50.958128 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.958113 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" May 29 16:56:50.958344 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.958328 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-dd9jr\"" May 29 16:56:50.958984 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.958968 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qdvbn" May 29 16:56:50.959365 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.959346 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" May 29 16:56:50.959479 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.959462 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" May 29 16:56:50.959560 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.959545 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-vwtrk\"" May 29 16:56:50.960122 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.960106 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4fwqn" May 29 16:56:50.960400 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.960382 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" May 29 16:56:50.960508 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.960493 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" May 29 16:56:50.960715 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.960699 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-vszf7\"" May 29 16:56:50.960920 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.960907 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" May 29 16:56:50.961450 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.961432 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-52zq6" May 29 16:56:50.961524 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.961498 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-xfdzr\"" May 29 16:56:50.961881 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.961862 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" May 29 16:56:50.961881 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.961874 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" May 29 16:56:50.962002 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.961865 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" May 29 16:56:50.962115 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.962103 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" May 29 16:56:50.962918 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.962903 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" May 29 16:56:50.963204 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.963187 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-5cdvw\"" May 29 16:56:50.963272 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.963215 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" May 29 16:56:50.967408 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.967391 2549 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" May 29 16:56:50.968088 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.968075 2549 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" May 29 16:56:50.972273 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.972258 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/56335f96-b27f-4a52-beb7-656001cdea57-cni-binary-copy\") pod \"multus-additional-cni-plugins-52zq6\" (UID: \"56335f96-b27f-4a52-beb7-656001cdea57\") " pod="openshift-multus/multus-additional-cni-plugins-52zq6" May 29 16:56:50.972362 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.972290 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f153a7a3-935e-44ac-8a76-d3ab6333809a-etc-sysctl-d\") pod \"tuned-9nn7d\" (UID: \"f153a7a3-935e-44ac-8a76-d3ab6333809a\") " pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:50.972362 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.972313 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/962b0454-8569-4239-b4b3-b88b0fe86570-serviceca\") pod \"node-ca-qdvbn\" (UID: \"962b0454-8569-4239-b4b3-b88b0fe86570\") " pod="openshift-image-registry/node-ca-qdvbn" May 29 16:56:50.972362 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.972337 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-multus-socket-dir-parent\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:50.972466 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.972379 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-host-var-lib-kubelet\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:50.972466 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.972404 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/46b01332-3570-4d2c-b52f-72165dbc2cf8-sys-fs\") pod \"aws-ebs-csi-driver-node-2m2f6\" (UID: \"46b01332-3570-4d2c-b52f-72165dbc2cf8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2m2f6" May 29 16:56:50.972466 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.972425 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnllv\" (UniqueName: \"kubernetes.io/projected/46b01332-3570-4d2c-b52f-72165dbc2cf8-kube-api-access-dnllv\") pod \"aws-ebs-csi-driver-node-2m2f6\" (UID: \"46b01332-3570-4d2c-b52f-72165dbc2cf8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2m2f6" May 29 16:56:50.972466 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.972454 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-host-run-ovn-kubernetes\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:50.972579 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.972477 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6b9d\" (UniqueName: \"kubernetes.io/projected/be92fdf2-3d2b-4075-8919-8697586347c6-kube-api-access-b6b9d\") pod \"network-check-target-crvwv\" (UID: \"be92fdf2-3d2b-4075-8919-8697586347c6\") " pod="openshift-network-diagnostics/network-check-target-crvwv" May 29 16:56:50.972579 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.972493 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-etc-kubernetes\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:50.972579 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.972513 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxxng\" (UniqueName: \"kubernetes.io/projected/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-kube-api-access-mxxng\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:50.972579 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.972528 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwj6s\" (UniqueName: \"kubernetes.io/projected/56335f96-b27f-4a52-beb7-656001cdea57-kube-api-access-gwj6s\") pod \"multus-additional-cni-plugins-52zq6\" (UID: \"56335f96-b27f-4a52-beb7-656001cdea57\") " pod="openshift-multus/multus-additional-cni-plugins-52zq6" May 29 16:56:50.972579 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.972546 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f6995bff-b11d-42e1-87ea-ecc131cd8810-ovnkube-script-lib\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:50.972579 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.972573 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f153a7a3-935e-44ac-8a76-d3ab6333809a-lib-modules\") pod \"tuned-9nn7d\" (UID: \"f153a7a3-935e-44ac-8a76-d3ab6333809a\") " pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:50.972745 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.972594 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-multus-daemon-config\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:50.972745 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.972611 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-host-run-multus-certs\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:50.972745 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.972626 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f153a7a3-935e-44ac-8a76-d3ab6333809a-etc-kubernetes\") pod \"tuned-9nn7d\" (UID: \"f153a7a3-935e-44ac-8a76-d3ab6333809a\") " pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:50.972745 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.972654 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5z6bc\" (UniqueName: \"kubernetes.io/projected/f153a7a3-935e-44ac-8a76-d3ab6333809a-kube-api-access-5z6bc\") pod \"tuned-9nn7d\" (UID: \"f153a7a3-935e-44ac-8a76-d3ab6333809a\") " pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:50.972745 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.972674 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/46b01332-3570-4d2c-b52f-72165dbc2cf8-etc-selinux\") pod \"aws-ebs-csi-driver-node-2m2f6\" (UID: \"46b01332-3570-4d2c-b52f-72165dbc2cf8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2m2f6" May 29 16:56:50.972745 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.972690 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f153a7a3-935e-44ac-8a76-d3ab6333809a-etc-sysconfig\") pod \"tuned-9nn7d\" (UID: \"f153a7a3-935e-44ac-8a76-d3ab6333809a\") " pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:50.972745 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.972711 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-host-var-lib-cni-multus\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:50.972745 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.972729 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/46b01332-3570-4d2c-b52f-72165dbc2cf8-socket-dir\") pod \"aws-ebs-csi-driver-node-2m2f6\" (UID: \"46b01332-3570-4d2c-b52f-72165dbc2cf8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2m2f6" May 29 16:56:50.972745 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.972742 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/56335f96-b27f-4a52-beb7-656001cdea57-cnibin\") pod \"multus-additional-cni-plugins-52zq6\" (UID: \"56335f96-b27f-4a52-beb7-656001cdea57\") " pod="openshift-multus/multus-additional-cni-plugins-52zq6" May 29 16:56:50.973119 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.972772 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f153a7a3-935e-44ac-8a76-d3ab6333809a-var-lib-kubelet\") pod \"tuned-9nn7d\" (UID: \"f153a7a3-935e-44ac-8a76-d3ab6333809a\") " pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:50.973119 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.972791 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-log-socket\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:50.973119 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.972807 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f6995bff-b11d-42e1-87ea-ecc131cd8810-ovn-node-metrics-cert\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:50.973119 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.972823 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkgng\" (UniqueName: \"kubernetes.io/projected/7d457970-4377-4157-9d77-81db07916925-kube-api-access-zkgng\") pod \"network-metrics-daemon-ps265\" (UID: \"7d457970-4377-4157-9d77-81db07916925\") " pod="openshift-multus/network-metrics-daemon-ps265" May 29 16:56:50.973119 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.972859 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f153a7a3-935e-44ac-8a76-d3ab6333809a-etc-modprobe-d\") pod \"tuned-9nn7d\" (UID: \"f153a7a3-935e-44ac-8a76-d3ab6333809a\") " pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:50.973119 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.972876 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f153a7a3-935e-44ac-8a76-d3ab6333809a-etc-tuned\") pod \"tuned-9nn7d\" (UID: \"f153a7a3-935e-44ac-8a76-d3ab6333809a\") " pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:50.973119 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.972918 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hn8kk\" (UniqueName: \"kubernetes.io/projected/962b0454-8569-4239-b4b3-b88b0fe86570-kube-api-access-hn8kk\") pod \"node-ca-qdvbn\" (UID: \"962b0454-8569-4239-b4b3-b88b0fe86570\") " pod="openshift-image-registry/node-ca-qdvbn" May 29 16:56:50.973119 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.972950 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e270884a-3a8f-4299-9cfa-a44e2bf1768f-iptables-alerter-script\") pod \"iptables-alerter-wt655\" (UID: \"e270884a-3a8f-4299-9cfa-a44e2bf1768f\") " pod="openshift-network-operator/iptables-alerter-wt655" May 29 16:56:50.973119 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.972975 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-systemd-units\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:50.973119 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.972998 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-host-run-netns\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:50.973119 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.973023 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjg4m\" (UniqueName: \"kubernetes.io/projected/f6995bff-b11d-42e1-87ea-ecc131cd8810-kube-api-access-wjg4m\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:50.973119 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.973046 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f153a7a3-935e-44ac-8a76-d3ab6333809a-host\") pod \"tuned-9nn7d\" (UID: \"f153a7a3-935e-44ac-8a76-d3ab6333809a\") " pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:50.973119 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.973070 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-cnibin\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:50.973119 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.973115 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-host-run-netns\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:50.973598 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.973155 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-system-cni-dir\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:50.973598 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.973180 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-multus-conf-dir\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:50.973598 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.973202 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/46b01332-3570-4d2c-b52f-72165dbc2cf8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2m2f6\" (UID: \"46b01332-3570-4d2c-b52f-72165dbc2cf8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2m2f6" May 29 16:56:50.973598 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.973217 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f153a7a3-935e-44ac-8a76-d3ab6333809a-run\") pod \"tuned-9nn7d\" (UID: \"f153a7a3-935e-44ac-8a76-d3ab6333809a\") " pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:50.973598 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.973232 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f153a7a3-935e-44ac-8a76-d3ab6333809a-sys\") pod \"tuned-9nn7d\" (UID: \"f153a7a3-935e-44ac-8a76-d3ab6333809a\") " pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:50.973598 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.973247 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/54c6a295-b9b0-4cda-9a11-33a3b53a7dfb-agent-certs\") pod \"konnectivity-agent-w9jdd\" (UID: \"54c6a295-b9b0-4cda-9a11-33a3b53a7dfb\") " pod="kube-system/konnectivity-agent-w9jdd" May 29 16:56:50.973598 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.973266 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-etc-openvswitch\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:50.973598 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.973281 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-run-ovn\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:50.973598 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.973317 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-host-cni-netd\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:50.973598 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.973351 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:50.973598 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.973379 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f6995bff-b11d-42e1-87ea-ecc131cd8810-env-overrides\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:50.973598 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.973402 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-host-kubelet\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:50.973598 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.973423 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-node-log\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:50.973598 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.973444 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f6995bff-b11d-42e1-87ea-ecc131cd8810-ovnkube-config\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:50.973598 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.973464 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d457970-4377-4157-9d77-81db07916925-metrics-certs\") pod \"network-metrics-daemon-ps265\" (UID: \"7d457970-4377-4157-9d77-81db07916925\") " pod="openshift-multus/network-metrics-daemon-ps265" May 29 16:56:50.973598 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.973478 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f153a7a3-935e-44ac-8a76-d3ab6333809a-tmp\") pod \"tuned-9nn7d\" (UID: \"f153a7a3-935e-44ac-8a76-d3ab6333809a\") " pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:50.973598 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.973493 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f153a7a3-935e-44ac-8a76-d3ab6333809a-etc-sysctl-conf\") pod \"tuned-9nn7d\" (UID: \"f153a7a3-935e-44ac-8a76-d3ab6333809a\") " pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:50.974342 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.973515 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-host-run-k8s-cni-cncf-io\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:50.974342 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.973535 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/46b01332-3570-4d2c-b52f-72165dbc2cf8-registration-dir\") pod \"aws-ebs-csi-driver-node-2m2f6\" (UID: \"46b01332-3570-4d2c-b52f-72165dbc2cf8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2m2f6" May 29 16:56:50.974342 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.973549 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/56335f96-b27f-4a52-beb7-656001cdea57-os-release\") pod \"multus-additional-cni-plugins-52zq6\" (UID: \"56335f96-b27f-4a52-beb7-656001cdea57\") " pod="openshift-multus/multus-additional-cni-plugins-52zq6" May 29 16:56:50.974342 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.973571 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-host-slash\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:50.974342 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.973591 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-var-lib-openvswitch\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:50.974342 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.973605 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-host-cni-bin\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:50.974342 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.973618 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-os-release\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:50.974342 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.973645 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-host-var-lib-cni-bin\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:50.974342 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.973679 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-multus-cni-dir\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:50.974342 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.973703 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/54c6a295-b9b0-4cda-9a11-33a3b53a7dfb-konnectivity-ca\") pod \"konnectivity-agent-w9jdd\" (UID: \"54c6a295-b9b0-4cda-9a11-33a3b53a7dfb\") " pod="kube-system/konnectivity-agent-w9jdd" May 29 16:56:50.974342 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.973728 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56335f96-b27f-4a52-beb7-656001cdea57-system-cni-dir\") pod \"multus-additional-cni-plugins-52zq6\" (UID: \"56335f96-b27f-4a52-beb7-656001cdea57\") " pod="openshift-multus/multus-additional-cni-plugins-52zq6" May 29 16:56:50.974342 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.973751 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/56335f96-b27f-4a52-beb7-656001cdea57-tuning-conf-dir\") pod \"multus-additional-cni-plugins-52zq6\" (UID: \"56335f96-b27f-4a52-beb7-656001cdea57\") " pod="openshift-multus/multus-additional-cni-plugins-52zq6" May 29 16:56:50.974342 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.973775 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/56335f96-b27f-4a52-beb7-656001cdea57-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-52zq6\" (UID: \"56335f96-b27f-4a52-beb7-656001cdea57\") " pod="openshift-multus/multus-additional-cni-plugins-52zq6" May 29 16:56:50.974342 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.973798 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-run-openvswitch\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:50.974342 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.973821 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f153a7a3-935e-44ac-8a76-d3ab6333809a-etc-systemd\") pod \"tuned-9nn7d\" (UID: \"f153a7a3-935e-44ac-8a76-d3ab6333809a\") " pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:50.974342 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.973859 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e270884a-3a8f-4299-9cfa-a44e2bf1768f-host-slash\") pod \"iptables-alerter-wt655\" (UID: \"e270884a-3a8f-4299-9cfa-a44e2bf1768f\") " pod="openshift-network-operator/iptables-alerter-wt655" May 29 16:56:50.974823 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.973884 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/56335f96-b27f-4a52-beb7-656001cdea57-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-52zq6\" (UID: \"56335f96-b27f-4a52-beb7-656001cdea57\") " pod="openshift-multus/multus-additional-cni-plugins-52zq6" May 29 16:56:50.974823 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.973926 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-run-systemd\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:50.974823 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.973948 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/962b0454-8569-4239-b4b3-b88b0fe86570-host\") pod \"node-ca-qdvbn\" (UID: \"962b0454-8569-4239-b4b3-b88b0fe86570\") " pod="openshift-image-registry/node-ca-qdvbn" May 29 16:56:50.974823 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.973972 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-cni-binary-copy\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:50.974823 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.973993 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-hostroot\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:50.974823 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.974017 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdv6p\" (UniqueName: \"kubernetes.io/projected/e270884a-3a8f-4299-9cfa-a44e2bf1768f-kube-api-access-rdv6p\") pod \"iptables-alerter-wt655\" (UID: \"e270884a-3a8f-4299-9cfa-a44e2bf1768f\") " pod="openshift-network-operator/iptables-alerter-wt655" May 29 16:56:50.974823 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.974040 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/46b01332-3570-4d2c-b52f-72165dbc2cf8-device-dir\") pod \"aws-ebs-csi-driver-node-2m2f6\" (UID: \"46b01332-3570-4d2c-b52f-72165dbc2cf8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2m2f6" May 29 16:56:50.976713 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:50.976691 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62bc43ff86f9d08efd1447f9d66d955f.slice/crio-52973df01d34749d420bbaf983ff6a812d9e6a9ce6ce8f3a84191e2bbacd687d WatchSource:0}: Error finding container 52973df01d34749d420bbaf983ff6a812d9e6a9ce6ce8f3a84191e2bbacd687d: Status 404 returned error can't find the container with id 52973df01d34749d420bbaf983ff6a812d9e6a9ce6ce8f3a84191e2bbacd687d May 29 16:56:50.976913 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:50.976892 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfaf61f08c1074bed45e475649697dac.slice/crio-0d5d3752d73591c0e3ac160b71de844158b9b98a1096ff1311c260d9581195df WatchSource:0}: Error finding container 0d5d3752d73591c0e3ac160b71de844158b9b98a1096ff1311c260d9581195df: Status 404 returned error can't find the container with id 0d5d3752d73591c0e3ac160b71de844158b9b98a1096ff1311c260d9581195df May 29 16:56:50.979381 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.979285 2549 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" May 29 16:56:50.981432 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.981419 2549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 29 16:56:50.998980 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:50.998958 2549 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-gjfh9" May 29 16:56:51.006335 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.006316 2549 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-gjfh9" May 29 16:56:51.075190 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075163 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f153a7a3-935e-44ac-8a76-d3ab6333809a-run\") pod \"tuned-9nn7d\" (UID: \"f153a7a3-935e-44ac-8a76-d3ab6333809a\") " pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:51.075190 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075191 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f153a7a3-935e-44ac-8a76-d3ab6333809a-sys\") pod \"tuned-9nn7d\" (UID: \"f153a7a3-935e-44ac-8a76-d3ab6333809a\") " pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:51.075353 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075208 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/54c6a295-b9b0-4cda-9a11-33a3b53a7dfb-agent-certs\") pod \"konnectivity-agent-w9jdd\" (UID: \"54c6a295-b9b0-4cda-9a11-33a3b53a7dfb\") " pod="kube-system/konnectivity-agent-w9jdd" May 29 16:56:51.075353 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075224 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-etc-openvswitch\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:51.075353 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075239 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-run-ovn\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:51.075353 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075254 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-host-cni-netd\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:51.075353 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075295 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f153a7a3-935e-44ac-8a76-d3ab6333809a-run\") pod \"tuned-9nn7d\" (UID: \"f153a7a3-935e-44ac-8a76-d3ab6333809a\") " pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:51.075353 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075305 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-host-cni-netd\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:51.075353 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075298 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-etc-openvswitch\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:51.075353 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075306 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-run-ovn\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:51.075353 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075324 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:51.075754 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075368 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f6995bff-b11d-42e1-87ea-ecc131cd8810-env-overrides\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:51.075754 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075394 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-host-kubelet\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:51.075754 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075417 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-node-log\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:51.075754 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075421 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-host-kubelet\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:51.075754 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075389 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f153a7a3-935e-44ac-8a76-d3ab6333809a-sys\") pod \"tuned-9nn7d\" (UID: \"f153a7a3-935e-44ac-8a76-d3ab6333809a\") " pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:51.075754 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075440 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f6995bff-b11d-42e1-87ea-ecc131cd8810-ovnkube-config\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:51.075754 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075453 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:51.075754 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075463 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-node-log\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:51.075754 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075465 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d457970-4377-4157-9d77-81db07916925-metrics-certs\") pod \"network-metrics-daemon-ps265\" (UID: \"7d457970-4377-4157-9d77-81db07916925\") " pod="openshift-multus/network-metrics-daemon-ps265" May 29 16:56:51.075754 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075489 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f153a7a3-935e-44ac-8a76-d3ab6333809a-tmp\") pod \"tuned-9nn7d\" (UID: \"f153a7a3-935e-44ac-8a76-d3ab6333809a\") " pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:51.075754 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075512 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f153a7a3-935e-44ac-8a76-d3ab6333809a-etc-sysctl-conf\") pod \"tuned-9nn7d\" (UID: \"f153a7a3-935e-44ac-8a76-d3ab6333809a\") " pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:51.075754 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075537 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-host-run-k8s-cni-cncf-io\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:51.075754 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:51.075564 2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 29 16:56:51.075754 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075587 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/46b01332-3570-4d2c-b52f-72165dbc2cf8-registration-dir\") pod \"aws-ebs-csi-driver-node-2m2f6\" (UID: \"46b01332-3570-4d2c-b52f-72165dbc2cf8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2m2f6" May 29 16:56:51.075754 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075596 2549 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" May 29 16:56:51.075754 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075612 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/56335f96-b27f-4a52-beb7-656001cdea57-os-release\") pod \"multus-additional-cni-plugins-52zq6\" (UID: \"56335f96-b27f-4a52-beb7-656001cdea57\") " pod="openshift-multus/multus-additional-cni-plugins-52zq6" May 29 16:56:51.075754 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:51.075660 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d457970-4377-4157-9d77-81db07916925-metrics-certs podName:7d457970-4377-4157-9d77-81db07916925 nodeName:}" failed. No retries permitted until 2026-05-29 16:56:51.575613592 +0000 UTC m=+2.062042793 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7d457970-4377-4157-9d77-81db07916925-metrics-certs") pod "network-metrics-daemon-ps265" (UID: "7d457970-4377-4157-9d77-81db07916925") : object "openshift-multus"/"metrics-daemon-secret" not registered May 29 16:56:51.076510 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075667 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f153a7a3-935e-44ac-8a76-d3ab6333809a-etc-sysctl-conf\") pod \"tuned-9nn7d\" (UID: \"f153a7a3-935e-44ac-8a76-d3ab6333809a\") " pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:51.076510 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075685 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-host-run-k8s-cni-cncf-io\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:51.076510 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075691 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-host-slash\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:51.076510 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075719 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/46b01332-3570-4d2c-b52f-72165dbc2cf8-registration-dir\") pod \"aws-ebs-csi-driver-node-2m2f6\" (UID: \"46b01332-3570-4d2c-b52f-72165dbc2cf8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2m2f6" May 29 16:56:51.076510 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075720 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-var-lib-openvswitch\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:51.076510 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075660 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/56335f96-b27f-4a52-beb7-656001cdea57-os-release\") pod \"multus-additional-cni-plugins-52zq6\" (UID: \"56335f96-b27f-4a52-beb7-656001cdea57\") " pod="openshift-multus/multus-additional-cni-plugins-52zq6" May 29 16:56:51.076510 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075755 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-host-cni-bin\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:51.076510 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075763 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-var-lib-openvswitch\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:51.076510 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075783 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-os-release\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:51.076510 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075802 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-host-slash\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:51.076510 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075807 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-host-var-lib-cni-bin\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:51.076510 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075833 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-multus-cni-dir\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:51.076510 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075876 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/54c6a295-b9b0-4cda-9a11-33a3b53a7dfb-konnectivity-ca\") pod \"konnectivity-agent-w9jdd\" (UID: \"54c6a295-b9b0-4cda-9a11-33a3b53a7dfb\") " pod="kube-system/konnectivity-agent-w9jdd" May 29 16:56:51.076510 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075881 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-host-var-lib-cni-bin\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:51.076510 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075879 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f6995bff-b11d-42e1-87ea-ecc131cd8810-env-overrides\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:51.076510 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075834 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-host-cni-bin\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:51.076510 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075906 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-os-release\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:51.076510 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075929 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56335f96-b27f-4a52-beb7-656001cdea57-system-cni-dir\") pod \"multus-additional-cni-plugins-52zq6\" (UID: \"56335f96-b27f-4a52-beb7-656001cdea57\") " pod="openshift-multus/multus-additional-cni-plugins-52zq6" May 29 16:56:51.077293 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075957 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/56335f96-b27f-4a52-beb7-656001cdea57-tuning-conf-dir\") pod \"multus-additional-cni-plugins-52zq6\" (UID: \"56335f96-b27f-4a52-beb7-656001cdea57\") " pod="openshift-multus/multus-additional-cni-plugins-52zq6" May 29 16:56:51.077293 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075984 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/56335f96-b27f-4a52-beb7-656001cdea57-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-52zq6\" (UID: \"56335f96-b27f-4a52-beb7-656001cdea57\") " pod="openshift-multus/multus-additional-cni-plugins-52zq6" May 29 16:56:51.077293 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075990 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56335f96-b27f-4a52-beb7-656001cdea57-system-cni-dir\") pod \"multus-additional-cni-plugins-52zq6\" (UID: \"56335f96-b27f-4a52-beb7-656001cdea57\") " pod="openshift-multus/multus-additional-cni-plugins-52zq6" May 29 16:56:51.077293 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.075999 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-multus-cni-dir\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:51.077293 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.076009 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-run-openvswitch\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:51.077293 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.076047 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f153a7a3-935e-44ac-8a76-d3ab6333809a-etc-systemd\") pod \"tuned-9nn7d\" (UID: \"f153a7a3-935e-44ac-8a76-d3ab6333809a\") " pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:51.077293 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.076075 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e270884a-3a8f-4299-9cfa-a44e2bf1768f-host-slash\") pod \"iptables-alerter-wt655\" (UID: \"e270884a-3a8f-4299-9cfa-a44e2bf1768f\") " pod="openshift-network-operator/iptables-alerter-wt655" May 29 16:56:51.077293 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.076091 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f6995bff-b11d-42e1-87ea-ecc131cd8810-ovnkube-config\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:51.077293 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.076114 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/56335f96-b27f-4a52-beb7-656001cdea57-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-52zq6\" (UID: \"56335f96-b27f-4a52-beb7-656001cdea57\") " pod="openshift-multus/multus-additional-cni-plugins-52zq6" May 29 16:56:51.077293 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.076119 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f153a7a3-935e-44ac-8a76-d3ab6333809a-etc-systemd\") pod \"tuned-9nn7d\" (UID: \"f153a7a3-935e-44ac-8a76-d3ab6333809a\") " pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:51.077293 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.076081 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-run-openvswitch\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:51.077293 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.076138 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-run-systemd\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:51.077293 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.076137 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/56335f96-b27f-4a52-beb7-656001cdea57-tuning-conf-dir\") pod \"multus-additional-cni-plugins-52zq6\" (UID: \"56335f96-b27f-4a52-beb7-656001cdea57\") " pod="openshift-multus/multus-additional-cni-plugins-52zq6" May 29 16:56:51.077293 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.076162 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e270884a-3a8f-4299-9cfa-a44e2bf1768f-host-slash\") pod \"iptables-alerter-wt655\" (UID: \"e270884a-3a8f-4299-9cfa-a44e2bf1768f\") " pod="openshift-network-operator/iptables-alerter-wt655" May 29 16:56:51.077293 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.076187 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-run-systemd\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:51.077293 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.076220 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/962b0454-8569-4239-b4b3-b88b0fe86570-host\") pod \"node-ca-qdvbn\" (UID: \"962b0454-8569-4239-b4b3-b88b0fe86570\") " pod="openshift-image-registry/node-ca-qdvbn" May 29 16:56:51.077293 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.076247 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-cni-binary-copy\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:51.078085 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.076267 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/962b0454-8569-4239-b4b3-b88b0fe86570-host\") pod \"node-ca-qdvbn\" (UID: \"962b0454-8569-4239-b4b3-b88b0fe86570\") " pod="openshift-image-registry/node-ca-qdvbn" May 29 16:56:51.078085 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.076270 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-hostroot\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:51.078085 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.076313 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rdv6p\" (UniqueName: \"kubernetes.io/projected/e270884a-3a8f-4299-9cfa-a44e2bf1768f-kube-api-access-rdv6p\") pod \"iptables-alerter-wt655\" (UID: \"e270884a-3a8f-4299-9cfa-a44e2bf1768f\") " pod="openshift-network-operator/iptables-alerter-wt655" May 29 16:56:51.078085 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.076349 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-hostroot\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:51.078085 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.076460 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/46b01332-3570-4d2c-b52f-72165dbc2cf8-device-dir\") pod \"aws-ebs-csi-driver-node-2m2f6\" (UID: \"46b01332-3570-4d2c-b52f-72165dbc2cf8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2m2f6" May 29 16:56:51.078085 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.076490 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/56335f96-b27f-4a52-beb7-656001cdea57-cni-binary-copy\") pod \"multus-additional-cni-plugins-52zq6\" (UID: \"56335f96-b27f-4a52-beb7-656001cdea57\") " pod="openshift-multus/multus-additional-cni-plugins-52zq6" May 29 16:56:51.078085 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.076516 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f153a7a3-935e-44ac-8a76-d3ab6333809a-etc-sysctl-d\") pod \"tuned-9nn7d\" (UID: \"f153a7a3-935e-44ac-8a76-d3ab6333809a\") " pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:51.078085 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.076540 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/962b0454-8569-4239-b4b3-b88b0fe86570-serviceca\") pod \"node-ca-qdvbn\" (UID: \"962b0454-8569-4239-b4b3-b88b0fe86570\") " pod="openshift-image-registry/node-ca-qdvbn" May 29 16:56:51.078085 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.076552 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/56335f96-b27f-4a52-beb7-656001cdea57-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-52zq6\" (UID: \"56335f96-b27f-4a52-beb7-656001cdea57\") " pod="openshift-multus/multus-additional-cni-plugins-52zq6" May 29 16:56:51.078085 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.076564 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-multus-socket-dir-parent\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:51.078085 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.076591 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-host-var-lib-kubelet\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:51.078085 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.076611 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/46b01332-3570-4d2c-b52f-72165dbc2cf8-sys-fs\") pod \"aws-ebs-csi-driver-node-2m2f6\" (UID: \"46b01332-3570-4d2c-b52f-72165dbc2cf8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2m2f6" May 29 16:56:51.078085 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.076633 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dnllv\" (UniqueName: \"kubernetes.io/projected/46b01332-3570-4d2c-b52f-72165dbc2cf8-kube-api-access-dnllv\") pod \"aws-ebs-csi-driver-node-2m2f6\" (UID: \"46b01332-3570-4d2c-b52f-72165dbc2cf8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2m2f6" May 29 16:56:51.078085 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.076637 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/56335f96-b27f-4a52-beb7-656001cdea57-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-52zq6\" (UID: \"56335f96-b27f-4a52-beb7-656001cdea57\") " pod="openshift-multus/multus-additional-cni-plugins-52zq6" May 29 16:56:51.078085 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.076655 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-host-run-ovn-kubernetes\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:51.078085 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.076681 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f153a7a3-935e-44ac-8a76-d3ab6333809a-etc-sysctl-d\") pod \"tuned-9nn7d\" (UID: \"f153a7a3-935e-44ac-8a76-d3ab6333809a\") " pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:51.078085 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.076682 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b6b9d\" (UniqueName: \"kubernetes.io/projected/be92fdf2-3d2b-4075-8919-8697586347c6-kube-api-access-b6b9d\") pod \"network-check-target-crvwv\" (UID: \"be92fdf2-3d2b-4075-8919-8697586347c6\") " pod="openshift-network-diagnostics/network-check-target-crvwv" May 29 16:56:51.078652 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.076702 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/46b01332-3570-4d2c-b52f-72165dbc2cf8-device-dir\") pod \"aws-ebs-csi-driver-node-2m2f6\" (UID: \"46b01332-3570-4d2c-b52f-72165dbc2cf8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2m2f6" May 29 16:56:51.078652 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.076720 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-etc-kubernetes\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:51.078652 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.076744 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-host-var-lib-kubelet\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:51.078652 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.076745 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mxxng\" (UniqueName: \"kubernetes.io/projected/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-kube-api-access-mxxng\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:51.078652 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.076783 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gwj6s\" (UniqueName: \"kubernetes.io/projected/56335f96-b27f-4a52-beb7-656001cdea57-kube-api-access-gwj6s\") pod \"multus-additional-cni-plugins-52zq6\" (UID: \"56335f96-b27f-4a52-beb7-656001cdea57\") " pod="openshift-multus/multus-additional-cni-plugins-52zq6" May 29 16:56:51.078652 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.076810 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f6995bff-b11d-42e1-87ea-ecc131cd8810-ovnkube-script-lib\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:51.078652 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.076882 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-etc-kubernetes\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:51.078652 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.076890 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f153a7a3-935e-44ac-8a76-d3ab6333809a-lib-modules\") pod \"tuned-9nn7d\" (UID: \"f153a7a3-935e-44ac-8a76-d3ab6333809a\") " pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:51.078652 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.076883 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-cni-binary-copy\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:51.078652 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.076913 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-multus-daemon-config\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:51.078652 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.076962 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-host-run-multus-certs\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:51.078652 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.076987 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f153a7a3-935e-44ac-8a76-d3ab6333809a-etc-kubernetes\") pod \"tuned-9nn7d\" (UID: \"f153a7a3-935e-44ac-8a76-d3ab6333809a\") " pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:51.078652 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077013 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5z6bc\" (UniqueName: \"kubernetes.io/projected/f153a7a3-935e-44ac-8a76-d3ab6333809a-kube-api-access-5z6bc\") pod \"tuned-9nn7d\" (UID: \"f153a7a3-935e-44ac-8a76-d3ab6333809a\") " pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:51.078652 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077016 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/56335f96-b27f-4a52-beb7-656001cdea57-cni-binary-copy\") pod \"multus-additional-cni-plugins-52zq6\" (UID: \"56335f96-b27f-4a52-beb7-656001cdea57\") " pod="openshift-multus/multus-additional-cni-plugins-52zq6" May 29 16:56:51.078652 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077039 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/46b01332-3570-4d2c-b52f-72165dbc2cf8-etc-selinux\") pod \"aws-ebs-csi-driver-node-2m2f6\" (UID: \"46b01332-3570-4d2c-b52f-72165dbc2cf8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2m2f6" May 29 16:56:51.078652 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077094 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f153a7a3-935e-44ac-8a76-d3ab6333809a-etc-sysconfig\") pod \"tuned-9nn7d\" (UID: \"f153a7a3-935e-44ac-8a76-d3ab6333809a\") " pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:51.078652 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077098 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-multus-socket-dir-parent\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:51.079164 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077142 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-host-run-multus-certs\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:51.079164 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077142 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-host-var-lib-cni-multus\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:51.079164 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077173 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/54c6a295-b9b0-4cda-9a11-33a3b53a7dfb-konnectivity-ca\") pod \"konnectivity-agent-w9jdd\" (UID: \"54c6a295-b9b0-4cda-9a11-33a3b53a7dfb\") " pod="kube-system/konnectivity-agent-w9jdd" May 29 16:56:51.079164 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077184 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/46b01332-3570-4d2c-b52f-72165dbc2cf8-socket-dir\") pod \"aws-ebs-csi-driver-node-2m2f6\" (UID: \"46b01332-3570-4d2c-b52f-72165dbc2cf8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2m2f6" May 29 16:56:51.079164 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077221 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/56335f96-b27f-4a52-beb7-656001cdea57-cnibin\") pod \"multus-additional-cni-plugins-52zq6\" (UID: \"56335f96-b27f-4a52-beb7-656001cdea57\") " pod="openshift-multus/multus-additional-cni-plugins-52zq6" May 29 16:56:51.079164 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077262 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f153a7a3-935e-44ac-8a76-d3ab6333809a-var-lib-kubelet\") pod \"tuned-9nn7d\" (UID: \"f153a7a3-935e-44ac-8a76-d3ab6333809a\") " pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:51.079164 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077287 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-log-socket\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:51.079164 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077294 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f153a7a3-935e-44ac-8a76-d3ab6333809a-lib-modules\") pod \"tuned-9nn7d\" (UID: \"f153a7a3-935e-44ac-8a76-d3ab6333809a\") " pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:51.079164 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077330 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f6995bff-b11d-42e1-87ea-ecc131cd8810-ovn-node-metrics-cert\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:51.079164 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077344 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-host-var-lib-cni-multus\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:51.079164 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077362 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/962b0454-8569-4239-b4b3-b88b0fe86570-serviceca\") pod \"node-ca-qdvbn\" (UID: \"962b0454-8569-4239-b4b3-b88b0fe86570\") " pod="openshift-image-registry/node-ca-qdvbn" May 29 16:56:51.079164 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077389 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zkgng\" (UniqueName: \"kubernetes.io/projected/7d457970-4377-4157-9d77-81db07916925-kube-api-access-zkgng\") pod \"network-metrics-daemon-ps265\" (UID: \"7d457970-4377-4157-9d77-81db07916925\") " pod="openshift-multus/network-metrics-daemon-ps265" May 29 16:56:51.079164 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077416 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-host-run-ovn-kubernetes\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:51.079164 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077415 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f153a7a3-935e-44ac-8a76-d3ab6333809a-etc-modprobe-d\") pod \"tuned-9nn7d\" (UID: \"f153a7a3-935e-44ac-8a76-d3ab6333809a\") " pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:51.079164 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077435 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f153a7a3-935e-44ac-8a76-d3ab6333809a-etc-sysconfig\") pod \"tuned-9nn7d\" (UID: \"f153a7a3-935e-44ac-8a76-d3ab6333809a\") " pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:51.079164 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077460 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f153a7a3-935e-44ac-8a76-d3ab6333809a-etc-tuned\") pod \"tuned-9nn7d\" (UID: \"f153a7a3-935e-44ac-8a76-d3ab6333809a\") " pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:51.079164 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077471 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/46b01332-3570-4d2c-b52f-72165dbc2cf8-etc-selinux\") pod \"aws-ebs-csi-driver-node-2m2f6\" (UID: \"46b01332-3570-4d2c-b52f-72165dbc2cf8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2m2f6" May 29 16:56:51.079671 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077488 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hn8kk\" (UniqueName: \"kubernetes.io/projected/962b0454-8569-4239-b4b3-b88b0fe86570-kube-api-access-hn8kk\") pod \"node-ca-qdvbn\" (UID: \"962b0454-8569-4239-b4b3-b88b0fe86570\") " pod="openshift-image-registry/node-ca-qdvbn" May 29 16:56:51.079671 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077505 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f153a7a3-935e-44ac-8a76-d3ab6333809a-etc-modprobe-d\") pod \"tuned-9nn7d\" (UID: \"f153a7a3-935e-44ac-8a76-d3ab6333809a\") " pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:51.079671 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077514 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e270884a-3a8f-4299-9cfa-a44e2bf1768f-iptables-alerter-script\") pod \"iptables-alerter-wt655\" (UID: \"e270884a-3a8f-4299-9cfa-a44e2bf1768f\") " pod="openshift-network-operator/iptables-alerter-wt655" May 29 16:56:51.079671 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077543 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-systemd-units\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:51.079671 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077548 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/46b01332-3570-4d2c-b52f-72165dbc2cf8-sys-fs\") pod \"aws-ebs-csi-driver-node-2m2f6\" (UID: \"46b01332-3570-4d2c-b52f-72165dbc2cf8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2m2f6" May 29 16:56:51.079671 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077573 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-host-run-netns\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:51.079671 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077598 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wjg4m\" (UniqueName: \"kubernetes.io/projected/f6995bff-b11d-42e1-87ea-ecc131cd8810-kube-api-access-wjg4m\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:51.079671 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077610 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/46b01332-3570-4d2c-b52f-72165dbc2cf8-socket-dir\") pod \"aws-ebs-csi-driver-node-2m2f6\" (UID: \"46b01332-3570-4d2c-b52f-72165dbc2cf8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2m2f6" May 29 16:56:51.079671 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077622 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f153a7a3-935e-44ac-8a76-d3ab6333809a-host\") pod \"tuned-9nn7d\" (UID: \"f153a7a3-935e-44ac-8a76-d3ab6333809a\") " pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:51.079671 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077645 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-cnibin\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:51.079671 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077650 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f6995bff-b11d-42e1-87ea-ecc131cd8810-ovnkube-script-lib\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:51.079671 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077670 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-host-run-netns\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:51.079671 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077682 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f153a7a3-935e-44ac-8a76-d3ab6333809a-var-lib-kubelet\") pod \"tuned-9nn7d\" (UID: \"f153a7a3-935e-44ac-8a76-d3ab6333809a\") " pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:51.079671 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077696 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-system-cni-dir\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:51.079671 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077699 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-systemd-units\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:51.079671 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077715 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f153a7a3-935e-44ac-8a76-d3ab6333809a-host\") pod \"tuned-9nn7d\" (UID: \"f153a7a3-935e-44ac-8a76-d3ab6333809a\") " pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:51.079671 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077723 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-multus-conf-dir\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:51.079671 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077752 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/46b01332-3570-4d2c-b52f-72165dbc2cf8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2m2f6\" (UID: \"46b01332-3570-4d2c-b52f-72165dbc2cf8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2m2f6" May 29 16:56:51.080232 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077834 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/46b01332-3570-4d2c-b52f-72165dbc2cf8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2m2f6\" (UID: \"46b01332-3570-4d2c-b52f-72165dbc2cf8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2m2f6" May 29 16:56:51.080232 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077904 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-cnibin\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:51.080232 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077938 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-host-run-netns\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:51.080232 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077669 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-host-run-netns\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:51.080232 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077990 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-system-cni-dir\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:51.080232 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.078027 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-multus-conf-dir\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:51.080232 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077727 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f6995bff-b11d-42e1-87ea-ecc131cd8810-log-socket\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:51.080232 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077391 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f153a7a3-935e-44ac-8a76-d3ab6333809a-etc-kubernetes\") pod \"tuned-9nn7d\" (UID: \"f153a7a3-935e-44ac-8a76-d3ab6333809a\") " pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:51.080232 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.078157 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-multus-daemon-config\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:51.080232 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.077507 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/56335f96-b27f-4a52-beb7-656001cdea57-cnibin\") pod \"multus-additional-cni-plugins-52zq6\" (UID: \"56335f96-b27f-4a52-beb7-656001cdea57\") " pod="openshift-multus/multus-additional-cni-plugins-52zq6" May 29 16:56:51.080232 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.078534 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e270884a-3a8f-4299-9cfa-a44e2bf1768f-iptables-alerter-script\") pod \"iptables-alerter-wt655\" (UID: \"e270884a-3a8f-4299-9cfa-a44e2bf1768f\") " pod="openshift-network-operator/iptables-alerter-wt655" May 29 16:56:51.080232 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.079005 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f153a7a3-935e-44ac-8a76-d3ab6333809a-tmp\") pod \"tuned-9nn7d\" (UID: \"f153a7a3-935e-44ac-8a76-d3ab6333809a\") " pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:51.080232 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.079433 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/54c6a295-b9b0-4cda-9a11-33a3b53a7dfb-agent-certs\") pod \"konnectivity-agent-w9jdd\" (UID: \"54c6a295-b9b0-4cda-9a11-33a3b53a7dfb\") " pod="kube-system/konnectivity-agent-w9jdd" May 29 16:56:51.080232 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.079622 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f153a7a3-935e-44ac-8a76-d3ab6333809a-etc-tuned\") pod \"tuned-9nn7d\" (UID: \"f153a7a3-935e-44ac-8a76-d3ab6333809a\") " pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:51.080232 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.079767 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f6995bff-b11d-42e1-87ea-ecc131cd8810-ovn-node-metrics-cert\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:51.082750 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.082714 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-197.ec2.internal" event={"ID":"dfaf61f08c1074bed45e475649697dac","Type":"ContainerStarted","Data":"0d5d3752d73591c0e3ac160b71de844158b9b98a1096ff1311c260d9581195df"} May 29 16:56:51.083619 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.083600 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-197.ec2.internal" event={"ID":"62bc43ff86f9d08efd1447f9d66d955f","Type":"ContainerStarted","Data":"52973df01d34749d420bbaf983ff6a812d9e6a9ce6ce8f3a84191e2bbacd687d"} May 29 16:56:51.088680 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:51.087482 2549 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 29 16:56:51.088680 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:51.087507 2549 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 29 16:56:51.088680 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:51.087520 2549 projected.go:194] Error preparing data for projected volume kube-api-access-b6b9d for pod openshift-network-diagnostics/network-check-target-crvwv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 29 16:56:51.088680 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:51.087590 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be92fdf2-3d2b-4075-8919-8697586347c6-kube-api-access-b6b9d podName:be92fdf2-3d2b-4075-8919-8697586347c6 nodeName:}" failed. No retries permitted until 2026-05-29 16:56:51.587574216 +0000 UTC m=+2.074003412 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-b6b9d" (UniqueName: "kubernetes.io/projected/be92fdf2-3d2b-4075-8919-8697586347c6-kube-api-access-b6b9d") pod "network-check-target-crvwv" (UID: "be92fdf2-3d2b-4075-8919-8697586347c6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 29 16:56:51.089947 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.089696 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdv6p\" (UniqueName: \"kubernetes.io/projected/e270884a-3a8f-4299-9cfa-a44e2bf1768f-kube-api-access-rdv6p\") pod \"iptables-alerter-wt655\" (UID: \"e270884a-3a8f-4299-9cfa-a44e2bf1768f\") " pod="openshift-network-operator/iptables-alerter-wt655" May 29 16:56:51.089947 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.089710 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hn8kk\" (UniqueName: \"kubernetes.io/projected/962b0454-8569-4239-b4b3-b88b0fe86570-kube-api-access-hn8kk\") pod \"node-ca-qdvbn\" (UID: \"962b0454-8569-4239-b4b3-b88b0fe86570\") " pod="openshift-image-registry/node-ca-qdvbn" May 29 16:56:51.092347 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.092163 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnllv\" (UniqueName: \"kubernetes.io/projected/46b01332-3570-4d2c-b52f-72165dbc2cf8-kube-api-access-dnllv\") pod \"aws-ebs-csi-driver-node-2m2f6\" (UID: \"46b01332-3570-4d2c-b52f-72165dbc2cf8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2m2f6" May 29 16:56:51.092347 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.092179 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxxng\" (UniqueName: \"kubernetes.io/projected/e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101-kube-api-access-mxxng\") pod \"multus-4fwqn\" (UID: \"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101\") " pod="openshift-multus/multus-4fwqn" May 29 16:56:51.092347 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.092222 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjg4m\" (UniqueName: \"kubernetes.io/projected/f6995bff-b11d-42e1-87ea-ecc131cd8810-kube-api-access-wjg4m\") pod \"ovnkube-node-nfv6s\" (UID: \"f6995bff-b11d-42e1-87ea-ecc131cd8810\") " pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:51.092854 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.092820 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5z6bc\" (UniqueName: \"kubernetes.io/projected/f153a7a3-935e-44ac-8a76-d3ab6333809a-kube-api-access-5z6bc\") pod \"tuned-9nn7d\" (UID: \"f153a7a3-935e-44ac-8a76-d3ab6333809a\") " pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:51.093255 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.093239 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkgng\" (UniqueName: \"kubernetes.io/projected/7d457970-4377-4157-9d77-81db07916925-kube-api-access-zkgng\") pod \"network-metrics-daemon-ps265\" (UID: \"7d457970-4377-4157-9d77-81db07916925\") " pod="openshift-multus/network-metrics-daemon-ps265" May 29 16:56:51.093485 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.093469 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwj6s\" (UniqueName: \"kubernetes.io/projected/56335f96-b27f-4a52-beb7-656001cdea57-kube-api-access-gwj6s\") pod \"multus-additional-cni-plugins-52zq6\" (UID: \"56335f96-b27f-4a52-beb7-656001cdea57\") " pod="openshift-multus/multus-additional-cni-plugins-52zq6" May 29 16:56:51.284678 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.284651 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:56:51.292325 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.292303 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2m2f6" May 29 16:56:51.295027 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:51.295004 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6995bff_b11d_42e1_87ea_ecc131cd8810.slice/crio-612fb045df533a6f63f6303425cafefdce2d664000e382ac96454f033772b643 WatchSource:0}: Error finding container 612fb045df533a6f63f6303425cafefdce2d664000e382ac96454f033772b643: Status 404 returned error can't find the container with id 612fb045df533a6f63f6303425cafefdce2d664000e382ac96454f033772b643 May 29 16:56:51.299417 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:51.299398 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46b01332_3570_4d2c_b52f_72165dbc2cf8.slice/crio-dac069d79e17ba87f83f5096b82250d60d9a9b9681b4296bd6cb28e2be8af59f WatchSource:0}: Error finding container dac069d79e17ba87f83f5096b82250d60d9a9b9681b4296bd6cb28e2be8af59f: Status 404 returned error can't find the container with id dac069d79e17ba87f83f5096b82250d60d9a9b9681b4296bd6cb28e2be8af59f May 29 16:56:51.300269 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.300252 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wt655" May 29 16:56:51.304282 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.304267 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-w9jdd" May 29 16:56:51.306577 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:51.306172 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode270884a_3a8f_4299_9cfa_a44e2bf1768f.slice/crio-783525e359610256ff9aa55a9c234703dd4f6fe9f8cf2c0b0e23bfcac465913d WatchSource:0}: Error finding container 783525e359610256ff9aa55a9c234703dd4f6fe9f8cf2c0b0e23bfcac465913d: Status 404 returned error can't find the container with id 783525e359610256ff9aa55a9c234703dd4f6fe9f8cf2c0b0e23bfcac465913d May 29 16:56:51.310048 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.310031 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" May 29 16:56:51.311573 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:51.311554 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54c6a295_b9b0_4cda_9a11_33a3b53a7dfb.slice/crio-a2acab5ce6e65df419919b18d09f13aedec52379e9a2febfdbc50bc767607fc8 WatchSource:0}: Error finding container a2acab5ce6e65df419919b18d09f13aedec52379e9a2febfdbc50bc767607fc8: Status 404 returned error can't find the container with id a2acab5ce6e65df419919b18d09f13aedec52379e9a2febfdbc50bc767607fc8 May 29 16:56:51.315563 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.315546 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qdvbn" May 29 16:56:51.315817 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:51.315795 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf153a7a3_935e_44ac_8a76_d3ab6333809a.slice/crio-795a23b6846fd1678fc7b7506619c99bea89403a7548616c10093ffd650bd768 WatchSource:0}: Error finding container 795a23b6846fd1678fc7b7506619c99bea89403a7548616c10093ffd650bd768: Status 404 returned error can't find the container with id 795a23b6846fd1678fc7b7506619c99bea89403a7548616c10093ffd650bd768 May 29 16:56:51.320669 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.320650 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4fwqn" May 29 16:56:51.320994 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:51.320958 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod962b0454_8569_4239_b4b3_b88b0fe86570.slice/crio-206791554a528d3a521b76451f76640d194948515c1016862cf0be282df3a5ca WatchSource:0}: Error finding container 206791554a528d3a521b76451f76640d194948515c1016862cf0be282df3a5ca: Status 404 returned error can't find the container with id 206791554a528d3a521b76451f76640d194948515c1016862cf0be282df3a5ca May 29 16:56:51.325655 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.325529 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-52zq6" May 29 16:56:51.327549 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:51.327239 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7bc2100_fd8c_4fe2_8ad5_3e9a3fe25101.slice/crio-63fd88fd1716cf1f95a88690e1902b67c93781718764715e9c227fdc94211354 WatchSource:0}: Error finding container 63fd88fd1716cf1f95a88690e1902b67c93781718764715e9c227fdc94211354: Status 404 returned error can't find the container with id 63fd88fd1716cf1f95a88690e1902b67c93781718764715e9c227fdc94211354 May 29 16:56:51.332927 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:56:51.332892 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56335f96_b27f_4a52_beb7_656001cdea57.slice/crio-bcf3cbead7f543ee27ce020a59779c5b61c1aeb70ddf59967f0b7b62d37b22cc WatchSource:0}: Error finding container bcf3cbead7f543ee27ce020a59779c5b61c1aeb70ddf59967f0b7b62d37b22cc: Status 404 returned error can't find the container with id bcf3cbead7f543ee27ce020a59779c5b61c1aeb70ddf59967f0b7b62d37b22cc May 29 16:56:51.582150 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.582109 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d457970-4377-4157-9d77-81db07916925-metrics-certs\") pod \"network-metrics-daemon-ps265\" (UID: \"7d457970-4377-4157-9d77-81db07916925\") " pod="openshift-multus/network-metrics-daemon-ps265" May 29 16:56:51.582327 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:51.582272 2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 29 16:56:51.582390 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:51.582334 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d457970-4377-4157-9d77-81db07916925-metrics-certs podName:7d457970-4377-4157-9d77-81db07916925 nodeName:}" failed. No retries permitted until 2026-05-29 16:56:52.582315789 +0000 UTC m=+3.068744982 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7d457970-4377-4157-9d77-81db07916925-metrics-certs") pod "network-metrics-daemon-ps265" (UID: "7d457970-4377-4157-9d77-81db07916925") : object "openshift-multus"/"metrics-daemon-secret" not registered May 29 16:56:51.683860 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.683189 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b6b9d\" (UniqueName: \"kubernetes.io/projected/be92fdf2-3d2b-4075-8919-8697586347c6-kube-api-access-b6b9d\") pod \"network-check-target-crvwv\" (UID: \"be92fdf2-3d2b-4075-8919-8697586347c6\") " pod="openshift-network-diagnostics/network-check-target-crvwv" May 29 16:56:51.683860 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:51.683358 2549 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 29 16:56:51.683860 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:51.683376 2549 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 29 16:56:51.683860 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:51.683389 2549 projected.go:194] Error preparing data for projected volume kube-api-access-b6b9d for pod openshift-network-diagnostics/network-check-target-crvwv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 29 16:56:51.683860 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:51.683447 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be92fdf2-3d2b-4075-8919-8697586347c6-kube-api-access-b6b9d podName:be92fdf2-3d2b-4075-8919-8697586347c6 nodeName:}" failed. No retries permitted until 2026-05-29 16:56:52.683430123 +0000 UTC m=+3.169859316 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-b6b9d" (UniqueName: "kubernetes.io/projected/be92fdf2-3d2b-4075-8919-8697586347c6-kube-api-access-b6b9d") pod "network-check-target-crvwv" (UID: "be92fdf2-3d2b-4075-8919-8697586347c6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 29 16:56:51.810455 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.810425 2549 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" May 29 16:56:51.842149 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:51.841877 2549 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" May 29 16:56:52.007699 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:52.007577 2549 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-28 16:51:50 +0000 UTC" deadline="2028-02-03 07:45:06.432293807 +0000 UTC" May 29 16:56:52.007699 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:52.007609 2549 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14750h48m14.424688765s" May 29 16:56:52.102995 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:52.102954 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wt655" event={"ID":"e270884a-3a8f-4299-9cfa-a44e2bf1768f","Type":"ContainerStarted","Data":"783525e359610256ff9aa55a9c234703dd4f6fe9f8cf2c0b0e23bfcac465913d"} May 29 16:56:52.107296 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:52.107246 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2m2f6" event={"ID":"46b01332-3570-4d2c-b52f-72165dbc2cf8","Type":"ContainerStarted","Data":"dac069d79e17ba87f83f5096b82250d60d9a9b9681b4296bd6cb28e2be8af59f"} May 29 16:56:52.136504 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:52.136467 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4fwqn" event={"ID":"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101","Type":"ContainerStarted","Data":"63fd88fd1716cf1f95a88690e1902b67c93781718764715e9c227fdc94211354"} May 29 16:56:52.152491 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:52.152420 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qdvbn" event={"ID":"962b0454-8569-4239-b4b3-b88b0fe86570","Type":"ContainerStarted","Data":"206791554a528d3a521b76451f76640d194948515c1016862cf0be282df3a5ca"} May 29 16:56:52.163435 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:52.163361 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" event={"ID":"f153a7a3-935e-44ac-8a76-d3ab6333809a","Type":"ContainerStarted","Data":"795a23b6846fd1678fc7b7506619c99bea89403a7548616c10093ffd650bd768"} May 29 16:56:52.168858 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:52.168814 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" event={"ID":"f6995bff-b11d-42e1-87ea-ecc131cd8810","Type":"ContainerStarted","Data":"612fb045df533a6f63f6303425cafefdce2d664000e382ac96454f033772b643"} May 29 16:56:52.172311 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:52.172258 2549 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" May 29 16:56:52.172894 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:52.172872 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-52zq6" event={"ID":"56335f96-b27f-4a52-beb7-656001cdea57","Type":"ContainerStarted","Data":"bcf3cbead7f543ee27ce020a59779c5b61c1aeb70ddf59967f0b7b62d37b22cc"} May 29 16:56:52.177159 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:52.177135 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-w9jdd" event={"ID":"54c6a295-b9b0-4cda-9a11-33a3b53a7dfb","Type":"ContainerStarted","Data":"a2acab5ce6e65df419919b18d09f13aedec52379e9a2febfdbc50bc767607fc8"} May 29 16:56:52.591282 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:52.591247 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d457970-4377-4157-9d77-81db07916925-metrics-certs\") pod \"network-metrics-daemon-ps265\" (UID: \"7d457970-4377-4157-9d77-81db07916925\") " pod="openshift-multus/network-metrics-daemon-ps265" May 29 16:56:52.591459 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:52.591424 2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 29 16:56:52.591532 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:52.591486 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d457970-4377-4157-9d77-81db07916925-metrics-certs podName:7d457970-4377-4157-9d77-81db07916925 nodeName:}" failed. No retries permitted until 2026-05-29 16:56:54.591468412 +0000 UTC m=+5.077897596 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7d457970-4377-4157-9d77-81db07916925-metrics-certs") pod "network-metrics-daemon-ps265" (UID: "7d457970-4377-4157-9d77-81db07916925") : object "openshift-multus"/"metrics-daemon-secret" not registered May 29 16:56:52.693137 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:52.692568 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b6b9d\" (UniqueName: \"kubernetes.io/projected/be92fdf2-3d2b-4075-8919-8697586347c6-kube-api-access-b6b9d\") pod \"network-check-target-crvwv\" (UID: \"be92fdf2-3d2b-4075-8919-8697586347c6\") " pod="openshift-network-diagnostics/network-check-target-crvwv" May 29 16:56:52.693137 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:52.692768 2549 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 29 16:56:52.693137 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:52.692787 2549 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 29 16:56:52.693137 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:52.692800 2549 projected.go:194] Error preparing data for projected volume kube-api-access-b6b9d for pod openshift-network-diagnostics/network-check-target-crvwv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 29 16:56:52.693137 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:52.692868 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be92fdf2-3d2b-4075-8919-8697586347c6-kube-api-access-b6b9d podName:be92fdf2-3d2b-4075-8919-8697586347c6 nodeName:}" failed. No retries permitted until 2026-05-29 16:56:54.692837417 +0000 UTC m=+5.179266607 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-b6b9d" (UniqueName: "kubernetes.io/projected/be92fdf2-3d2b-4075-8919-8697586347c6-kube-api-access-b6b9d") pod "network-check-target-crvwv" (UID: "be92fdf2-3d2b-4075-8919-8697586347c6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 29 16:56:53.008186 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:53.007951 2549 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-28 16:51:50 +0000 UTC" deadline="2028-01-05 20:57:04.048086951 +0000 UTC" May 29 16:56:53.008186 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:53.007993 2549 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14068h0m11.040097558s" May 29 16:56:53.080766 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:53.080733 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-crvwv" May 29 16:56:53.080957 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:53.080750 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ps265" May 29 16:56:53.080957 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:53.080886 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-crvwv" podUID="be92fdf2-3d2b-4075-8919-8697586347c6" May 29 16:56:53.081077 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:53.080955 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ps265" podUID="7d457970-4377-4157-9d77-81db07916925" May 29 16:56:54.606632 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:54.606592 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d457970-4377-4157-9d77-81db07916925-metrics-certs\") pod \"network-metrics-daemon-ps265\" (UID: \"7d457970-4377-4157-9d77-81db07916925\") " pod="openshift-multus/network-metrics-daemon-ps265" May 29 16:56:54.607107 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:54.606761 2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 29 16:56:54.607107 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:54.606857 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d457970-4377-4157-9d77-81db07916925-metrics-certs podName:7d457970-4377-4157-9d77-81db07916925 nodeName:}" failed. No retries permitted until 2026-05-29 16:56:58.606819615 +0000 UTC m=+9.093248810 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7d457970-4377-4157-9d77-81db07916925-metrics-certs") pod "network-metrics-daemon-ps265" (UID: "7d457970-4377-4157-9d77-81db07916925") : object "openshift-multus"/"metrics-daemon-secret" not registered May 29 16:56:54.707903 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:54.707864 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b6b9d\" (UniqueName: \"kubernetes.io/projected/be92fdf2-3d2b-4075-8919-8697586347c6-kube-api-access-b6b9d\") pod \"network-check-target-crvwv\" (UID: \"be92fdf2-3d2b-4075-8919-8697586347c6\") " pod="openshift-network-diagnostics/network-check-target-crvwv" May 29 16:56:54.708077 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:54.708021 2549 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 29 16:56:54.708077 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:54.708042 2549 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 29 16:56:54.708077 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:54.708053 2549 projected.go:194] Error preparing data for projected volume kube-api-access-b6b9d for pod openshift-network-diagnostics/network-check-target-crvwv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 29 16:56:54.708236 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:54.708117 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be92fdf2-3d2b-4075-8919-8697586347c6-kube-api-access-b6b9d podName:be92fdf2-3d2b-4075-8919-8697586347c6 nodeName:}" failed. No retries permitted until 2026-05-29 16:56:58.708098875 +0000 UTC m=+9.194528057 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-b6b9d" (UniqueName: "kubernetes.io/projected/be92fdf2-3d2b-4075-8919-8697586347c6-kube-api-access-b6b9d") pod "network-check-target-crvwv" (UID: "be92fdf2-3d2b-4075-8919-8697586347c6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 29 16:56:55.080137 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:55.080082 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ps265" May 29 16:56:55.080323 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:55.080215 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ps265" podUID="7d457970-4377-4157-9d77-81db07916925" May 29 16:56:55.080679 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:55.080082 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-crvwv" May 29 16:56:55.080679 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:55.080637 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-crvwv" podUID="be92fdf2-3d2b-4075-8919-8697586347c6" May 29 16:56:57.080031 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:57.079991 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-crvwv" May 29 16:56:57.080512 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:57.080138 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-crvwv" podUID="be92fdf2-3d2b-4075-8919-8697586347c6" May 29 16:56:57.080512 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:57.080193 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ps265" May 29 16:56:57.080512 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:57.080318 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ps265" podUID="7d457970-4377-4157-9d77-81db07916925" May 29 16:56:58.638066 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:58.638025 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d457970-4377-4157-9d77-81db07916925-metrics-certs\") pod \"network-metrics-daemon-ps265\" (UID: \"7d457970-4377-4157-9d77-81db07916925\") " pod="openshift-multus/network-metrics-daemon-ps265" May 29 16:56:58.638520 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:58.638167 2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 29 16:56:58.638520 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:58.638234 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d457970-4377-4157-9d77-81db07916925-metrics-certs podName:7d457970-4377-4157-9d77-81db07916925 nodeName:}" failed. No retries permitted until 2026-05-29 16:57:06.638213837 +0000 UTC m=+17.124643019 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7d457970-4377-4157-9d77-81db07916925-metrics-certs") pod "network-metrics-daemon-ps265" (UID: "7d457970-4377-4157-9d77-81db07916925") : object "openshift-multus"/"metrics-daemon-secret" not registered May 29 16:56:58.738469 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:58.738435 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b6b9d\" (UniqueName: \"kubernetes.io/projected/be92fdf2-3d2b-4075-8919-8697586347c6-kube-api-access-b6b9d\") pod \"network-check-target-crvwv\" (UID: \"be92fdf2-3d2b-4075-8919-8697586347c6\") " pod="openshift-network-diagnostics/network-check-target-crvwv" May 29 16:56:58.738639 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:58.738616 2549 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 29 16:56:58.738639 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:58.738634 2549 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 29 16:56:58.738761 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:58.738646 2549 projected.go:194] Error preparing data for projected volume kube-api-access-b6b9d for pod openshift-network-diagnostics/network-check-target-crvwv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 29 16:56:58.738761 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:58.738709 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be92fdf2-3d2b-4075-8919-8697586347c6-kube-api-access-b6b9d podName:be92fdf2-3d2b-4075-8919-8697586347c6 nodeName:}" failed. No retries permitted until 2026-05-29 16:57:06.73868993 +0000 UTC m=+17.225119111 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-b6b9d" (UniqueName: "kubernetes.io/projected/be92fdf2-3d2b-4075-8919-8697586347c6-kube-api-access-b6b9d") pod "network-check-target-crvwv" (UID: "be92fdf2-3d2b-4075-8919-8697586347c6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 29 16:56:59.079895 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:59.079856 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ps265" May 29 16:56:59.080062 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:56:59.079870 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-crvwv" May 29 16:56:59.080062 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:59.080019 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ps265" podUID="7d457970-4377-4157-9d77-81db07916925" May 29 16:56:59.080188 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:56:59.080106 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-crvwv" podUID="be92fdf2-3d2b-4075-8919-8697586347c6" May 29 16:57:01.080356 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:01.080321 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ps265" May 29 16:57:01.080760 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:01.080321 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-crvwv" May 29 16:57:01.080760 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:01.080444 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ps265" podUID="7d457970-4377-4157-9d77-81db07916925" May 29 16:57:01.080760 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:01.080546 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-crvwv" podUID="be92fdf2-3d2b-4075-8919-8697586347c6" May 29 16:57:01.506649 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:01.506607 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-dhvms"] May 29 16:57:01.508352 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:01.508320 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dhvms" May 29 16:57:01.508468 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:01.508402 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dhvms" podUID="eb980098-9793-4340-b57d-b0600c17b07b" May 29 16:57:01.558293 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:01.558255 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/eb980098-9793-4340-b57d-b0600c17b07b-dbus\") pod \"global-pull-secret-syncer-dhvms\" (UID: \"eb980098-9793-4340-b57d-b0600c17b07b\") " pod="kube-system/global-pull-secret-syncer-dhvms" May 29 16:57:01.558458 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:01.558300 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eb980098-9793-4340-b57d-b0600c17b07b-original-pull-secret\") pod \"global-pull-secret-syncer-dhvms\" (UID: \"eb980098-9793-4340-b57d-b0600c17b07b\") " pod="kube-system/global-pull-secret-syncer-dhvms" May 29 16:57:01.558458 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:01.558348 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/eb980098-9793-4340-b57d-b0600c17b07b-kubelet-config\") pod \"global-pull-secret-syncer-dhvms\" (UID: \"eb980098-9793-4340-b57d-b0600c17b07b\") " pod="kube-system/global-pull-secret-syncer-dhvms" May 29 16:57:01.659170 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:01.659079 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/eb980098-9793-4340-b57d-b0600c17b07b-dbus\") pod \"global-pull-secret-syncer-dhvms\" (UID: \"eb980098-9793-4340-b57d-b0600c17b07b\") " pod="kube-system/global-pull-secret-syncer-dhvms" May 29 16:57:01.659170 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:01.659131 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eb980098-9793-4340-b57d-b0600c17b07b-original-pull-secret\") pod \"global-pull-secret-syncer-dhvms\" (UID: \"eb980098-9793-4340-b57d-b0600c17b07b\") " pod="kube-system/global-pull-secret-syncer-dhvms" May 29 16:57:01.659170 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:01.659170 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/eb980098-9793-4340-b57d-b0600c17b07b-kubelet-config\") pod \"global-pull-secret-syncer-dhvms\" (UID: \"eb980098-9793-4340-b57d-b0600c17b07b\") " pod="kube-system/global-pull-secret-syncer-dhvms" May 29 16:57:01.659422 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:01.659288 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/eb980098-9793-4340-b57d-b0600c17b07b-kubelet-config\") pod \"global-pull-secret-syncer-dhvms\" (UID: \"eb980098-9793-4340-b57d-b0600c17b07b\") " pod="kube-system/global-pull-secret-syncer-dhvms" May 29 16:57:01.659422 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:01.659288 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/eb980098-9793-4340-b57d-b0600c17b07b-dbus\") pod \"global-pull-secret-syncer-dhvms\" (UID: \"eb980098-9793-4340-b57d-b0600c17b07b\") " pod="kube-system/global-pull-secret-syncer-dhvms" May 29 16:57:01.659422 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:01.659340 2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 29 16:57:01.659554 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:01.659436 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb980098-9793-4340-b57d-b0600c17b07b-original-pull-secret podName:eb980098-9793-4340-b57d-b0600c17b07b nodeName:}" failed. No retries permitted until 2026-05-29 16:57:02.159414827 +0000 UTC m=+12.645844031 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/eb980098-9793-4340-b57d-b0600c17b07b-original-pull-secret") pod "global-pull-secret-syncer-dhvms" (UID: "eb980098-9793-4340-b57d-b0600c17b07b") : object "kube-system"/"original-pull-secret" not registered May 29 16:57:02.162353 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:02.162319 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eb980098-9793-4340-b57d-b0600c17b07b-original-pull-secret\") pod \"global-pull-secret-syncer-dhvms\" (UID: \"eb980098-9793-4340-b57d-b0600c17b07b\") " pod="kube-system/global-pull-secret-syncer-dhvms" May 29 16:57:02.162785 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:02.162466 2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 29 16:57:02.162785 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:02.162533 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb980098-9793-4340-b57d-b0600c17b07b-original-pull-secret podName:eb980098-9793-4340-b57d-b0600c17b07b nodeName:}" failed. No retries permitted until 2026-05-29 16:57:03.162513764 +0000 UTC m=+13.648942948 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/eb980098-9793-4340-b57d-b0600c17b07b-original-pull-secret") pod "global-pull-secret-syncer-dhvms" (UID: "eb980098-9793-4340-b57d-b0600c17b07b") : object "kube-system"/"original-pull-secret" not registered May 29 16:57:03.080169 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:03.080125 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dhvms" May 29 16:57:03.080169 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:03.080132 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-crvwv" May 29 16:57:03.080375 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:03.080129 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ps265" May 29 16:57:03.080375 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:03.080246 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dhvms" podUID="eb980098-9793-4340-b57d-b0600c17b07b" May 29 16:57:03.080375 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:03.080298 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-crvwv" podUID="be92fdf2-3d2b-4075-8919-8697586347c6" May 29 16:57:03.080494 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:03.080394 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ps265" podUID="7d457970-4377-4157-9d77-81db07916925" May 29 16:57:03.169788 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:03.169746 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eb980098-9793-4340-b57d-b0600c17b07b-original-pull-secret\") pod \"global-pull-secret-syncer-dhvms\" (UID: \"eb980098-9793-4340-b57d-b0600c17b07b\") " pod="kube-system/global-pull-secret-syncer-dhvms" May 29 16:57:03.170175 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:03.169875 2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 29 16:57:03.170175 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:03.169942 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb980098-9793-4340-b57d-b0600c17b07b-original-pull-secret podName:eb980098-9793-4340-b57d-b0600c17b07b nodeName:}" failed. No retries permitted until 2026-05-29 16:57:05.169927335 +0000 UTC m=+15.656356518 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/eb980098-9793-4340-b57d-b0600c17b07b-original-pull-secret") pod "global-pull-secret-syncer-dhvms" (UID: "eb980098-9793-4340-b57d-b0600c17b07b") : object "kube-system"/"original-pull-secret" not registered May 29 16:57:05.079893 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:05.079856 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-crvwv" May 29 16:57:05.079893 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:05.079878 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ps265" May 29 16:57:05.080328 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:05.079866 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dhvms" May 29 16:57:05.080328 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:05.079987 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-crvwv" podUID="be92fdf2-3d2b-4075-8919-8697586347c6" May 29 16:57:05.080328 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:05.080075 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dhvms" podUID="eb980098-9793-4340-b57d-b0600c17b07b" May 29 16:57:05.080328 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:05.080203 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ps265" podUID="7d457970-4377-4157-9d77-81db07916925" May 29 16:57:05.185104 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:05.185024 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eb980098-9793-4340-b57d-b0600c17b07b-original-pull-secret\") pod \"global-pull-secret-syncer-dhvms\" (UID: \"eb980098-9793-4340-b57d-b0600c17b07b\") " pod="kube-system/global-pull-secret-syncer-dhvms" May 29 16:57:05.185269 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:05.185153 2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 29 16:57:05.185269 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:05.185223 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb980098-9793-4340-b57d-b0600c17b07b-original-pull-secret podName:eb980098-9793-4340-b57d-b0600c17b07b nodeName:}" failed. No retries permitted until 2026-05-29 16:57:09.185204964 +0000 UTC m=+19.671634164 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/eb980098-9793-4340-b57d-b0600c17b07b-original-pull-secret") pod "global-pull-secret-syncer-dhvms" (UID: "eb980098-9793-4340-b57d-b0600c17b07b") : object "kube-system"/"original-pull-secret" not registered May 29 16:57:06.694820 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:06.694785 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d457970-4377-4157-9d77-81db07916925-metrics-certs\") pod \"network-metrics-daemon-ps265\" (UID: \"7d457970-4377-4157-9d77-81db07916925\") " pod="openshift-multus/network-metrics-daemon-ps265" May 29 16:57:06.695286 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:06.694935 2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 29 16:57:06.695286 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:06.694996 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d457970-4377-4157-9d77-81db07916925-metrics-certs podName:7d457970-4377-4157-9d77-81db07916925 nodeName:}" failed. No retries permitted until 2026-05-29 16:57:22.694980284 +0000 UTC m=+33.181409485 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7d457970-4377-4157-9d77-81db07916925-metrics-certs") pod "network-metrics-daemon-ps265" (UID: "7d457970-4377-4157-9d77-81db07916925") : object "openshift-multus"/"metrics-daemon-secret" not registered May 29 16:57:06.795302 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:06.795257 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b6b9d\" (UniqueName: \"kubernetes.io/projected/be92fdf2-3d2b-4075-8919-8697586347c6-kube-api-access-b6b9d\") pod \"network-check-target-crvwv\" (UID: \"be92fdf2-3d2b-4075-8919-8697586347c6\") " pod="openshift-network-diagnostics/network-check-target-crvwv" May 29 16:57:06.795489 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:06.795439 2549 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 29 16:57:06.795489 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:06.795468 2549 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 29 16:57:06.795489 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:06.795480 2549 projected.go:194] Error preparing data for projected volume kube-api-access-b6b9d for pod openshift-network-diagnostics/network-check-target-crvwv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 29 16:57:06.795644 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:06.795533 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be92fdf2-3d2b-4075-8919-8697586347c6-kube-api-access-b6b9d podName:be92fdf2-3d2b-4075-8919-8697586347c6 nodeName:}" failed. No retries permitted until 2026-05-29 16:57:22.795518564 +0000 UTC m=+33.281947743 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-b6b9d" (UniqueName: "kubernetes.io/projected/be92fdf2-3d2b-4075-8919-8697586347c6-kube-api-access-b6b9d") pod "network-check-target-crvwv" (UID: "be92fdf2-3d2b-4075-8919-8697586347c6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 29 16:57:07.079855 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:07.079820 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-crvwv" May 29 16:57:07.080027 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:07.079820 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ps265" May 29 16:57:07.080027 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:07.079933 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-crvwv" podUID="be92fdf2-3d2b-4075-8919-8697586347c6" May 29 16:57:07.080128 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:07.080029 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ps265" podUID="7d457970-4377-4157-9d77-81db07916925" May 29 16:57:07.080128 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:07.079820 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dhvms" May 29 16:57:07.080128 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:07.080120 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dhvms" podUID="eb980098-9793-4340-b57d-b0600c17b07b" May 29 16:57:09.080752 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:09.080719 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-crvwv" May 29 16:57:09.081248 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:09.080719 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dhvms" May 29 16:57:09.081248 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:09.080833 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-crvwv" podUID="be92fdf2-3d2b-4075-8919-8697586347c6" May 29 16:57:09.081248 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:09.080730 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ps265" May 29 16:57:09.081248 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:09.080934 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dhvms" podUID="eb980098-9793-4340-b57d-b0600c17b07b" May 29 16:57:09.081248 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:09.081022 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ps265" podUID="7d457970-4377-4157-9d77-81db07916925" May 29 16:57:09.209537 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:09.209506 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eb980098-9793-4340-b57d-b0600c17b07b-original-pull-secret\") pod \"global-pull-secret-syncer-dhvms\" (UID: \"eb980098-9793-4340-b57d-b0600c17b07b\") " pod="kube-system/global-pull-secret-syncer-dhvms" May 29 16:57:09.209684 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:09.209635 2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 29 16:57:09.209684 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:09.209684 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb980098-9793-4340-b57d-b0600c17b07b-original-pull-secret podName:eb980098-9793-4340-b57d-b0600c17b07b nodeName:}" failed. No retries permitted until 2026-05-29 16:57:17.209671983 +0000 UTC m=+27.696101163 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/eb980098-9793-4340-b57d-b0600c17b07b-original-pull-secret") pod "global-pull-secret-syncer-dhvms" (UID: "eb980098-9793-4340-b57d-b0600c17b07b") : object "kube-system"/"original-pull-secret" not registered May 29 16:57:10.207755 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:10.207657 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-197.ec2.internal" event={"ID":"62bc43ff86f9d08efd1447f9d66d955f","Type":"ContainerStarted","Data":"eedf5ec1363415e561775a6484309cb2da0c1a99dee344c3d47d910caf2f5644"} May 29 16:57:10.208934 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:10.208909 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4fwqn" event={"ID":"e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101","Type":"ContainerStarted","Data":"3ac72c63975094ad9754ac998dee35f2b954f1f244ec29f4ec964ba74a48e8ff"} May 29 16:57:10.210046 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:10.210026 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" event={"ID":"f153a7a3-935e-44ac-8a76-d3ab6333809a","Type":"ContainerStarted","Data":"8b47799d6b85f8a4da1435ca56c30f2dc1927c44060e5c814010ce428507e22d"} May 29 16:57:10.211447 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:10.211427 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" event={"ID":"f6995bff-b11d-42e1-87ea-ecc131cd8810","Type":"ContainerStarted","Data":"779e851957116380784bb6d1fb90f54f4a6fb0a56bb7790077290f9a620d479e"} May 29 16:57:10.211513 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:10.211449 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" event={"ID":"f6995bff-b11d-42e1-87ea-ecc131cd8810","Type":"ContainerStarted","Data":"0121b463a21800f5f8ba68011f4afee3ed97a8a5b133f208a5437d75ab46cae4"} May 29 16:57:10.212467 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:10.212448 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-w9jdd" event={"ID":"54c6a295-b9b0-4cda-9a11-33a3b53a7dfb","Type":"ContainerStarted","Data":"1052e6147cd9d6104efc27877185d4d0a072c888b7bfd5061e6de824ab26c80f"} May 29 16:57:10.220879 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:10.220815 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-197.ec2.internal" podStartSLOduration=20.220805521 podStartE2EDuration="20.220805521s" podCreationTimestamp="2026-05-29 16:56:50 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 16:57:10.220652686 +0000 UTC m=+20.707081889" watchObservedRunningTime="2026-05-29 16:57:10.220805521 +0000 UTC m=+20.707234723" May 29 16:57:10.235282 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:10.235216 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-4fwqn" podStartSLOduration=2.119319376 podStartE2EDuration="20.235200983s" podCreationTimestamp="2026-05-29 16:56:50 +0000 UTC" firstStartedPulling="2026-05-29 16:56:51.329355049 +0000 UTC m=+1.815784233" lastFinishedPulling="2026-05-29 16:57:09.445236655 +0000 UTC m=+19.931665840" observedRunningTime="2026-05-29 16:57:10.235155141 +0000 UTC m=+20.721584342" watchObservedRunningTime="2026-05-29 16:57:10.235200983 +0000 UTC m=+20.721630166" May 29 16:57:10.249581 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:10.249550 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9nn7d" podStartSLOduration=2.140575883 podStartE2EDuration="20.249539852s" podCreationTimestamp="2026-05-29 16:56:50 +0000 UTC" firstStartedPulling="2026-05-29 16:56:51.318028807 +0000 UTC m=+1.804457988" lastFinishedPulling="2026-05-29 16:57:09.426992773 +0000 UTC m=+19.913421957" observedRunningTime="2026-05-29 16:57:10.249289496 +0000 UTC m=+20.735718697" watchObservedRunningTime="2026-05-29 16:57:10.249539852 +0000 UTC m=+20.735969054" May 29 16:57:10.261158 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:10.261129 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-w9jdd" podStartSLOduration=2.147862097 podStartE2EDuration="20.261120401s" podCreationTimestamp="2026-05-29 16:56:50 +0000 UTC" firstStartedPulling="2026-05-29 16:56:51.313351759 +0000 UTC m=+1.799780950" lastFinishedPulling="2026-05-29 16:57:09.426610056 +0000 UTC m=+19.913039254" observedRunningTime="2026-05-29 16:57:10.260816271 +0000 UTC m=+20.747245472" watchObservedRunningTime="2026-05-29 16:57:10.261120401 +0000 UTC m=+20.747549603" May 29 16:57:11.080802 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:11.080769 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dhvms" May 29 16:57:11.080992 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:11.080888 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ps265" May 29 16:57:11.080992 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:11.080904 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-crvwv" May 29 16:57:11.080992 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:11.080906 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dhvms" podUID="eb980098-9793-4340-b57d-b0600c17b07b" May 29 16:57:11.080992 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:11.080975 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ps265" podUID="7d457970-4377-4157-9d77-81db07916925" May 29 16:57:11.081131 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:11.081058 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-crvwv" podUID="be92fdf2-3d2b-4075-8919-8697586347c6" May 29 16:57:11.191049 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:11.191026 2549 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" May 29 16:57:11.217057 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:11.217029 2549 generic.go:358] "Generic (PLEG): container finished" podID="56335f96-b27f-4a52-beb7-656001cdea57" containerID="50177e04f7827ce51e2e273f83e4924f7b838e6d49f1835efd5f7e70f09ae07b" exitCode=0 May 29 16:57:11.217416 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:11.217096 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-52zq6" event={"ID":"56335f96-b27f-4a52-beb7-656001cdea57","Type":"ContainerDied","Data":"50177e04f7827ce51e2e273f83e4924f7b838e6d49f1835efd5f7e70f09ae07b"} May 29 16:57:11.218352 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:11.218329 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wt655" event={"ID":"e270884a-3a8f-4299-9cfa-a44e2bf1768f","Type":"ContainerStarted","Data":"2f1fcf88beddee875b239d4cf4f123cbb55b5d0821512d5a7631086e4d15bd9e"} May 29 16:57:11.219835 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:11.219804 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2m2f6" event={"ID":"46b01332-3570-4d2c-b52f-72165dbc2cf8","Type":"ContainerStarted","Data":"b263eed1a07030923c54a3cdd4e1b3307ab496ab61e2e15b9beca6eba788373c"} May 29 16:57:11.219835 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:11.219830 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2m2f6" event={"ID":"46b01332-3570-4d2c-b52f-72165dbc2cf8","Type":"ContainerStarted","Data":"89b63273384c5c304139e419036b4c004e516868c4e04e0c62f0b6200afe3d04"} May 29 16:57:11.221153 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:11.221133 2549 generic.go:358] "Generic (PLEG): container finished" podID="dfaf61f08c1074bed45e475649697dac" containerID="9b057d391ba3db9e19a657b0d452abcb6cd424a31af1342e04761e4b834c00a7" exitCode=0 May 29 16:57:11.221230 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:11.221190 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-197.ec2.internal" event={"ID":"dfaf61f08c1074bed45e475649697dac","Type":"ContainerDied","Data":"9b057d391ba3db9e19a657b0d452abcb6cd424a31af1342e04761e4b834c00a7"} May 29 16:57:11.222444 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:11.222422 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qdvbn" event={"ID":"962b0454-8569-4239-b4b3-b88b0fe86570","Type":"ContainerStarted","Data":"716f2764adb379857b6fa62f55b124915fc30189ab8a5d9f7525b1651333edbe"} May 29 16:57:11.225385 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:11.225356 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" event={"ID":"f6995bff-b11d-42e1-87ea-ecc131cd8810","Type":"ContainerStarted","Data":"10255ba1e16caa4a97f13044aadca0a1eff9e3b9167d3200ce1a3b14be9296e9"} May 29 16:57:11.225470 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:11.225394 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" event={"ID":"f6995bff-b11d-42e1-87ea-ecc131cd8810","Type":"ContainerStarted","Data":"03e6e384e43b511681038cfa1ff1aec610a3037466c6e31e91c8f300411d104b"} May 29 16:57:11.225470 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:11.225409 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" event={"ID":"f6995bff-b11d-42e1-87ea-ecc131cd8810","Type":"ContainerStarted","Data":"697c5aef52a974b81f1276234948640d4cb21a8ee7daaa82fb307f39a682fdbb"} May 29 16:57:11.225470 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:11.225421 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" event={"ID":"f6995bff-b11d-42e1-87ea-ecc131cd8810","Type":"ContainerStarted","Data":"83437cffecfed41c0c97bc586a02ff8dd12b2f4f7d9a961aacf2919e667974db"} May 29 16:57:11.247499 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:11.247456 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qdvbn" podStartSLOduration=7.7303913600000005 podStartE2EDuration="21.247445668s" podCreationTimestamp="2026-05-29 16:56:50 +0000 UTC" firstStartedPulling="2026-05-29 16:56:51.324147878 +0000 UTC m=+1.810577066" lastFinishedPulling="2026-05-29 16:57:04.841202181 +0000 UTC m=+15.327631374" observedRunningTime="2026-05-29 16:57:11.246741385 +0000 UTC m=+21.733170589" watchObservedRunningTime="2026-05-29 16:57:11.247445668 +0000 UTC m=+21.733874869" May 29 16:57:11.270340 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:11.270304 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wt655" podStartSLOduration=3.180106867 podStartE2EDuration="21.270294256s" podCreationTimestamp="2026-05-29 16:56:50 +0000 UTC" firstStartedPulling="2026-05-29 16:56:51.308447395 +0000 UTC m=+1.794876588" lastFinishedPulling="2026-05-29 16:57:09.398634794 +0000 UTC m=+19.885063977" observedRunningTime="2026-05-29 16:57:11.270063247 +0000 UTC m=+21.756492450" watchObservedRunningTime="2026-05-29 16:57:11.270294256 +0000 UTC m=+21.756723457" May 29 16:57:12.049648 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:12.049559 2549 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-29T16:57:11.191042425Z","UUID":"6ecfc809-7bea-4f6b-a0b2-6a9d8c522230","Handler":null,"Name":"","Endpoint":""} May 29 16:57:12.052191 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:12.052167 2549 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 May 29 16:57:12.052285 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:12.052198 2549 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock May 29 16:57:12.097127 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:12.097100 2549 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-w9jdd" May 29 16:57:12.097708 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:12.097685 2549 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-w9jdd" May 29 16:57:12.231280 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:12.231189 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2m2f6" event={"ID":"46b01332-3570-4d2c-b52f-72165dbc2cf8","Type":"ContainerStarted","Data":"938ce0da452ebfe1dd0d347f3d20aae9f9a51de7659f90aa17fd60babf5ddf7c"} May 29 16:57:12.233111 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:12.233072 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-197.ec2.internal" event={"ID":"dfaf61f08c1074bed45e475649697dac","Type":"ContainerStarted","Data":"2d892f58360882649e2f61687c0b178998353f13202d2f6a9dad7c6e7d7626cf"} May 29 16:57:12.233606 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:12.233418 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-w9jdd" May 29 16:57:12.233964 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:12.233943 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-w9jdd" May 29 16:57:12.263111 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:12.263067 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2m2f6" podStartSLOduration=1.602981997 podStartE2EDuration="22.263038625s" podCreationTimestamp="2026-05-29 16:56:50 +0000 UTC" firstStartedPulling="2026-05-29 16:56:51.301257446 +0000 UTC m=+1.787686626" lastFinishedPulling="2026-05-29 16:57:11.96131406 +0000 UTC m=+22.447743254" observedRunningTime="2026-05-29 16:57:12.246654423 +0000 UTC m=+22.733083630" watchObservedRunningTime="2026-05-29 16:57:12.263038625 +0000 UTC m=+22.749467839" May 29 16:57:12.263239 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:12.263172 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-197.ec2.internal" podStartSLOduration=22.263167306 podStartE2EDuration="22.263167306s" podCreationTimestamp="2026-05-29 16:56:50 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 16:57:12.262717067 +0000 UTC m=+22.749146268" watchObservedRunningTime="2026-05-29 16:57:12.263167306 +0000 UTC m=+22.749596508" May 29 16:57:13.080607 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:13.080578 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dhvms" May 29 16:57:13.080794 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:13.080615 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-crvwv" May 29 16:57:13.080794 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:13.080683 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dhvms" podUID="eb980098-9793-4340-b57d-b0600c17b07b" May 29 16:57:13.080794 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:13.080758 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ps265" May 29 16:57:13.080926 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:13.080891 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ps265" podUID="7d457970-4377-4157-9d77-81db07916925" May 29 16:57:13.080989 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:13.080973 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-crvwv" podUID="be92fdf2-3d2b-4075-8919-8697586347c6" May 29 16:57:13.238782 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:13.238741 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" event={"ID":"f6995bff-b11d-42e1-87ea-ecc131cd8810","Type":"ContainerStarted","Data":"d0557e78265fce07ca5aa1e0994ba77bb2c4bd7a47a859d1c48869a6bf769944"} May 29 16:57:15.080094 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:15.080060 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ps265" May 29 16:57:15.080543 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:15.080060 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-crvwv" May 29 16:57:15.080543 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:15.080187 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ps265" podUID="7d457970-4377-4157-9d77-81db07916925" May 29 16:57:15.080543 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:15.080211 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dhvms" May 29 16:57:15.080543 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:15.080285 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dhvms" podUID="eb980098-9793-4340-b57d-b0600c17b07b" May 29 16:57:15.080543 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:15.080353 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-crvwv" podUID="be92fdf2-3d2b-4075-8919-8697586347c6" May 29 16:57:16.245527 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:16.245340 2549 generic.go:358] "Generic (PLEG): container finished" podID="56335f96-b27f-4a52-beb7-656001cdea57" containerID="b701c1430bdec27f27f0e9a5d52bce0e7978b27c7efa1786276b3a01cbeaaff0" exitCode=0 May 29 16:57:16.246360 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:16.245424 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-52zq6" event={"ID":"56335f96-b27f-4a52-beb7-656001cdea57","Type":"ContainerDied","Data":"b701c1430bdec27f27f0e9a5d52bce0e7978b27c7efa1786276b3a01cbeaaff0"} May 29 16:57:16.248597 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:16.248575 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" event={"ID":"f6995bff-b11d-42e1-87ea-ecc131cd8810","Type":"ContainerStarted","Data":"39c4e70b1bc97b764c0d100557cfd53c6926754729bf4370a44857968fae1ebe"} May 29 16:57:16.248890 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:16.248869 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:57:16.249019 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:16.248896 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:57:16.264031 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:16.264010 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:57:16.290049 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:16.290012 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" podStartSLOduration=7.657762755 podStartE2EDuration="26.289999896s" podCreationTimestamp="2026-05-29 16:56:50 +0000 UTC" firstStartedPulling="2026-05-29 16:56:51.297084082 +0000 UTC m=+1.783513261" lastFinishedPulling="2026-05-29 16:57:09.929321206 +0000 UTC m=+20.415750402" observedRunningTime="2026-05-29 16:57:16.289657294 +0000 UTC m=+26.776086496" watchObservedRunningTime="2026-05-29 16:57:16.289999896 +0000 UTC m=+26.776429097" May 29 16:57:16.768622 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:16.768593 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-z94bv"] May 29 16:57:16.770441 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:16.770406 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-z94bv" May 29 16:57:16.775264 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:16.772407 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" May 29 16:57:16.775264 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:16.773192 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-fxlbd\"" May 29 16:57:16.775264 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:16.773620 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" May 29 16:57:16.870864 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:16.870196 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c547c68-54a9-4af7-afb9-494d1a1b6a03-tmp-dir\") pod \"node-resolver-z94bv\" (UID: \"5c547c68-54a9-4af7-afb9-494d1a1b6a03\") " pod="openshift-dns/node-resolver-z94bv" May 29 16:57:16.870864 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:16.870277 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5c547c68-54a9-4af7-afb9-494d1a1b6a03-hosts-file\") pod \"node-resolver-z94bv\" (UID: \"5c547c68-54a9-4af7-afb9-494d1a1b6a03\") " pod="openshift-dns/node-resolver-z94bv" May 29 16:57:16.870864 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:16.870310 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hb2m5\" (UniqueName: \"kubernetes.io/projected/5c547c68-54a9-4af7-afb9-494d1a1b6a03-kube-api-access-hb2m5\") pod \"node-resolver-z94bv\" (UID: \"5c547c68-54a9-4af7-afb9-494d1a1b6a03\") " pod="openshift-dns/node-resolver-z94bv" May 29 16:57:16.971698 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:16.971440 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5c547c68-54a9-4af7-afb9-494d1a1b6a03-hosts-file\") pod \"node-resolver-z94bv\" (UID: \"5c547c68-54a9-4af7-afb9-494d1a1b6a03\") " pod="openshift-dns/node-resolver-z94bv" May 29 16:57:16.971698 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:16.971490 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hb2m5\" (UniqueName: \"kubernetes.io/projected/5c547c68-54a9-4af7-afb9-494d1a1b6a03-kube-api-access-hb2m5\") pod \"node-resolver-z94bv\" (UID: \"5c547c68-54a9-4af7-afb9-494d1a1b6a03\") " pod="openshift-dns/node-resolver-z94bv" May 29 16:57:16.971698 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:16.971560 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c547c68-54a9-4af7-afb9-494d1a1b6a03-tmp-dir\") pod \"node-resolver-z94bv\" (UID: \"5c547c68-54a9-4af7-afb9-494d1a1b6a03\") " pod="openshift-dns/node-resolver-z94bv" May 29 16:57:16.971972 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:16.971781 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5c547c68-54a9-4af7-afb9-494d1a1b6a03-hosts-file\") pod \"node-resolver-z94bv\" (UID: \"5c547c68-54a9-4af7-afb9-494d1a1b6a03\") " pod="openshift-dns/node-resolver-z94bv" May 29 16:57:16.971972 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:16.971945 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c547c68-54a9-4af7-afb9-494d1a1b6a03-tmp-dir\") pod \"node-resolver-z94bv\" (UID: \"5c547c68-54a9-4af7-afb9-494d1a1b6a03\") " pod="openshift-dns/node-resolver-z94bv" May 29 16:57:16.981170 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:16.981143 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hb2m5\" (UniqueName: \"kubernetes.io/projected/5c547c68-54a9-4af7-afb9-494d1a1b6a03-kube-api-access-hb2m5\") pod \"node-resolver-z94bv\" (UID: \"5c547c68-54a9-4af7-afb9-494d1a1b6a03\") " pod="openshift-dns/node-resolver-z94bv" May 29 16:57:17.079957 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:17.079922 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-crvwv" May 29 16:57:17.080087 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:17.079995 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ps265" May 29 16:57:17.080087 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:17.080023 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dhvms" May 29 16:57:17.080164 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:17.080103 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ps265" podUID="7d457970-4377-4157-9d77-81db07916925" May 29 16:57:17.080204 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:17.080171 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dhvms" podUID="eb980098-9793-4340-b57d-b0600c17b07b" May 29 16:57:17.080235 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:17.080203 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-crvwv" podUID="be92fdf2-3d2b-4075-8919-8697586347c6" May 29 16:57:17.084003 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:17.083980 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-z94bv" May 29 16:57:17.102679 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:57:17.102651 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c547c68_54a9_4af7_afb9_494d1a1b6a03.slice/crio-472114630e43c5343b2533638f232d8f4b32cb763febc5b68bcbb99bbd3b7666 WatchSource:0}: Error finding container 472114630e43c5343b2533638f232d8f4b32cb763febc5b68bcbb99bbd3b7666: Status 404 returned error can't find the container with id 472114630e43c5343b2533638f232d8f4b32cb763febc5b68bcbb99bbd3b7666 May 29 16:57:17.252423 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:17.252390 2549 generic.go:358] "Generic (PLEG): container finished" podID="56335f96-b27f-4a52-beb7-656001cdea57" containerID="bd3658297d68ede3a000dc0608eea9d261b29998cf322791f9d5c470945ab77d" exitCode=0 May 29 16:57:17.252864 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:17.252463 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-52zq6" event={"ID":"56335f96-b27f-4a52-beb7-656001cdea57","Type":"ContainerDied","Data":"bd3658297d68ede3a000dc0608eea9d261b29998cf322791f9d5c470945ab77d"} May 29 16:57:17.253924 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:17.253799 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-z94bv" event={"ID":"5c547c68-54a9-4af7-afb9-494d1a1b6a03","Type":"ContainerStarted","Data":"07199b0ec50d4e91f35cd2ff9332b211c5e167ca55bc6590bbd0b62c6ecd8568"} May 29 16:57:17.253924 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:17.253853 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-z94bv" event={"ID":"5c547c68-54a9-4af7-afb9-494d1a1b6a03","Type":"ContainerStarted","Data":"472114630e43c5343b2533638f232d8f4b32cb763febc5b68bcbb99bbd3b7666"} May 29 16:57:17.254542 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:17.254513 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:57:17.269254 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:17.269230 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:57:17.274159 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:17.274084 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eb980098-9793-4340-b57d-b0600c17b07b-original-pull-secret\") pod \"global-pull-secret-syncer-dhvms\" (UID: \"eb980098-9793-4340-b57d-b0600c17b07b\") " pod="kube-system/global-pull-secret-syncer-dhvms" May 29 16:57:17.274283 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:17.274177 2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 29 16:57:17.274283 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:17.274223 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb980098-9793-4340-b57d-b0600c17b07b-original-pull-secret podName:eb980098-9793-4340-b57d-b0600c17b07b nodeName:}" failed. No retries permitted until 2026-05-29 16:57:33.274210255 +0000 UTC m=+43.760639435 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/eb980098-9793-4340-b57d-b0600c17b07b-original-pull-secret") pod "global-pull-secret-syncer-dhvms" (UID: "eb980098-9793-4340-b57d-b0600c17b07b") : object "kube-system"/"original-pull-secret" not registered May 29 16:57:17.284757 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:17.284715 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-z94bv" podStartSLOduration=1.284701523 podStartE2EDuration="1.284701523s" podCreationTimestamp="2026-05-29 16:57:16 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 16:57:17.284369909 +0000 UTC m=+27.770799113" watchObservedRunningTime="2026-05-29 16:57:17.284701523 +0000 UTC m=+27.771130725" May 29 16:57:17.571325 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:17.570455 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dhvms"] May 29 16:57:17.571325 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:17.570600 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dhvms" May 29 16:57:17.571325 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:17.570742 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dhvms" podUID="eb980098-9793-4340-b57d-b0600c17b07b" May 29 16:57:17.572409 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:17.572377 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-crvwv"] May 29 16:57:17.572520 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:17.572498 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-crvwv" May 29 16:57:17.572634 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:17.572601 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-crvwv" podUID="be92fdf2-3d2b-4075-8919-8697586347c6" May 29 16:57:17.573279 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:17.573251 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ps265"] May 29 16:57:17.573521 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:17.573502 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ps265" May 29 16:57:17.573678 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:17.573653 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ps265" podUID="7d457970-4377-4157-9d77-81db07916925" May 29 16:57:18.257444 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:18.257219 2549 generic.go:358] "Generic (PLEG): container finished" podID="56335f96-b27f-4a52-beb7-656001cdea57" containerID="44916dc5a8693c01a810e9912e6f2217b656e0b6db30a036fca41319b3a0c80d" exitCode=0 May 29 16:57:18.257811 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:18.257300 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-52zq6" event={"ID":"56335f96-b27f-4a52-beb7-656001cdea57","Type":"ContainerDied","Data":"44916dc5a8693c01a810e9912e6f2217b656e0b6db30a036fca41319b3a0c80d"} May 29 16:57:19.079946 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:19.079911 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dhvms" May 29 16:57:19.080134 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:19.079919 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-crvwv" May 29 16:57:19.080134 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:19.079932 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ps265" May 29 16:57:19.080134 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:19.080086 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dhvms" podUID="eb980098-9793-4340-b57d-b0600c17b07b" May 29 16:57:19.080257 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:19.080179 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-crvwv" podUID="be92fdf2-3d2b-4075-8919-8697586347c6" May 29 16:57:19.080296 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:19.080275 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ps265" podUID="7d457970-4377-4157-9d77-81db07916925" May 29 16:57:21.080802 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:21.080770 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-crvwv" May 29 16:57:21.081389 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:21.080799 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ps265" May 29 16:57:21.081389 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:21.080770 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dhvms" May 29 16:57:21.081389 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:21.080896 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-crvwv" podUID="be92fdf2-3d2b-4075-8919-8697586347c6" May 29 16:57:21.081389 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:21.081001 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dhvms" podUID="eb980098-9793-4340-b57d-b0600c17b07b" May 29 16:57:21.081389 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:21.081089 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ps265" podUID="7d457970-4377-4157-9d77-81db07916925" May 29 16:57:22.713206 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:22.713168 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d457970-4377-4157-9d77-81db07916925-metrics-certs\") pod \"network-metrics-daemon-ps265\" (UID: \"7d457970-4377-4157-9d77-81db07916925\") " pod="openshift-multus/network-metrics-daemon-ps265" May 29 16:57:22.713708 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:22.713293 2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 29 16:57:22.713708 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:22.713380 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d457970-4377-4157-9d77-81db07916925-metrics-certs podName:7d457970-4377-4157-9d77-81db07916925 nodeName:}" failed. No retries permitted until 2026-05-29 16:57:54.713355561 +0000 UTC m=+65.199784794 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7d457970-4377-4157-9d77-81db07916925-metrics-certs") pod "network-metrics-daemon-ps265" (UID: "7d457970-4377-4157-9d77-81db07916925") : object "openshift-multus"/"metrics-daemon-secret" not registered May 29 16:57:22.814477 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:22.814441 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b6b9d\" (UniqueName: \"kubernetes.io/projected/be92fdf2-3d2b-4075-8919-8697586347c6-kube-api-access-b6b9d\") pod \"network-check-target-crvwv\" (UID: \"be92fdf2-3d2b-4075-8919-8697586347c6\") " pod="openshift-network-diagnostics/network-check-target-crvwv" May 29 16:57:22.814659 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:22.814628 2549 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 29 16:57:22.814659 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:22.814657 2549 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 29 16:57:22.814765 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:22.814672 2549 projected.go:194] Error preparing data for projected volume kube-api-access-b6b9d for pod openshift-network-diagnostics/network-check-target-crvwv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 29 16:57:22.814765 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:22.814736 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be92fdf2-3d2b-4075-8919-8697586347c6-kube-api-access-b6b9d podName:be92fdf2-3d2b-4075-8919-8697586347c6 nodeName:}" failed. No retries permitted until 2026-05-29 16:57:54.814717808 +0000 UTC m=+65.301146992 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-b6b9d" (UniqueName: "kubernetes.io/projected/be92fdf2-3d2b-4075-8919-8697586347c6-kube-api-access-b6b9d") pod "network-check-target-crvwv" (UID: "be92fdf2-3d2b-4075-8919-8697586347c6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 29 16:57:23.080363 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.080326 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ps265" May 29 16:57:23.080571 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.080326 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-crvwv" May 29 16:57:23.080571 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:23.080478 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ps265" podUID="7d457970-4377-4157-9d77-81db07916925" May 29 16:57:23.080571 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:23.080541 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-crvwv" podUID="be92fdf2-3d2b-4075-8919-8697586347c6" May 29 16:57:23.080571 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.080327 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dhvms" May 29 16:57:23.080765 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:23.080633 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dhvms" podUID="eb980098-9793-4340-b57d-b0600c17b07b" May 29 16:57:23.295302 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.295260 2549 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-197.ec2.internal" event="NodeReady" May 29 16:57:23.295463 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.295398 2549 kubelet_node_status.go:550] "Fast updating node status as it just became ready" May 29 16:57:23.335180 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.335089 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-76f776b797-lhxhm"] May 29 16:57:23.361786 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.361761 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76f776b797-lhxhm" May 29 16:57:23.364359 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.364329 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" May 29 16:57:23.364921 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.364899 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" May 29 16:57:23.365510 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.365393 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-kzj26"] May 29 16:57:23.371358 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.371322 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" May 29 16:57:23.371904 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.371834 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vq84q\"" May 29 16:57:23.383467 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.383443 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" May 29 16:57:23.388890 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.388867 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76f776b797-lhxhm"] May 29 16:57:23.388997 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.388896 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-wnqbv"] May 29 16:57:23.388997 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.388963 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kzj26" May 29 16:57:23.390941 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.390921 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" May 29 16:57:23.391040 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.390921 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xwwt6\"" May 29 16:57:23.391040 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.390965 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" May 29 16:57:23.412994 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.412970 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-kzj26"] May 29 16:57:23.412994 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.412995 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wnqbv"] May 29 16:57:23.413168 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.413093 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wnqbv" May 29 16:57:23.415148 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.415109 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" May 29 16:57:23.415262 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.415127 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" May 29 16:57:23.415262 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.415168 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-fgcrt\"" May 29 16:57:23.415405 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.415171 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" May 29 16:57:23.519861 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.519812 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f35fbbaf-13e2-4a63-865d-00375eb46e97-image-registry-private-configuration\") pod \"image-registry-76f776b797-lhxhm\" (UID: \"f35fbbaf-13e2-4a63-865d-00375eb46e97\") " pod="openshift-image-registry/image-registry-76f776b797-lhxhm" May 29 16:57:23.520057 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.519871 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f35fbbaf-13e2-4a63-865d-00375eb46e97-registry-tls\") pod \"image-registry-76f776b797-lhxhm\" (UID: \"f35fbbaf-13e2-4a63-865d-00375eb46e97\") " pod="openshift-image-registry/image-registry-76f776b797-lhxhm" May 29 16:57:23.520057 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.519896 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f35fbbaf-13e2-4a63-865d-00375eb46e97-trusted-ca\") pod \"image-registry-76f776b797-lhxhm\" (UID: \"f35fbbaf-13e2-4a63-865d-00375eb46e97\") " pod="openshift-image-registry/image-registry-76f776b797-lhxhm" May 29 16:57:23.520057 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.519921 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bca913b1-2d98-44c7-906e-bdb4c8450658-metrics-tls\") pod \"dns-default-kzj26\" (UID: \"bca913b1-2d98-44c7-906e-bdb4c8450658\") " pod="openshift-dns/dns-default-kzj26" May 29 16:57:23.520057 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.519945 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f35fbbaf-13e2-4a63-865d-00375eb46e97-installation-pull-secrets\") pod \"image-registry-76f776b797-lhxhm\" (UID: \"f35fbbaf-13e2-4a63-865d-00375eb46e97\") " pod="openshift-image-registry/image-registry-76f776b797-lhxhm" May 29 16:57:23.520057 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.519973 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8sv5\" (UniqueName: \"kubernetes.io/projected/f35fbbaf-13e2-4a63-865d-00375eb46e97-kube-api-access-p8sv5\") pod \"image-registry-76f776b797-lhxhm\" (UID: \"f35fbbaf-13e2-4a63-865d-00375eb46e97\") " pod="openshift-image-registry/image-registry-76f776b797-lhxhm" May 29 16:57:23.520057 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.519995 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtv2h\" (UniqueName: \"kubernetes.io/projected/bca913b1-2d98-44c7-906e-bdb4c8450658-kube-api-access-vtv2h\") pod \"dns-default-kzj26\" (UID: \"bca913b1-2d98-44c7-906e-bdb4c8450658\") " pod="openshift-dns/dns-default-kzj26" May 29 16:57:23.520057 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.520041 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f35fbbaf-13e2-4a63-865d-00375eb46e97-ca-trust-extracted\") pod \"image-registry-76f776b797-lhxhm\" (UID: \"f35fbbaf-13e2-4a63-865d-00375eb46e97\") " pod="openshift-image-registry/image-registry-76f776b797-lhxhm" May 29 16:57:23.520332 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.520085 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f35fbbaf-13e2-4a63-865d-00375eb46e97-registry-certificates\") pod \"image-registry-76f776b797-lhxhm\" (UID: \"f35fbbaf-13e2-4a63-865d-00375eb46e97\") " pod="openshift-image-registry/image-registry-76f776b797-lhxhm" May 29 16:57:23.520332 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.520109 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/17e14531-cc91-412b-9804-9c1c9fe52eb8-cert\") pod \"ingress-canary-wnqbv\" (UID: \"17e14531-cc91-412b-9804-9c1c9fe52eb8\") " pod="openshift-ingress-canary/ingress-canary-wnqbv" May 29 16:57:23.520332 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.520175 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f35fbbaf-13e2-4a63-865d-00375eb46e97-bound-sa-token\") pod \"image-registry-76f776b797-lhxhm\" (UID: \"f35fbbaf-13e2-4a63-865d-00375eb46e97\") " pod="openshift-image-registry/image-registry-76f776b797-lhxhm" May 29 16:57:23.520332 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.520227 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bca913b1-2d98-44c7-906e-bdb4c8450658-config-volume\") pod \"dns-default-kzj26\" (UID: \"bca913b1-2d98-44c7-906e-bdb4c8450658\") " pod="openshift-dns/dns-default-kzj26" May 29 16:57:23.520332 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.520268 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hw5wv\" (UniqueName: \"kubernetes.io/projected/17e14531-cc91-412b-9804-9c1c9fe52eb8-kube-api-access-hw5wv\") pod \"ingress-canary-wnqbv\" (UID: \"17e14531-cc91-412b-9804-9c1c9fe52eb8\") " pod="openshift-ingress-canary/ingress-canary-wnqbv" May 29 16:57:23.520332 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.520299 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bca913b1-2d98-44c7-906e-bdb4c8450658-tmp-dir\") pod \"dns-default-kzj26\" (UID: \"bca913b1-2d98-44c7-906e-bdb4c8450658\") " pod="openshift-dns/dns-default-kzj26" May 29 16:57:23.621723 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.621638 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f35fbbaf-13e2-4a63-865d-00375eb46e97-image-registry-private-configuration\") pod \"image-registry-76f776b797-lhxhm\" (UID: \"f35fbbaf-13e2-4a63-865d-00375eb46e97\") " pod="openshift-image-registry/image-registry-76f776b797-lhxhm" May 29 16:57:23.621723 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.621687 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f35fbbaf-13e2-4a63-865d-00375eb46e97-registry-tls\") pod \"image-registry-76f776b797-lhxhm\" (UID: \"f35fbbaf-13e2-4a63-865d-00375eb46e97\") " pod="openshift-image-registry/image-registry-76f776b797-lhxhm" May 29 16:57:23.622002 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.621788 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f35fbbaf-13e2-4a63-865d-00375eb46e97-trusted-ca\") pod \"image-registry-76f776b797-lhxhm\" (UID: \"f35fbbaf-13e2-4a63-865d-00375eb46e97\") " pod="openshift-image-registry/image-registry-76f776b797-lhxhm" May 29 16:57:23.622002 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.621820 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bca913b1-2d98-44c7-906e-bdb4c8450658-metrics-tls\") pod \"dns-default-kzj26\" (UID: \"bca913b1-2d98-44c7-906e-bdb4c8450658\") " pod="openshift-dns/dns-default-kzj26" May 29 16:57:23.622002 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.621853 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f35fbbaf-13e2-4a63-865d-00375eb46e97-installation-pull-secrets\") pod \"image-registry-76f776b797-lhxhm\" (UID: \"f35fbbaf-13e2-4a63-865d-00375eb46e97\") " pod="openshift-image-registry/image-registry-76f776b797-lhxhm" May 29 16:57:23.622002 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:23.621872 2549 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 29 16:57:23.622002 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:23.621896 2549 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76f776b797-lhxhm: secret "image-registry-tls" not found May 29 16:57:23.622002 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.621880 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p8sv5\" (UniqueName: \"kubernetes.io/projected/f35fbbaf-13e2-4a63-865d-00375eb46e97-kube-api-access-p8sv5\") pod \"image-registry-76f776b797-lhxhm\" (UID: \"f35fbbaf-13e2-4a63-865d-00375eb46e97\") " pod="openshift-image-registry/image-registry-76f776b797-lhxhm" May 29 16:57:23.622002 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:23.621958 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f35fbbaf-13e2-4a63-865d-00375eb46e97-registry-tls podName:f35fbbaf-13e2-4a63-865d-00375eb46e97 nodeName:}" failed. No retries permitted until 2026-05-29 16:57:24.121936493 +0000 UTC m=+34.608365687 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f35fbbaf-13e2-4a63-865d-00375eb46e97-registry-tls") pod "image-registry-76f776b797-lhxhm" (UID: "f35fbbaf-13e2-4a63-865d-00375eb46e97") : secret "image-registry-tls" not found May 29 16:57:23.622002 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:23.621966 2549 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 29 16:57:23.622265 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.622017 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vtv2h\" (UniqueName: \"kubernetes.io/projected/bca913b1-2d98-44c7-906e-bdb4c8450658-kube-api-access-vtv2h\") pod \"dns-default-kzj26\" (UID: \"bca913b1-2d98-44c7-906e-bdb4c8450658\") " pod="openshift-dns/dns-default-kzj26" May 29 16:57:23.622265 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.622051 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f35fbbaf-13e2-4a63-865d-00375eb46e97-ca-trust-extracted\") pod \"image-registry-76f776b797-lhxhm\" (UID: \"f35fbbaf-13e2-4a63-865d-00375eb46e97\") " pod="openshift-image-registry/image-registry-76f776b797-lhxhm" May 29 16:57:23.622265 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:23.622076 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bca913b1-2d98-44c7-906e-bdb4c8450658-metrics-tls podName:bca913b1-2d98-44c7-906e-bdb4c8450658 nodeName:}" failed. No retries permitted until 2026-05-29 16:57:24.12205701 +0000 UTC m=+34.608486205 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bca913b1-2d98-44c7-906e-bdb4c8450658-metrics-tls") pod "dns-default-kzj26" (UID: "bca913b1-2d98-44c7-906e-bdb4c8450658") : secret "dns-default-metrics-tls" not found May 29 16:57:23.622265 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.622125 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f35fbbaf-13e2-4a63-865d-00375eb46e97-registry-certificates\") pod \"image-registry-76f776b797-lhxhm\" (UID: \"f35fbbaf-13e2-4a63-865d-00375eb46e97\") " pod="openshift-image-registry/image-registry-76f776b797-lhxhm" May 29 16:57:23.622265 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.622152 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/17e14531-cc91-412b-9804-9c1c9fe52eb8-cert\") pod \"ingress-canary-wnqbv\" (UID: \"17e14531-cc91-412b-9804-9c1c9fe52eb8\") " pod="openshift-ingress-canary/ingress-canary-wnqbv" May 29 16:57:23.622265 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.622186 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f35fbbaf-13e2-4a63-865d-00375eb46e97-bound-sa-token\") pod \"image-registry-76f776b797-lhxhm\" (UID: \"f35fbbaf-13e2-4a63-865d-00375eb46e97\") " pod="openshift-image-registry/image-registry-76f776b797-lhxhm" May 29 16:57:23.622265 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.622210 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bca913b1-2d98-44c7-906e-bdb4c8450658-config-volume\") pod \"dns-default-kzj26\" (UID: \"bca913b1-2d98-44c7-906e-bdb4c8450658\") " pod="openshift-dns/dns-default-kzj26" May 29 16:57:23.622265 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.622231 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hw5wv\" (UniqueName: \"kubernetes.io/projected/17e14531-cc91-412b-9804-9c1c9fe52eb8-kube-api-access-hw5wv\") pod \"ingress-canary-wnqbv\" (UID: \"17e14531-cc91-412b-9804-9c1c9fe52eb8\") " pod="openshift-ingress-canary/ingress-canary-wnqbv" May 29 16:57:23.622626 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.622271 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bca913b1-2d98-44c7-906e-bdb4c8450658-tmp-dir\") pod \"dns-default-kzj26\" (UID: \"bca913b1-2d98-44c7-906e-bdb4c8450658\") " pod="openshift-dns/dns-default-kzj26" May 29 16:57:23.622626 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:23.622364 2549 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 29 16:57:23.622626 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:23.622421 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17e14531-cc91-412b-9804-9c1c9fe52eb8-cert podName:17e14531-cc91-412b-9804-9c1c9fe52eb8 nodeName:}" failed. No retries permitted until 2026-05-29 16:57:24.122406968 +0000 UTC m=+34.608836169 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/17e14531-cc91-412b-9804-9c1c9fe52eb8-cert") pod "ingress-canary-wnqbv" (UID: "17e14531-cc91-412b-9804-9c1c9fe52eb8") : secret "canary-serving-cert" not found May 29 16:57:23.622778 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.622632 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bca913b1-2d98-44c7-906e-bdb4c8450658-tmp-dir\") pod \"dns-default-kzj26\" (UID: \"bca913b1-2d98-44c7-906e-bdb4c8450658\") " pod="openshift-dns/dns-default-kzj26" May 29 16:57:23.622835 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.622798 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f35fbbaf-13e2-4a63-865d-00375eb46e97-ca-trust-extracted\") pod \"image-registry-76f776b797-lhxhm\" (UID: \"f35fbbaf-13e2-4a63-865d-00375eb46e97\") " pod="openshift-image-registry/image-registry-76f776b797-lhxhm" May 29 16:57:23.622963 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.622928 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f35fbbaf-13e2-4a63-865d-00375eb46e97-trusted-ca\") pod \"image-registry-76f776b797-lhxhm\" (UID: \"f35fbbaf-13e2-4a63-865d-00375eb46e97\") " pod="openshift-image-registry/image-registry-76f776b797-lhxhm" May 29 16:57:23.623189 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.623166 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f35fbbaf-13e2-4a63-865d-00375eb46e97-registry-certificates\") pod \"image-registry-76f776b797-lhxhm\" (UID: \"f35fbbaf-13e2-4a63-865d-00375eb46e97\") " pod="openshift-image-registry/image-registry-76f776b797-lhxhm" May 29 16:57:23.623314 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.623291 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bca913b1-2d98-44c7-906e-bdb4c8450658-config-volume\") pod \"dns-default-kzj26\" (UID: \"bca913b1-2d98-44c7-906e-bdb4c8450658\") " pod="openshift-dns/dns-default-kzj26" May 29 16:57:23.626279 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.626261 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f35fbbaf-13e2-4a63-865d-00375eb46e97-installation-pull-secrets\") pod \"image-registry-76f776b797-lhxhm\" (UID: \"f35fbbaf-13e2-4a63-865d-00375eb46e97\") " pod="openshift-image-registry/image-registry-76f776b797-lhxhm" May 29 16:57:23.626378 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.626278 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f35fbbaf-13e2-4a63-865d-00375eb46e97-image-registry-private-configuration\") pod \"image-registry-76f776b797-lhxhm\" (UID: \"f35fbbaf-13e2-4a63-865d-00375eb46e97\") " pod="openshift-image-registry/image-registry-76f776b797-lhxhm" May 29 16:57:23.632594 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.632550 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f35fbbaf-13e2-4a63-865d-00375eb46e97-bound-sa-token\") pod \"image-registry-76f776b797-lhxhm\" (UID: \"f35fbbaf-13e2-4a63-865d-00375eb46e97\") " pod="openshift-image-registry/image-registry-76f776b797-lhxhm" May 29 16:57:23.632594 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.632575 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8sv5\" (UniqueName: \"kubernetes.io/projected/f35fbbaf-13e2-4a63-865d-00375eb46e97-kube-api-access-p8sv5\") pod \"image-registry-76f776b797-lhxhm\" (UID: \"f35fbbaf-13e2-4a63-865d-00375eb46e97\") " pod="openshift-image-registry/image-registry-76f776b797-lhxhm" May 29 16:57:23.633698 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.633680 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtv2h\" (UniqueName: \"kubernetes.io/projected/bca913b1-2d98-44c7-906e-bdb4c8450658-kube-api-access-vtv2h\") pod \"dns-default-kzj26\" (UID: \"bca913b1-2d98-44c7-906e-bdb4c8450658\") " pod="openshift-dns/dns-default-kzj26" May 29 16:57:23.634058 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:23.634039 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hw5wv\" (UniqueName: \"kubernetes.io/projected/17e14531-cc91-412b-9804-9c1c9fe52eb8-kube-api-access-hw5wv\") pod \"ingress-canary-wnqbv\" (UID: \"17e14531-cc91-412b-9804-9c1c9fe52eb8\") " pod="openshift-ingress-canary/ingress-canary-wnqbv" May 29 16:57:24.125492 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:24.125455 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f35fbbaf-13e2-4a63-865d-00375eb46e97-registry-tls\") pod \"image-registry-76f776b797-lhxhm\" (UID: \"f35fbbaf-13e2-4a63-865d-00375eb46e97\") " pod="openshift-image-registry/image-registry-76f776b797-lhxhm" May 29 16:57:24.125492 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:24.125491 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bca913b1-2d98-44c7-906e-bdb4c8450658-metrics-tls\") pod \"dns-default-kzj26\" (UID: \"bca913b1-2d98-44c7-906e-bdb4c8450658\") " pod="openshift-dns/dns-default-kzj26" May 29 16:57:24.126008 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:24.125588 2549 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 29 16:57:24.126008 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:24.125593 2549 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 29 16:57:24.126008 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:24.125612 2549 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76f776b797-lhxhm: secret "image-registry-tls" not found May 29 16:57:24.126008 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:24.125614 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/17e14531-cc91-412b-9804-9c1c9fe52eb8-cert\") pod \"ingress-canary-wnqbv\" (UID: \"17e14531-cc91-412b-9804-9c1c9fe52eb8\") " pod="openshift-ingress-canary/ingress-canary-wnqbv" May 29 16:57:24.126008 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:24.125647 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bca913b1-2d98-44c7-906e-bdb4c8450658-metrics-tls podName:bca913b1-2d98-44c7-906e-bdb4c8450658 nodeName:}" failed. No retries permitted until 2026-05-29 16:57:25.12562903 +0000 UTC m=+35.612058214 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bca913b1-2d98-44c7-906e-bdb4c8450658-metrics-tls") pod "dns-default-kzj26" (UID: "bca913b1-2d98-44c7-906e-bdb4c8450658") : secret "dns-default-metrics-tls" not found May 29 16:57:24.126008 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:24.125669 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f35fbbaf-13e2-4a63-865d-00375eb46e97-registry-tls podName:f35fbbaf-13e2-4a63-865d-00375eb46e97 nodeName:}" failed. No retries permitted until 2026-05-29 16:57:25.125659078 +0000 UTC m=+35.612088266 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f35fbbaf-13e2-4a63-865d-00375eb46e97-registry-tls") pod "image-registry-76f776b797-lhxhm" (UID: "f35fbbaf-13e2-4a63-865d-00375eb46e97") : secret "image-registry-tls" not found May 29 16:57:24.126008 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:24.125676 2549 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 29 16:57:24.126008 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:24.125717 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17e14531-cc91-412b-9804-9c1c9fe52eb8-cert podName:17e14531-cc91-412b-9804-9c1c9fe52eb8 nodeName:}" failed. No retries permitted until 2026-05-29 16:57:25.125707204 +0000 UTC m=+35.612136391 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/17e14531-cc91-412b-9804-9c1c9fe52eb8-cert") pod "ingress-canary-wnqbv" (UID: "17e14531-cc91-412b-9804-9c1c9fe52eb8") : secret "canary-serving-cert" not found May 29 16:57:25.080929 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:25.080728 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ps265" May 29 16:57:25.081095 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:25.080728 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-crvwv" May 29 16:57:25.081095 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:25.080727 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dhvms" May 29 16:57:25.083687 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:25.083662 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" May 29 16:57:25.084118 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:25.084100 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" May 29 16:57:25.084118 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:25.084109 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gvkkc\"" May 29 16:57:25.084277 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:25.084113 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fng4t\"" May 29 16:57:25.084277 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:25.084103 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" May 29 16:57:25.084277 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:25.084210 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" May 29 16:57:25.132645 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:25.132617 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f35fbbaf-13e2-4a63-865d-00375eb46e97-registry-tls\") pod \"image-registry-76f776b797-lhxhm\" (UID: \"f35fbbaf-13e2-4a63-865d-00375eb46e97\") " pod="openshift-image-registry/image-registry-76f776b797-lhxhm" May 29 16:57:25.133036 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:25.132648 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bca913b1-2d98-44c7-906e-bdb4c8450658-metrics-tls\") pod \"dns-default-kzj26\" (UID: \"bca913b1-2d98-44c7-906e-bdb4c8450658\") " pod="openshift-dns/dns-default-kzj26" May 29 16:57:25.133036 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:25.132681 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/17e14531-cc91-412b-9804-9c1c9fe52eb8-cert\") pod \"ingress-canary-wnqbv\" (UID: \"17e14531-cc91-412b-9804-9c1c9fe52eb8\") " pod="openshift-ingress-canary/ingress-canary-wnqbv" May 29 16:57:25.133036 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:25.132752 2549 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 29 16:57:25.133036 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:25.132771 2549 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76f776b797-lhxhm: secret "image-registry-tls" not found May 29 16:57:25.133036 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:25.132775 2549 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 29 16:57:25.133036 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:25.132778 2549 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 29 16:57:25.133036 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:25.132823 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f35fbbaf-13e2-4a63-865d-00375eb46e97-registry-tls podName:f35fbbaf-13e2-4a63-865d-00375eb46e97 nodeName:}" failed. No retries permitted until 2026-05-29 16:57:27.132805496 +0000 UTC m=+37.619234676 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f35fbbaf-13e2-4a63-865d-00375eb46e97-registry-tls") pod "image-registry-76f776b797-lhxhm" (UID: "f35fbbaf-13e2-4a63-865d-00375eb46e97") : secret "image-registry-tls" not found May 29 16:57:25.133036 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:25.132835 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17e14531-cc91-412b-9804-9c1c9fe52eb8-cert podName:17e14531-cc91-412b-9804-9c1c9fe52eb8 nodeName:}" failed. No retries permitted until 2026-05-29 16:57:27.132828787 +0000 UTC m=+37.619257967 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/17e14531-cc91-412b-9804-9c1c9fe52eb8-cert") pod "ingress-canary-wnqbv" (UID: "17e14531-cc91-412b-9804-9c1c9fe52eb8") : secret "canary-serving-cert" not found May 29 16:57:25.133036 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:25.132866 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bca913b1-2d98-44c7-906e-bdb4c8450658-metrics-tls podName:bca913b1-2d98-44c7-906e-bdb4c8450658 nodeName:}" failed. No retries permitted until 2026-05-29 16:57:27.132858364 +0000 UTC m=+37.619287547 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bca913b1-2d98-44c7-906e-bdb4c8450658-metrics-tls") pod "dns-default-kzj26" (UID: "bca913b1-2d98-44c7-906e-bdb4c8450658") : secret "dns-default-metrics-tls" not found May 29 16:57:25.270857 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:25.270812 2549 generic.go:358] "Generic (PLEG): container finished" podID="56335f96-b27f-4a52-beb7-656001cdea57" containerID="bc05a00c2d09651d3ec0b9b60f61aed4106a7550597245e285124251408b848a" exitCode=0 May 29 16:57:25.271012 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:25.270886 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-52zq6" event={"ID":"56335f96-b27f-4a52-beb7-656001cdea57","Type":"ContainerDied","Data":"bc05a00c2d09651d3ec0b9b60f61aed4106a7550597245e285124251408b848a"} May 29 16:57:26.275237 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:26.275206 2549 generic.go:358] "Generic (PLEG): container finished" podID="56335f96-b27f-4a52-beb7-656001cdea57" containerID="846cdaad888e6e6bdbf51fee068e59e5d5ec27a67283eeec146f40b1346f513a" exitCode=0 May 29 16:57:26.275596 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:26.275265 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-52zq6" event={"ID":"56335f96-b27f-4a52-beb7-656001cdea57","Type":"ContainerDied","Data":"846cdaad888e6e6bdbf51fee068e59e5d5ec27a67283eeec146f40b1346f513a"} May 29 16:57:27.148406 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:27.148367 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bca913b1-2d98-44c7-906e-bdb4c8450658-metrics-tls\") pod \"dns-default-kzj26\" (UID: \"bca913b1-2d98-44c7-906e-bdb4c8450658\") " pod="openshift-dns/dns-default-kzj26" May 29 16:57:27.148516 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:27.148426 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/17e14531-cc91-412b-9804-9c1c9fe52eb8-cert\") pod \"ingress-canary-wnqbv\" (UID: \"17e14531-cc91-412b-9804-9c1c9fe52eb8\") " pod="openshift-ingress-canary/ingress-canary-wnqbv" May 29 16:57:27.148516 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:27.148495 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f35fbbaf-13e2-4a63-865d-00375eb46e97-registry-tls\") pod \"image-registry-76f776b797-lhxhm\" (UID: \"f35fbbaf-13e2-4a63-865d-00375eb46e97\") " pod="openshift-image-registry/image-registry-76f776b797-lhxhm" May 29 16:57:27.148601 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:27.148516 2549 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 29 16:57:27.148601 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:27.148587 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bca913b1-2d98-44c7-906e-bdb4c8450658-metrics-tls podName:bca913b1-2d98-44c7-906e-bdb4c8450658 nodeName:}" failed. No retries permitted until 2026-05-29 16:57:31.148570379 +0000 UTC m=+41.634999582 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bca913b1-2d98-44c7-906e-bdb4c8450658-metrics-tls") pod "dns-default-kzj26" (UID: "bca913b1-2d98-44c7-906e-bdb4c8450658") : secret "dns-default-metrics-tls" not found May 29 16:57:27.148668 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:27.148595 2549 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 29 16:57:27.148668 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:27.148598 2549 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 29 16:57:27.148668 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:27.148644 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17e14531-cc91-412b-9804-9c1c9fe52eb8-cert podName:17e14531-cc91-412b-9804-9c1c9fe52eb8 nodeName:}" failed. No retries permitted until 2026-05-29 16:57:31.14863153 +0000 UTC m=+41.635060716 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/17e14531-cc91-412b-9804-9c1c9fe52eb8-cert") pod "ingress-canary-wnqbv" (UID: "17e14531-cc91-412b-9804-9c1c9fe52eb8") : secret "canary-serving-cert" not found May 29 16:57:27.148668 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:27.148651 2549 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76f776b797-lhxhm: secret "image-registry-tls" not found May 29 16:57:27.148790 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:27.148693 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f35fbbaf-13e2-4a63-865d-00375eb46e97-registry-tls podName:f35fbbaf-13e2-4a63-865d-00375eb46e97 nodeName:}" failed. No retries permitted until 2026-05-29 16:57:31.148680089 +0000 UTC m=+41.635109273 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f35fbbaf-13e2-4a63-865d-00375eb46e97-registry-tls") pod "image-registry-76f776b797-lhxhm" (UID: "f35fbbaf-13e2-4a63-865d-00375eb46e97") : secret "image-registry-tls" not found May 29 16:57:27.279245 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:27.279215 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-52zq6" event={"ID":"56335f96-b27f-4a52-beb7-656001cdea57","Type":"ContainerStarted","Data":"1fd24059d8bf993213585f3b9f9a35efc4c280ba88656d9150b3c0e2bf270ab8"} May 29 16:57:27.298488 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:27.298444 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-52zq6" podStartSLOduration=4.393964973 podStartE2EDuration="37.298430826s" podCreationTimestamp="2026-05-29 16:56:50 +0000 UTC" firstStartedPulling="2026-05-29 16:56:51.334332252 +0000 UTC m=+1.820761445" lastFinishedPulling="2026-05-29 16:57:24.238798116 +0000 UTC m=+34.725227298" observedRunningTime="2026-05-29 16:57:27.297654344 +0000 UTC m=+37.784083548" watchObservedRunningTime="2026-05-29 16:57:27.298430826 +0000 UTC m=+37.784860028" May 29 16:57:31.178884 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:31.178835 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-z94bv_5c547c68-54a9-4af7-afb9-494d1a1b6a03/dns-node-resolver/0.log" May 29 16:57:31.181122 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:31.181101 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f35fbbaf-13e2-4a63-865d-00375eb46e97-registry-tls\") pod \"image-registry-76f776b797-lhxhm\" (UID: \"f35fbbaf-13e2-4a63-865d-00375eb46e97\") " pod="openshift-image-registry/image-registry-76f776b797-lhxhm" May 29 16:57:31.181176 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:31.181131 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bca913b1-2d98-44c7-906e-bdb4c8450658-metrics-tls\") pod \"dns-default-kzj26\" (UID: \"bca913b1-2d98-44c7-906e-bdb4c8450658\") " pod="openshift-dns/dns-default-kzj26" May 29 16:57:31.181176 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:31.181163 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/17e14531-cc91-412b-9804-9c1c9fe52eb8-cert\") pod \"ingress-canary-wnqbv\" (UID: \"17e14531-cc91-412b-9804-9c1c9fe52eb8\") " pod="openshift-ingress-canary/ingress-canary-wnqbv" May 29 16:57:31.181250 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:31.181235 2549 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 29 16:57:31.181299 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:31.181291 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bca913b1-2d98-44c7-906e-bdb4c8450658-metrics-tls podName:bca913b1-2d98-44c7-906e-bdb4c8450658 nodeName:}" failed. No retries permitted until 2026-05-29 16:57:39.181276394 +0000 UTC m=+49.667705573 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bca913b1-2d98-44c7-906e-bdb4c8450658-metrics-tls") pod "dns-default-kzj26" (UID: "bca913b1-2d98-44c7-906e-bdb4c8450658") : secret "dns-default-metrics-tls" not found May 29 16:57:31.181347 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:31.181238 2549 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 29 16:57:31.181379 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:31.181360 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17e14531-cc91-412b-9804-9c1c9fe52eb8-cert podName:17e14531-cc91-412b-9804-9c1c9fe52eb8 nodeName:}" failed. No retries permitted until 2026-05-29 16:57:39.181348207 +0000 UTC m=+49.667777393 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/17e14531-cc91-412b-9804-9c1c9fe52eb8-cert") pod "ingress-canary-wnqbv" (UID: "17e14531-cc91-412b-9804-9c1c9fe52eb8") : secret "canary-serving-cert" not found May 29 16:57:31.181379 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:31.181240 2549 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 29 16:57:31.181379 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:31.181376 2549 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76f776b797-lhxhm: secret "image-registry-tls" not found May 29 16:57:31.181470 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:31.181400 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f35fbbaf-13e2-4a63-865d-00375eb46e97-registry-tls podName:f35fbbaf-13e2-4a63-865d-00375eb46e97 nodeName:}" failed. No retries permitted until 2026-05-29 16:57:39.181390532 +0000 UTC m=+49.667819712 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f35fbbaf-13e2-4a63-865d-00375eb46e97-registry-tls") pod "image-registry-76f776b797-lhxhm" (UID: "f35fbbaf-13e2-4a63-865d-00375eb46e97") : secret "image-registry-tls" not found May 29 16:57:32.179508 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:32.179478 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qdvbn_962b0454-8569-4239-b4b3-b88b0fe86570/node-ca/0.log" May 29 16:57:32.530989 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:32.530957 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-n9wt9"] May 29 16:57:32.557760 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:32.557731 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-n9wt9"] May 29 16:57:32.557916 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:32.557867 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-n9wt9" May 29 16:57:32.562797 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:32.562772 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" May 29 16:57:32.563397 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:32.563368 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" May 29 16:57:32.563397 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:32.563395 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kt6kk\"" May 29 16:57:32.563564 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:32.563433 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" May 29 16:57:32.564085 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:32.564057 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" May 29 16:57:32.692173 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:32.692130 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d59b7498-802e-4d3b-85ed-54913e4bc2ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n9wt9\" (UID: \"d59b7498-802e-4d3b-85ed-54913e4bc2ea\") " pod="openshift-insights/insights-runtime-extractor-n9wt9" May 29 16:57:32.692173 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:32.692175 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlksp\" (UniqueName: \"kubernetes.io/projected/d59b7498-802e-4d3b-85ed-54913e4bc2ea-kube-api-access-hlksp\") pod \"insights-runtime-extractor-n9wt9\" (UID: \"d59b7498-802e-4d3b-85ed-54913e4bc2ea\") " pod="openshift-insights/insights-runtime-extractor-n9wt9" May 29 16:57:32.692376 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:32.692194 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d59b7498-802e-4d3b-85ed-54913e4bc2ea-data-volume\") pod \"insights-runtime-extractor-n9wt9\" (UID: \"d59b7498-802e-4d3b-85ed-54913e4bc2ea\") " pod="openshift-insights/insights-runtime-extractor-n9wt9" May 29 16:57:32.692376 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:32.692211 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d59b7498-802e-4d3b-85ed-54913e4bc2ea-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n9wt9\" (UID: \"d59b7498-802e-4d3b-85ed-54913e4bc2ea\") " pod="openshift-insights/insights-runtime-extractor-n9wt9" May 29 16:57:32.692376 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:32.692287 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d59b7498-802e-4d3b-85ed-54913e4bc2ea-crio-socket\") pod \"insights-runtime-extractor-n9wt9\" (UID: \"d59b7498-802e-4d3b-85ed-54913e4bc2ea\") " pod="openshift-insights/insights-runtime-extractor-n9wt9" May 29 16:57:32.793219 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:32.793152 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d59b7498-802e-4d3b-85ed-54913e4bc2ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n9wt9\" (UID: \"d59b7498-802e-4d3b-85ed-54913e4bc2ea\") " pod="openshift-insights/insights-runtime-extractor-n9wt9" May 29 16:57:32.793219 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:32.793191 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hlksp\" (UniqueName: \"kubernetes.io/projected/d59b7498-802e-4d3b-85ed-54913e4bc2ea-kube-api-access-hlksp\") pod \"insights-runtime-extractor-n9wt9\" (UID: \"d59b7498-802e-4d3b-85ed-54913e4bc2ea\") " pod="openshift-insights/insights-runtime-extractor-n9wt9" May 29 16:57:32.793219 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:32.793208 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d59b7498-802e-4d3b-85ed-54913e4bc2ea-data-volume\") pod \"insights-runtime-extractor-n9wt9\" (UID: \"d59b7498-802e-4d3b-85ed-54913e4bc2ea\") " pod="openshift-insights/insights-runtime-extractor-n9wt9" May 29 16:57:32.793377 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:32.793233 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d59b7498-802e-4d3b-85ed-54913e4bc2ea-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n9wt9\" (UID: \"d59b7498-802e-4d3b-85ed-54913e4bc2ea\") " pod="openshift-insights/insights-runtime-extractor-n9wt9" May 29 16:57:32.793377 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:32.793269 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d59b7498-802e-4d3b-85ed-54913e4bc2ea-crio-socket\") pod \"insights-runtime-extractor-n9wt9\" (UID: \"d59b7498-802e-4d3b-85ed-54913e4bc2ea\") " pod="openshift-insights/insights-runtime-extractor-n9wt9" May 29 16:57:32.793377 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:32.793308 2549 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found May 29 16:57:32.793488 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:32.793390 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d59b7498-802e-4d3b-85ed-54913e4bc2ea-insights-runtime-extractor-tls podName:d59b7498-802e-4d3b-85ed-54913e4bc2ea nodeName:}" failed. No retries permitted until 2026-05-29 16:57:33.293368639 +0000 UTC m=+43.779797838 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d59b7498-802e-4d3b-85ed-54913e4bc2ea-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n9wt9" (UID: "d59b7498-802e-4d3b-85ed-54913e4bc2ea") : secret "insights-runtime-extractor-tls" not found May 29 16:57:32.793542 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:32.793528 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d59b7498-802e-4d3b-85ed-54913e4bc2ea-crio-socket\") pod \"insights-runtime-extractor-n9wt9\" (UID: \"d59b7498-802e-4d3b-85ed-54913e4bc2ea\") " pod="openshift-insights/insights-runtime-extractor-n9wt9" May 29 16:57:32.793595 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:32.793579 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d59b7498-802e-4d3b-85ed-54913e4bc2ea-data-volume\") pod \"insights-runtime-extractor-n9wt9\" (UID: \"d59b7498-802e-4d3b-85ed-54913e4bc2ea\") " pod="openshift-insights/insights-runtime-extractor-n9wt9" May 29 16:57:32.793868 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:32.793830 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d59b7498-802e-4d3b-85ed-54913e4bc2ea-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n9wt9\" (UID: \"d59b7498-802e-4d3b-85ed-54913e4bc2ea\") " pod="openshift-insights/insights-runtime-extractor-n9wt9" May 29 16:57:32.805137 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:32.805105 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlksp\" (UniqueName: \"kubernetes.io/projected/d59b7498-802e-4d3b-85ed-54913e4bc2ea-kube-api-access-hlksp\") pod \"insights-runtime-extractor-n9wt9\" (UID: \"d59b7498-802e-4d3b-85ed-54913e4bc2ea\") " pod="openshift-insights/insights-runtime-extractor-n9wt9" May 29 16:57:33.297314 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:33.297283 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eb980098-9793-4340-b57d-b0600c17b07b-original-pull-secret\") pod \"global-pull-secret-syncer-dhvms\" (UID: \"eb980098-9793-4340-b57d-b0600c17b07b\") " pod="kube-system/global-pull-secret-syncer-dhvms" May 29 16:57:33.297679 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:33.297373 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d59b7498-802e-4d3b-85ed-54913e4bc2ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n9wt9\" (UID: \"d59b7498-802e-4d3b-85ed-54913e4bc2ea\") " pod="openshift-insights/insights-runtime-extractor-n9wt9" May 29 16:57:33.297679 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:33.297502 2549 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found May 29 16:57:33.297679 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:33.297587 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d59b7498-802e-4d3b-85ed-54913e4bc2ea-insights-runtime-extractor-tls podName:d59b7498-802e-4d3b-85ed-54913e4bc2ea nodeName:}" failed. No retries permitted until 2026-05-29 16:57:34.297570121 +0000 UTC m=+44.783999307 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d59b7498-802e-4d3b-85ed-54913e4bc2ea-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n9wt9" (UID: "d59b7498-802e-4d3b-85ed-54913e4bc2ea") : secret "insights-runtime-extractor-tls" not found May 29 16:57:33.299500 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:33.299482 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eb980098-9793-4340-b57d-b0600c17b07b-original-pull-secret\") pod \"global-pull-secret-syncer-dhvms\" (UID: \"eb980098-9793-4340-b57d-b0600c17b07b\") " pod="kube-system/global-pull-secret-syncer-dhvms" May 29 16:57:33.501062 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:33.501025 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dhvms" May 29 16:57:33.670373 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:33.670344 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dhvms"] May 29 16:57:33.673285 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:57:33.673255 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb980098_9793_4340_b57d_b0600c17b07b.slice/crio-77739993d30a08b1728aa81499d9ec86d2e4af5661dfd1652e349f15143522d7 WatchSource:0}: Error finding container 77739993d30a08b1728aa81499d9ec86d2e4af5661dfd1652e349f15143522d7: Status 404 returned error can't find the container with id 77739993d30a08b1728aa81499d9ec86d2e4af5661dfd1652e349f15143522d7 May 29 16:57:34.294339 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:34.294302 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dhvms" event={"ID":"eb980098-9793-4340-b57d-b0600c17b07b","Type":"ContainerStarted","Data":"77739993d30a08b1728aa81499d9ec86d2e4af5661dfd1652e349f15143522d7"} May 29 16:57:34.305747 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:34.305721 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d59b7498-802e-4d3b-85ed-54913e4bc2ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n9wt9\" (UID: \"d59b7498-802e-4d3b-85ed-54913e4bc2ea\") " pod="openshift-insights/insights-runtime-extractor-n9wt9" May 29 16:57:34.306128 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:34.305860 2549 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found May 29 16:57:34.306128 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:34.305927 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d59b7498-802e-4d3b-85ed-54913e4bc2ea-insights-runtime-extractor-tls podName:d59b7498-802e-4d3b-85ed-54913e4bc2ea nodeName:}" failed. No retries permitted until 2026-05-29 16:57:36.30590885 +0000 UTC m=+46.792338033 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d59b7498-802e-4d3b-85ed-54913e4bc2ea-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n9wt9" (UID: "d59b7498-802e-4d3b-85ed-54913e4bc2ea") : secret "insights-runtime-extractor-tls" not found May 29 16:57:36.321449 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:36.321408 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d59b7498-802e-4d3b-85ed-54913e4bc2ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n9wt9\" (UID: \"d59b7498-802e-4d3b-85ed-54913e4bc2ea\") " pod="openshift-insights/insights-runtime-extractor-n9wt9" May 29 16:57:36.321893 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:36.321579 2549 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found May 29 16:57:36.321893 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:36.321666 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d59b7498-802e-4d3b-85ed-54913e4bc2ea-insights-runtime-extractor-tls podName:d59b7498-802e-4d3b-85ed-54913e4bc2ea nodeName:}" failed. No retries permitted until 2026-05-29 16:57:40.321645143 +0000 UTC m=+50.808074323 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d59b7498-802e-4d3b-85ed-54913e4bc2ea-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n9wt9" (UID: "d59b7498-802e-4d3b-85ed-54913e4bc2ea") : secret "insights-runtime-extractor-tls" not found May 29 16:57:38.303567 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:38.303533 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dhvms" event={"ID":"eb980098-9793-4340-b57d-b0600c17b07b","Type":"ContainerStarted","Data":"330baaa059a190bdc899d2cfb7c00f7e9d4f898c433eb087adc7490c1dae8278"} May 29 16:57:38.317664 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:38.317612 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-dhvms" podStartSLOduration=33.496992887 podStartE2EDuration="37.317599527s" podCreationTimestamp="2026-05-29 16:57:01 +0000 UTC" firstStartedPulling="2026-05-29 16:57:33.674858351 +0000 UTC m=+44.161287534" lastFinishedPulling="2026-05-29 16:57:37.495464991 +0000 UTC m=+47.981894174" observedRunningTime="2026-05-29 16:57:38.317397712 +0000 UTC m=+48.803826915" watchObservedRunningTime="2026-05-29 16:57:38.317599527 +0000 UTC m=+48.804028725" May 29 16:57:39.248546 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:39.248503 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f35fbbaf-13e2-4a63-865d-00375eb46e97-registry-tls\") pod \"image-registry-76f776b797-lhxhm\" (UID: \"f35fbbaf-13e2-4a63-865d-00375eb46e97\") " pod="openshift-image-registry/image-registry-76f776b797-lhxhm" May 29 16:57:39.248546 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:39.248550 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bca913b1-2d98-44c7-906e-bdb4c8450658-metrics-tls\") pod \"dns-default-kzj26\" (UID: \"bca913b1-2d98-44c7-906e-bdb4c8450658\") " pod="openshift-dns/dns-default-kzj26" May 29 16:57:39.248818 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:39.248590 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/17e14531-cc91-412b-9804-9c1c9fe52eb8-cert\") pod \"ingress-canary-wnqbv\" (UID: \"17e14531-cc91-412b-9804-9c1c9fe52eb8\") " pod="openshift-ingress-canary/ingress-canary-wnqbv" May 29 16:57:39.248818 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:39.248658 2549 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 29 16:57:39.248818 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:39.248681 2549 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76f776b797-lhxhm: secret "image-registry-tls" not found May 29 16:57:39.248818 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:39.248704 2549 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 29 16:57:39.248818 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:39.248704 2549 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 29 16:57:39.248818 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:39.248737 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f35fbbaf-13e2-4a63-865d-00375eb46e97-registry-tls podName:f35fbbaf-13e2-4a63-865d-00375eb46e97 nodeName:}" failed. No retries permitted until 2026-05-29 16:57:55.248721148 +0000 UTC m=+65.735150331 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f35fbbaf-13e2-4a63-865d-00375eb46e97-registry-tls") pod "image-registry-76f776b797-lhxhm" (UID: "f35fbbaf-13e2-4a63-865d-00375eb46e97") : secret "image-registry-tls" not found May 29 16:57:39.248818 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:39.248781 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17e14531-cc91-412b-9804-9c1c9fe52eb8-cert podName:17e14531-cc91-412b-9804-9c1c9fe52eb8 nodeName:}" failed. No retries permitted until 2026-05-29 16:57:55.248765241 +0000 UTC m=+65.735194424 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/17e14531-cc91-412b-9804-9c1c9fe52eb8-cert") pod "ingress-canary-wnqbv" (UID: "17e14531-cc91-412b-9804-9c1c9fe52eb8") : secret "canary-serving-cert" not found May 29 16:57:39.248818 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:39.248804 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bca913b1-2d98-44c7-906e-bdb4c8450658-metrics-tls podName:bca913b1-2d98-44c7-906e-bdb4c8450658 nodeName:}" failed. No retries permitted until 2026-05-29 16:57:55.2487946 +0000 UTC m=+65.735223786 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bca913b1-2d98-44c7-906e-bdb4c8450658-metrics-tls") pod "dns-default-kzj26" (UID: "bca913b1-2d98-44c7-906e-bdb4c8450658") : secret "dns-default-metrics-tls" not found May 29 16:57:40.355935 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:40.355895 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d59b7498-802e-4d3b-85ed-54913e4bc2ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n9wt9\" (UID: \"d59b7498-802e-4d3b-85ed-54913e4bc2ea\") " pod="openshift-insights/insights-runtime-extractor-n9wt9" May 29 16:57:40.356424 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:40.356034 2549 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found May 29 16:57:40.356424 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:40.356094 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d59b7498-802e-4d3b-85ed-54913e4bc2ea-insights-runtime-extractor-tls podName:d59b7498-802e-4d3b-85ed-54913e4bc2ea nodeName:}" failed. No retries permitted until 2026-05-29 16:57:48.356076084 +0000 UTC m=+58.842505266 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d59b7498-802e-4d3b-85ed-54913e4bc2ea-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n9wt9" (UID: "d59b7498-802e-4d3b-85ed-54913e4bc2ea") : secret "insights-runtime-extractor-tls" not found May 29 16:57:48.417110 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:48.417071 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d59b7498-802e-4d3b-85ed-54913e4bc2ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n9wt9\" (UID: \"d59b7498-802e-4d3b-85ed-54913e4bc2ea\") " pod="openshift-insights/insights-runtime-extractor-n9wt9" May 29 16:57:48.417600 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:48.417222 2549 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found May 29 16:57:48.417600 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:57:48.417293 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d59b7498-802e-4d3b-85ed-54913e4bc2ea-insights-runtime-extractor-tls podName:d59b7498-802e-4d3b-85ed-54913e4bc2ea nodeName:}" failed. No retries permitted until 2026-05-29 16:58:04.417277393 +0000 UTC m=+74.903706573 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d59b7498-802e-4d3b-85ed-54913e4bc2ea-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n9wt9" (UID: "d59b7498-802e-4d3b-85ed-54913e4bc2ea") : secret "insights-runtime-extractor-tls" not found May 29 16:57:49.269216 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:49.269190 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nfv6s" May 29 16:57:54.767500 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:54.767436 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d457970-4377-4157-9d77-81db07916925-metrics-certs\") pod \"network-metrics-daemon-ps265\" (UID: \"7d457970-4377-4157-9d77-81db07916925\") " pod="openshift-multus/network-metrics-daemon-ps265" May 29 16:57:54.769754 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:54.769737 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" May 29 16:57:54.780434 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:54.780410 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d457970-4377-4157-9d77-81db07916925-metrics-certs\") pod \"network-metrics-daemon-ps265\" (UID: \"7d457970-4377-4157-9d77-81db07916925\") " pod="openshift-multus/network-metrics-daemon-ps265" May 29 16:57:54.792227 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:54.792206 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fng4t\"" May 29 16:57:54.800805 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:54.800788 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ps265" May 29 16:57:54.868431 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:54.868402 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b6b9d\" (UniqueName: \"kubernetes.io/projected/be92fdf2-3d2b-4075-8919-8697586347c6-kube-api-access-b6b9d\") pod \"network-check-target-crvwv\" (UID: \"be92fdf2-3d2b-4075-8919-8697586347c6\") " pod="openshift-network-diagnostics/network-check-target-crvwv" May 29 16:57:54.870257 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:54.870239 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" May 29 16:57:54.880350 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:54.880330 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" May 29 16:57:54.891025 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:54.891004 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6b9d\" (UniqueName: \"kubernetes.io/projected/be92fdf2-3d2b-4075-8919-8697586347c6-kube-api-access-b6b9d\") pod \"network-check-target-crvwv\" (UID: \"be92fdf2-3d2b-4075-8919-8697586347c6\") " pod="openshift-network-diagnostics/network-check-target-crvwv" May 29 16:57:54.916072 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:54.916046 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ps265"] May 29 16:57:54.919751 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:57:54.919731 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d457970_4377_4157_9d77_81db07916925.slice/crio-2682ce80db0426dd83c49b4df88563f63924adc40037a4fba6034b0eb87f2392 WatchSource:0}: Error finding container 2682ce80db0426dd83c49b4df88563f63924adc40037a4fba6034b0eb87f2392: Status 404 returned error can't find the container with id 2682ce80db0426dd83c49b4df88563f63924adc40037a4fba6034b0eb87f2392 May 29 16:57:55.097592 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:55.097569 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gvkkc\"" May 29 16:57:55.106101 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:55.106085 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-crvwv" May 29 16:57:55.217424 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:55.217391 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-crvwv"] May 29 16:57:55.220642 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:57:55.220613 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe92fdf2_3d2b_4075_8919_8697586347c6.slice/crio-d91ca80e9e50eab704fa2d59d69770a52f78f615f00cf3e612b22211277aae81 WatchSource:0}: Error finding container d91ca80e9e50eab704fa2d59d69770a52f78f615f00cf3e612b22211277aae81: Status 404 returned error can't find the container with id d91ca80e9e50eab704fa2d59d69770a52f78f615f00cf3e612b22211277aae81 May 29 16:57:55.272370 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:55.272344 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bca913b1-2d98-44c7-906e-bdb4c8450658-metrics-tls\") pod \"dns-default-kzj26\" (UID: \"bca913b1-2d98-44c7-906e-bdb4c8450658\") " pod="openshift-dns/dns-default-kzj26" May 29 16:57:55.272468 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:55.272404 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/17e14531-cc91-412b-9804-9c1c9fe52eb8-cert\") pod \"ingress-canary-wnqbv\" (UID: \"17e14531-cc91-412b-9804-9c1c9fe52eb8\") " pod="openshift-ingress-canary/ingress-canary-wnqbv" May 29 16:57:55.272526 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:55.272510 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f35fbbaf-13e2-4a63-865d-00375eb46e97-registry-tls\") pod \"image-registry-76f776b797-lhxhm\" (UID: \"f35fbbaf-13e2-4a63-865d-00375eb46e97\") " pod="openshift-image-registry/image-registry-76f776b797-lhxhm" May 29 16:57:55.274895 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:55.274871 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bca913b1-2d98-44c7-906e-bdb4c8450658-metrics-tls\") pod \"dns-default-kzj26\" (UID: \"bca913b1-2d98-44c7-906e-bdb4c8450658\") " pod="openshift-dns/dns-default-kzj26" May 29 16:57:55.274978 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:55.274906 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f35fbbaf-13e2-4a63-865d-00375eb46e97-registry-tls\") pod \"image-registry-76f776b797-lhxhm\" (UID: \"f35fbbaf-13e2-4a63-865d-00375eb46e97\") " pod="openshift-image-registry/image-registry-76f776b797-lhxhm" May 29 16:57:55.274978 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:55.274945 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/17e14531-cc91-412b-9804-9c1c9fe52eb8-cert\") pod \"ingress-canary-wnqbv\" (UID: \"17e14531-cc91-412b-9804-9c1c9fe52eb8\") " pod="openshift-ingress-canary/ingress-canary-wnqbv" May 29 16:57:55.336701 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:55.336664 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-crvwv" event={"ID":"be92fdf2-3d2b-4075-8919-8697586347c6","Type":"ContainerStarted","Data":"d91ca80e9e50eab704fa2d59d69770a52f78f615f00cf3e612b22211277aae81"} May 29 16:57:55.337685 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:55.337661 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ps265" event={"ID":"7d457970-4377-4157-9d77-81db07916925","Type":"ContainerStarted","Data":"2682ce80db0426dd83c49b4df88563f63924adc40037a4fba6034b0eb87f2392"} May 29 16:57:55.474280 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:55.474196 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vq84q\"" May 29 16:57:55.483711 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:55.483681 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76f776b797-lhxhm" May 29 16:57:55.501183 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:55.501157 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xwwt6\"" May 29 16:57:55.510572 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:55.510538 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kzj26" May 29 16:57:55.525585 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:55.525559 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-fgcrt\"" May 29 16:57:55.533857 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:55.533817 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wnqbv" May 29 16:57:55.640075 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:55.640024 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76f776b797-lhxhm"] May 29 16:57:55.642399 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:57:55.642312 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf35fbbaf_13e2_4a63_865d_00375eb46e97.slice/crio-1d24da7d724874cb2563253a598fb38cfbf824b0cfdd215348a5945e437caa3c WatchSource:0}: Error finding container 1d24da7d724874cb2563253a598fb38cfbf824b0cfdd215348a5945e437caa3c: Status 404 returned error can't find the container with id 1d24da7d724874cb2563253a598fb38cfbf824b0cfdd215348a5945e437caa3c May 29 16:57:55.662662 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:55.662616 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-kzj26"] May 29 16:57:55.673136 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:57:55.673099 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbca913b1_2d98_44c7_906e_bdb4c8450658.slice/crio-3202112a432f1bbe2dc511be6ac07e295916614b462c194c349e513b29c1f912 WatchSource:0}: Error finding container 3202112a432f1bbe2dc511be6ac07e295916614b462c194c349e513b29c1f912: Status 404 returned error can't find the container with id 3202112a432f1bbe2dc511be6ac07e295916614b462c194c349e513b29c1f912 May 29 16:57:55.696055 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:55.696001 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wnqbv"] May 29 16:57:55.700182 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:57:55.700154 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17e14531_cc91_412b_9804_9c1c9fe52eb8.slice/crio-74d220b320f71ffcef964f01f8ec0cdfd06273687dec94ebf27a08cd6ca07002 WatchSource:0}: Error finding container 74d220b320f71ffcef964f01f8ec0cdfd06273687dec94ebf27a08cd6ca07002: Status 404 returned error can't find the container with id 74d220b320f71ffcef964f01f8ec0cdfd06273687dec94ebf27a08cd6ca07002 May 29 16:57:56.343821 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:56.343727 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wnqbv" event={"ID":"17e14531-cc91-412b-9804-9c1c9fe52eb8","Type":"ContainerStarted","Data":"74d220b320f71ffcef964f01f8ec0cdfd06273687dec94ebf27a08cd6ca07002"} May 29 16:57:56.345832 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:56.345785 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kzj26" event={"ID":"bca913b1-2d98-44c7-906e-bdb4c8450658","Type":"ContainerStarted","Data":"3202112a432f1bbe2dc511be6ac07e295916614b462c194c349e513b29c1f912"} May 29 16:57:56.349038 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:56.349006 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76f776b797-lhxhm" event={"ID":"f35fbbaf-13e2-4a63-865d-00375eb46e97","Type":"ContainerStarted","Data":"a9320ff79397aca4caa084e2f7f63c4a15e53e0ffe87b8ef993dce9d76d86fa9"} May 29 16:57:56.349142 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:56.349046 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76f776b797-lhxhm" event={"ID":"f35fbbaf-13e2-4a63-865d-00375eb46e97","Type":"ContainerStarted","Data":"1d24da7d724874cb2563253a598fb38cfbf824b0cfdd215348a5945e437caa3c"} May 29 16:57:56.349526 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:56.349500 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-76f776b797-lhxhm" May 29 16:57:56.353203 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:56.353162 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ps265" event={"ID":"7d457970-4377-4157-9d77-81db07916925","Type":"ContainerStarted","Data":"b03f8d1fc23526780e3f47f7e4ab5cc5f15db32f84630fca31ce3a3b0c15fe53"} May 29 16:57:56.368622 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:56.368560 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-76f776b797-lhxhm" podStartSLOduration=66.368483834 podStartE2EDuration="1m6.368483834s" podCreationTimestamp="2026-05-29 16:56:50 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 16:57:56.367736772 +0000 UTC m=+66.854165975" watchObservedRunningTime="2026-05-29 16:57:56.368483834 +0000 UTC m=+66.854913037" May 29 16:57:57.356895 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:57.356829 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ps265" event={"ID":"7d457970-4377-4157-9d77-81db07916925","Type":"ContainerStarted","Data":"6b386d0739f8bf074f70fe4aa3dbd7d4888e64b60f8cf278231887eb2148f52c"} May 29 16:57:57.371338 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:57:57.371291 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ps265" podStartSLOduration=66.141462971 podStartE2EDuration="1m7.371274027s" podCreationTimestamp="2026-05-29 16:56:50 +0000 UTC" firstStartedPulling="2026-05-29 16:57:54.921628451 +0000 UTC m=+65.408057631" lastFinishedPulling="2026-05-29 16:57:56.151439508 +0000 UTC m=+66.637868687" observedRunningTime="2026-05-29 16:57:57.370867908 +0000 UTC m=+67.857297111" watchObservedRunningTime="2026-05-29 16:57:57.371274027 +0000 UTC m=+67.857703230" May 29 16:58:00.365087 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:00.365051 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wnqbv" event={"ID":"17e14531-cc91-412b-9804-9c1c9fe52eb8","Type":"ContainerStarted","Data":"7cdc300d6c62f7a52f41b285cbce2e8dad2f17f0197c1f14a7350234e68d252a"} May 29 16:58:00.366470 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:00.366445 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-crvwv" event={"ID":"be92fdf2-3d2b-4075-8919-8697586347c6","Type":"ContainerStarted","Data":"1840d8763d86818128d3accfbf3551c11eb7f5a4c404604ca1007e61e4d40329"} May 29 16:58:00.366595 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:00.366579 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-crvwv" May 29 16:58:00.368006 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:00.367984 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kzj26" event={"ID":"bca913b1-2d98-44c7-906e-bdb4c8450658","Type":"ContainerStarted","Data":"8769e5d0c8a141a6c55fd8b04b1d26161050586bfbbfe83584ffda9556f4c754"} May 29 16:58:00.368113 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:00.368010 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kzj26" event={"ID":"bca913b1-2d98-44c7-906e-bdb4c8450658","Type":"ContainerStarted","Data":"dbb316967b247845e2f8092f841bb6dd214ea1cac07972e4b1d4ad5bc971f6e0"} May 29 16:58:00.368159 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:00.368132 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-kzj26" May 29 16:58:00.380198 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:00.380155 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-wnqbv" podStartSLOduration=33.685528255 podStartE2EDuration="37.380144043s" podCreationTimestamp="2026-05-29 16:57:23 +0000 UTC" firstStartedPulling="2026-05-29 16:57:55.703424229 +0000 UTC m=+66.189853428" lastFinishedPulling="2026-05-29 16:57:59.398040037 +0000 UTC m=+69.884469216" observedRunningTime="2026-05-29 16:58:00.379030199 +0000 UTC m=+70.865459400" watchObservedRunningTime="2026-05-29 16:58:00.380144043 +0000 UTC m=+70.866573244" May 29 16:58:00.392128 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:00.392079 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-crvwv" podStartSLOduration=66.206920336 podStartE2EDuration="1m10.392069414s" podCreationTimestamp="2026-05-29 16:56:50 +0000 UTC" firstStartedPulling="2026-05-29 16:57:55.222484199 +0000 UTC m=+65.708913378" lastFinishedPulling="2026-05-29 16:57:59.407633271 +0000 UTC m=+69.894062456" observedRunningTime="2026-05-29 16:58:00.391243618 +0000 UTC m=+70.877672819" watchObservedRunningTime="2026-05-29 16:58:00.392069414 +0000 UTC m=+70.878498615" May 29 16:58:00.404925 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:00.404883 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-kzj26" podStartSLOduration=33.682661702 podStartE2EDuration="37.404874419s" podCreationTimestamp="2026-05-29 16:57:23 +0000 UTC" firstStartedPulling="2026-05-29 16:57:55.675758945 +0000 UTC m=+66.162188132" lastFinishedPulling="2026-05-29 16:57:59.397971653 +0000 UTC m=+69.884400849" observedRunningTime="2026-05-29 16:58:00.404465662 +0000 UTC m=+70.890894865" watchObservedRunningTime="2026-05-29 16:58:00.404874419 +0000 UTC m=+70.891303619" May 29 16:58:02.812740 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:02.812706 2549 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-76f776b797-lhxhm"] May 29 16:58:02.815914 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:02.815893 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-76c9bb5fdf-n6n9z"] May 29 16:58:02.818360 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:02.818343 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-n6n9z" May 29 16:58:02.820238 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:02.820220 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" May 29 16:58:02.820318 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:02.820222 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" May 29 16:58:02.820318 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:02.820222 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-8gm97\"" May 29 16:58:02.828926 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:02.828904 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-76c9bb5fdf-n6n9z"] May 29 16:58:02.921259 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:02.921229 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-56d4bd47c8-2rw2c"] May 29 16:58:02.923030 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:02.923016 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56d4bd47c8-2rw2c" May 29 16:58:02.929325 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:02.929305 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8921009e-fb12-4b6b-9bef-9120a210fb34-nginx-conf\") pod \"networking-console-plugin-76c9bb5fdf-n6n9z\" (UID: \"8921009e-fb12-4b6b-9bef-9120a210fb34\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-n6n9z" May 29 16:58:02.929422 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:02.929334 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8921009e-fb12-4b6b-9bef-9120a210fb34-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-n6n9z\" (UID: \"8921009e-fb12-4b6b-9bef-9120a210fb34\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-n6n9z" May 29 16:58:02.936767 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:02.936744 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56d4bd47c8-2rw2c"] May 29 16:58:03.030541 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:03.030512 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/34245c5b-a05b-46f7-817f-5da3e5af1e42-trusted-ca\") pod \"image-registry-56d4bd47c8-2rw2c\" (UID: \"34245c5b-a05b-46f7-817f-5da3e5af1e42\") " pod="openshift-image-registry/image-registry-56d4bd47c8-2rw2c" May 29 16:58:03.030668 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:03.030544 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/34245c5b-a05b-46f7-817f-5da3e5af1e42-installation-pull-secrets\") pod \"image-registry-56d4bd47c8-2rw2c\" (UID: \"34245c5b-a05b-46f7-817f-5da3e5af1e42\") " pod="openshift-image-registry/image-registry-56d4bd47c8-2rw2c" May 29 16:58:03.030668 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:03.030566 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/34245c5b-a05b-46f7-817f-5da3e5af1e42-registry-tls\") pod \"image-registry-56d4bd47c8-2rw2c\" (UID: \"34245c5b-a05b-46f7-817f-5da3e5af1e42\") " pod="openshift-image-registry/image-registry-56d4bd47c8-2rw2c" May 29 16:58:03.030668 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:03.030603 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/34245c5b-a05b-46f7-817f-5da3e5af1e42-ca-trust-extracted\") pod \"image-registry-56d4bd47c8-2rw2c\" (UID: \"34245c5b-a05b-46f7-817f-5da3e5af1e42\") " pod="openshift-image-registry/image-registry-56d4bd47c8-2rw2c" May 29 16:58:03.030668 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:03.030632 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/34245c5b-a05b-46f7-817f-5da3e5af1e42-bound-sa-token\") pod \"image-registry-56d4bd47c8-2rw2c\" (UID: \"34245c5b-a05b-46f7-817f-5da3e5af1e42\") " pod="openshift-image-registry/image-registry-56d4bd47c8-2rw2c" May 29 16:58:03.030668 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:03.030669 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbj6t\" (UniqueName: \"kubernetes.io/projected/34245c5b-a05b-46f7-817f-5da3e5af1e42-kube-api-access-xbj6t\") pod \"image-registry-56d4bd47c8-2rw2c\" (UID: \"34245c5b-a05b-46f7-817f-5da3e5af1e42\") " pod="openshift-image-registry/image-registry-56d4bd47c8-2rw2c" May 29 16:58:03.030933 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:03.030695 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/34245c5b-a05b-46f7-817f-5da3e5af1e42-image-registry-private-configuration\") pod \"image-registry-56d4bd47c8-2rw2c\" (UID: \"34245c5b-a05b-46f7-817f-5da3e5af1e42\") " pod="openshift-image-registry/image-registry-56d4bd47c8-2rw2c" May 29 16:58:03.030933 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:03.030724 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8921009e-fb12-4b6b-9bef-9120a210fb34-nginx-conf\") pod \"networking-console-plugin-76c9bb5fdf-n6n9z\" (UID: \"8921009e-fb12-4b6b-9bef-9120a210fb34\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-n6n9z" May 29 16:58:03.030933 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:03.030773 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8921009e-fb12-4b6b-9bef-9120a210fb34-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-n6n9z\" (UID: \"8921009e-fb12-4b6b-9bef-9120a210fb34\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-n6n9z" May 29 16:58:03.030933 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:03.030795 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/34245c5b-a05b-46f7-817f-5da3e5af1e42-registry-certificates\") pod \"image-registry-56d4bd47c8-2rw2c\" (UID: \"34245c5b-a05b-46f7-817f-5da3e5af1e42\") " pod="openshift-image-registry/image-registry-56d4bd47c8-2rw2c" May 29 16:58:03.031284 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:03.031269 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8921009e-fb12-4b6b-9bef-9120a210fb34-nginx-conf\") pod \"networking-console-plugin-76c9bb5fdf-n6n9z\" (UID: \"8921009e-fb12-4b6b-9bef-9120a210fb34\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-n6n9z" May 29 16:58:03.032995 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:03.032977 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8921009e-fb12-4b6b-9bef-9120a210fb34-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-n6n9z\" (UID: \"8921009e-fb12-4b6b-9bef-9120a210fb34\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-n6n9z" May 29 16:58:03.127365 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:03.127294 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-n6n9z" May 29 16:58:03.131051 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:03.131030 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/34245c5b-a05b-46f7-817f-5da3e5af1e42-image-registry-private-configuration\") pod \"image-registry-56d4bd47c8-2rw2c\" (UID: \"34245c5b-a05b-46f7-817f-5da3e5af1e42\") " pod="openshift-image-registry/image-registry-56d4bd47c8-2rw2c" May 29 16:58:03.131162 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:03.131080 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/34245c5b-a05b-46f7-817f-5da3e5af1e42-registry-certificates\") pod \"image-registry-56d4bd47c8-2rw2c\" (UID: \"34245c5b-a05b-46f7-817f-5da3e5af1e42\") " pod="openshift-image-registry/image-registry-56d4bd47c8-2rw2c" May 29 16:58:03.131162 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:03.131119 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/34245c5b-a05b-46f7-817f-5da3e5af1e42-trusted-ca\") pod \"image-registry-56d4bd47c8-2rw2c\" (UID: \"34245c5b-a05b-46f7-817f-5da3e5af1e42\") " pod="openshift-image-registry/image-registry-56d4bd47c8-2rw2c" May 29 16:58:03.131162 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:03.131155 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/34245c5b-a05b-46f7-817f-5da3e5af1e42-installation-pull-secrets\") pod \"image-registry-56d4bd47c8-2rw2c\" (UID: \"34245c5b-a05b-46f7-817f-5da3e5af1e42\") " pod="openshift-image-registry/image-registry-56d4bd47c8-2rw2c" May 29 16:58:03.131321 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:03.131185 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/34245c5b-a05b-46f7-817f-5da3e5af1e42-registry-tls\") pod \"image-registry-56d4bd47c8-2rw2c\" (UID: \"34245c5b-a05b-46f7-817f-5da3e5af1e42\") " pod="openshift-image-registry/image-registry-56d4bd47c8-2rw2c" May 29 16:58:03.131321 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:03.131214 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/34245c5b-a05b-46f7-817f-5da3e5af1e42-ca-trust-extracted\") pod \"image-registry-56d4bd47c8-2rw2c\" (UID: \"34245c5b-a05b-46f7-817f-5da3e5af1e42\") " pod="openshift-image-registry/image-registry-56d4bd47c8-2rw2c" May 29 16:58:03.131321 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:03.131244 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/34245c5b-a05b-46f7-817f-5da3e5af1e42-bound-sa-token\") pod \"image-registry-56d4bd47c8-2rw2c\" (UID: \"34245c5b-a05b-46f7-817f-5da3e5af1e42\") " pod="openshift-image-registry/image-registry-56d4bd47c8-2rw2c" May 29 16:58:03.131321 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:03.131291 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xbj6t\" (UniqueName: \"kubernetes.io/projected/34245c5b-a05b-46f7-817f-5da3e5af1e42-kube-api-access-xbj6t\") pod \"image-registry-56d4bd47c8-2rw2c\" (UID: \"34245c5b-a05b-46f7-817f-5da3e5af1e42\") " pod="openshift-image-registry/image-registry-56d4bd47c8-2rw2c" May 29 16:58:03.131941 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:03.131886 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/34245c5b-a05b-46f7-817f-5da3e5af1e42-ca-trust-extracted\") pod \"image-registry-56d4bd47c8-2rw2c\" (UID: \"34245c5b-a05b-46f7-817f-5da3e5af1e42\") " pod="openshift-image-registry/image-registry-56d4bd47c8-2rw2c" May 29 16:58:03.132036 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:03.132008 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/34245c5b-a05b-46f7-817f-5da3e5af1e42-registry-certificates\") pod \"image-registry-56d4bd47c8-2rw2c\" (UID: \"34245c5b-a05b-46f7-817f-5da3e5af1e42\") " pod="openshift-image-registry/image-registry-56d4bd47c8-2rw2c" May 29 16:58:03.132194 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:03.132165 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/34245c5b-a05b-46f7-817f-5da3e5af1e42-trusted-ca\") pod \"image-registry-56d4bd47c8-2rw2c\" (UID: \"34245c5b-a05b-46f7-817f-5da3e5af1e42\") " pod="openshift-image-registry/image-registry-56d4bd47c8-2rw2c" May 29 16:58:03.133812 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:03.133787 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/34245c5b-a05b-46f7-817f-5da3e5af1e42-installation-pull-secrets\") pod \"image-registry-56d4bd47c8-2rw2c\" (UID: \"34245c5b-a05b-46f7-817f-5da3e5af1e42\") " pod="openshift-image-registry/image-registry-56d4bd47c8-2rw2c" May 29 16:58:03.133916 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:03.133873 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/34245c5b-a05b-46f7-817f-5da3e5af1e42-image-registry-private-configuration\") pod \"image-registry-56d4bd47c8-2rw2c\" (UID: \"34245c5b-a05b-46f7-817f-5da3e5af1e42\") " pod="openshift-image-registry/image-registry-56d4bd47c8-2rw2c" May 29 16:58:03.134083 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:03.134063 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/34245c5b-a05b-46f7-817f-5da3e5af1e42-registry-tls\") pod \"image-registry-56d4bd47c8-2rw2c\" (UID: \"34245c5b-a05b-46f7-817f-5da3e5af1e42\") " pod="openshift-image-registry/image-registry-56d4bd47c8-2rw2c" May 29 16:58:03.139384 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:03.139360 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbj6t\" (UniqueName: \"kubernetes.io/projected/34245c5b-a05b-46f7-817f-5da3e5af1e42-kube-api-access-xbj6t\") pod \"image-registry-56d4bd47c8-2rw2c\" (UID: \"34245c5b-a05b-46f7-817f-5da3e5af1e42\") " pod="openshift-image-registry/image-registry-56d4bd47c8-2rw2c" May 29 16:58:03.141721 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:03.141702 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/34245c5b-a05b-46f7-817f-5da3e5af1e42-bound-sa-token\") pod \"image-registry-56d4bd47c8-2rw2c\" (UID: \"34245c5b-a05b-46f7-817f-5da3e5af1e42\") " pod="openshift-image-registry/image-registry-56d4bd47c8-2rw2c" May 29 16:58:03.231377 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:03.231348 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56d4bd47c8-2rw2c" May 29 16:58:03.248186 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:03.248158 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-76c9bb5fdf-n6n9z"] May 29 16:58:03.251584 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:58:03.251558 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8921009e_fb12_4b6b_9bef_9120a210fb34.slice/crio-3b396fe9592ebf057c0f0d0e0334e8ec338be63981b96fc06d5d5708b84018ed WatchSource:0}: Error finding container 3b396fe9592ebf057c0f0d0e0334e8ec338be63981b96fc06d5d5708b84018ed: Status 404 returned error can't find the container with id 3b396fe9592ebf057c0f0d0e0334e8ec338be63981b96fc06d5d5708b84018ed May 29 16:58:03.351302 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:03.351271 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56d4bd47c8-2rw2c"] May 29 16:58:03.355164 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:58:03.355142 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34245c5b_a05b_46f7_817f_5da3e5af1e42.slice/crio-4d75edb32eda0769aee0562d74baa49c91125faa1b031a1ea3b3dfa57b01b917 WatchSource:0}: Error finding container 4d75edb32eda0769aee0562d74baa49c91125faa1b031a1ea3b3dfa57b01b917: Status 404 returned error can't find the container with id 4d75edb32eda0769aee0562d74baa49c91125faa1b031a1ea3b3dfa57b01b917 May 29 16:58:03.377007 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:03.376977 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56d4bd47c8-2rw2c" event={"ID":"34245c5b-a05b-46f7-817f-5da3e5af1e42","Type":"ContainerStarted","Data":"4d75edb32eda0769aee0562d74baa49c91125faa1b031a1ea3b3dfa57b01b917"} May 29 16:58:03.377896 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:03.377856 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-n6n9z" event={"ID":"8921009e-fb12-4b6b-9bef-9120a210fb34","Type":"ContainerStarted","Data":"3b396fe9592ebf057c0f0d0e0334e8ec338be63981b96fc06d5d5708b84018ed"} May 29 16:58:04.382340 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:04.382304 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56d4bd47c8-2rw2c" event={"ID":"34245c5b-a05b-46f7-817f-5da3e5af1e42","Type":"ContainerStarted","Data":"34fbfd92f0b4d75eafae6cc6126e019bd6cc8ed4d8299e413aa89a6d78efc874"} May 29 16:58:04.382718 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:04.382359 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-56d4bd47c8-2rw2c" May 29 16:58:04.440998 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:04.440960 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d59b7498-802e-4d3b-85ed-54913e4bc2ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n9wt9\" (UID: \"d59b7498-802e-4d3b-85ed-54913e4bc2ea\") " pod="openshift-insights/insights-runtime-extractor-n9wt9" May 29 16:58:04.443375 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:04.443345 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d59b7498-802e-4d3b-85ed-54913e4bc2ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n9wt9\" (UID: \"d59b7498-802e-4d3b-85ed-54913e4bc2ea\") " pod="openshift-insights/insights-runtime-extractor-n9wt9" May 29 16:58:04.668302 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:04.668233 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kt6kk\"" May 29 16:58:04.677588 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:04.677561 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-n9wt9" May 29 16:58:04.792727 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:04.792677 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-56d4bd47c8-2rw2c" podStartSLOduration=2.792660138 podStartE2EDuration="2.792660138s" podCreationTimestamp="2026-05-29 16:58:02 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 16:58:04.399395162 +0000 UTC m=+74.885824377" watchObservedRunningTime="2026-05-29 16:58:04.792660138 +0000 UTC m=+75.279089340" May 29 16:58:04.793221 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:04.793201 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-n9wt9"] May 29 16:58:04.796471 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:58:04.796445 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd59b7498_802e_4d3b_85ed_54913e4bc2ea.slice/crio-5076ae3a7fc891ee57438da6ce82502c4c9078cca7d7b42cc8b745d6b1bd7b65 WatchSource:0}: Error finding container 5076ae3a7fc891ee57438da6ce82502c4c9078cca7d7b42cc8b745d6b1bd7b65: Status 404 returned error can't find the container with id 5076ae3a7fc891ee57438da6ce82502c4c9078cca7d7b42cc8b745d6b1bd7b65 May 29 16:58:05.386552 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:05.386519 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-n6n9z" event={"ID":"8921009e-fb12-4b6b-9bef-9120a210fb34","Type":"ContainerStarted","Data":"03e2e6a9b7d53e2267d3c205851e1e8c69873ad1ef5801de42ab7ed9e3e86cfc"} May 29 16:58:05.388041 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:05.388008 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n9wt9" event={"ID":"d59b7498-802e-4d3b-85ed-54913e4bc2ea","Type":"ContainerStarted","Data":"50582bc336dd0bf2919a5de5b46f55080389db3fc8bde003d8d610c7306cc59c"} May 29 16:58:05.388041 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:05.388040 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n9wt9" event={"ID":"d59b7498-802e-4d3b-85ed-54913e4bc2ea","Type":"ContainerStarted","Data":"5076ae3a7fc891ee57438da6ce82502c4c9078cca7d7b42cc8b745d6b1bd7b65"} May 29 16:58:06.393154 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:06.393121 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n9wt9" event={"ID":"d59b7498-802e-4d3b-85ed-54913e4bc2ea","Type":"ContainerStarted","Data":"ed8bbaef741482f0280113929fa9718f0eb4fd05f9610e88a551ea534803d3ab"} May 29 16:58:07.397746 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:07.397709 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n9wt9" event={"ID":"d59b7498-802e-4d3b-85ed-54913e4bc2ea","Type":"ContainerStarted","Data":"3aa7596045989fb39a5969a3f8340df9d42962720822bbee5d4d520505c1bec1"} May 29 16:58:07.412550 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:07.412494 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-n6n9z" podStartSLOduration=4.146480359 podStartE2EDuration="5.412480746s" podCreationTimestamp="2026-05-29 16:58:02 +0000 UTC" firstStartedPulling="2026-05-29 16:58:03.253482224 +0000 UTC m=+73.739911404" lastFinishedPulling="2026-05-29 16:58:04.519482611 +0000 UTC m=+75.005911791" observedRunningTime="2026-05-29 16:58:05.399652606 +0000 UTC m=+75.886081805" watchObservedRunningTime="2026-05-29 16:58:07.412480746 +0000 UTC m=+77.898909948" May 29 16:58:07.412880 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:07.412832 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-n9wt9" podStartSLOduration=33.397465279 podStartE2EDuration="35.41282646s" podCreationTimestamp="2026-05-29 16:57:32 +0000 UTC" firstStartedPulling="2026-05-29 16:58:04.846138929 +0000 UTC m=+75.332568109" lastFinishedPulling="2026-05-29 16:58:06.861500098 +0000 UTC m=+77.347929290" observedRunningTime="2026-05-29 16:58:07.411722847 +0000 UTC m=+77.898152049" watchObservedRunningTime="2026-05-29 16:58:07.41282646 +0000 UTC m=+77.899255662" May 29 16:58:10.373631 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:10.373530 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-kzj26" May 29 16:58:12.818374 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:12.818343 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-76f776b797-lhxhm" May 29 16:58:25.392036 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:25.392004 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-56d4bd47c8-2rw2c" May 29 16:58:27.833611 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:27.833534 2549 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-76f776b797-lhxhm" podUID="f35fbbaf-13e2-4a63-865d-00375eb46e97" containerName="registry" containerID="cri-o://a9320ff79397aca4caa084e2f7f63c4a15e53e0ffe87b8ef993dce9d76d86fa9" gracePeriod=30 May 29 16:58:28.071420 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:28.071397 2549 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76f776b797-lhxhm" May 29 16:58:28.203266 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:28.203172 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p8sv5\" (UniqueName: \"kubernetes.io/projected/f35fbbaf-13e2-4a63-865d-00375eb46e97-kube-api-access-p8sv5\") pod \"f35fbbaf-13e2-4a63-865d-00375eb46e97\" (UID: \"f35fbbaf-13e2-4a63-865d-00375eb46e97\") " May 29 16:58:28.203266 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:28.203253 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f35fbbaf-13e2-4a63-865d-00375eb46e97-installation-pull-secrets\") pod \"f35fbbaf-13e2-4a63-865d-00375eb46e97\" (UID: \"f35fbbaf-13e2-4a63-865d-00375eb46e97\") " May 29 16:58:28.203486 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:28.203276 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f35fbbaf-13e2-4a63-865d-00375eb46e97-registry-tls\") pod \"f35fbbaf-13e2-4a63-865d-00375eb46e97\" (UID: \"f35fbbaf-13e2-4a63-865d-00375eb46e97\") " May 29 16:58:28.203486 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:28.203303 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f35fbbaf-13e2-4a63-865d-00375eb46e97-image-registry-private-configuration\") pod \"f35fbbaf-13e2-4a63-865d-00375eb46e97\" (UID: \"f35fbbaf-13e2-4a63-865d-00375eb46e97\") " May 29 16:58:28.203486 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:28.203325 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f35fbbaf-13e2-4a63-865d-00375eb46e97-registry-certificates\") pod \"f35fbbaf-13e2-4a63-865d-00375eb46e97\" (UID: \"f35fbbaf-13e2-4a63-865d-00375eb46e97\") " May 29 16:58:28.203486 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:28.203344 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f35fbbaf-13e2-4a63-865d-00375eb46e97-bound-sa-token\") pod \"f35fbbaf-13e2-4a63-865d-00375eb46e97\" (UID: \"f35fbbaf-13e2-4a63-865d-00375eb46e97\") " May 29 16:58:28.203486 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:28.203367 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f35fbbaf-13e2-4a63-865d-00375eb46e97-trusted-ca\") pod \"f35fbbaf-13e2-4a63-865d-00375eb46e97\" (UID: \"f35fbbaf-13e2-4a63-865d-00375eb46e97\") " May 29 16:58:28.203486 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:28.203384 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f35fbbaf-13e2-4a63-865d-00375eb46e97-ca-trust-extracted\") pod \"f35fbbaf-13e2-4a63-865d-00375eb46e97\" (UID: \"f35fbbaf-13e2-4a63-865d-00375eb46e97\") " May 29 16:58:28.204009 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:28.203945 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f35fbbaf-13e2-4a63-865d-00375eb46e97-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "f35fbbaf-13e2-4a63-865d-00375eb46e97" (UID: "f35fbbaf-13e2-4a63-865d-00375eb46e97"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 29 16:58:28.204121 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:28.204028 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f35fbbaf-13e2-4a63-865d-00375eb46e97-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "f35fbbaf-13e2-4a63-865d-00375eb46e97" (UID: "f35fbbaf-13e2-4a63-865d-00375eb46e97"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 29 16:58:28.206190 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:28.206139 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f35fbbaf-13e2-4a63-865d-00375eb46e97-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f35fbbaf-13e2-4a63-865d-00375eb46e97" (UID: "f35fbbaf-13e2-4a63-865d-00375eb46e97"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 29 16:58:28.206190 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:28.206154 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f35fbbaf-13e2-4a63-865d-00375eb46e97-kube-api-access-p8sv5" (OuterVolumeSpecName: "kube-api-access-p8sv5") pod "f35fbbaf-13e2-4a63-865d-00375eb46e97" (UID: "f35fbbaf-13e2-4a63-865d-00375eb46e97"). InnerVolumeSpecName "kube-api-access-p8sv5". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 29 16:58:28.206345 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:28.206222 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f35fbbaf-13e2-4a63-865d-00375eb46e97-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "f35fbbaf-13e2-4a63-865d-00375eb46e97" (UID: "f35fbbaf-13e2-4a63-865d-00375eb46e97"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 29 16:58:28.206345 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:28.206317 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f35fbbaf-13e2-4a63-865d-00375eb46e97-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "f35fbbaf-13e2-4a63-865d-00375eb46e97" (UID: "f35fbbaf-13e2-4a63-865d-00375eb46e97"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 29 16:58:28.206426 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:28.206388 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f35fbbaf-13e2-4a63-865d-00375eb46e97-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "f35fbbaf-13e2-4a63-865d-00375eb46e97" (UID: "f35fbbaf-13e2-4a63-865d-00375eb46e97"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 29 16:58:28.212706 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:28.212680 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f35fbbaf-13e2-4a63-865d-00375eb46e97-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "f35fbbaf-13e2-4a63-865d-00375eb46e97" (UID: "f35fbbaf-13e2-4a63-865d-00375eb46e97"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 29 16:58:28.304519 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:28.304489 2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p8sv5\" (UniqueName: \"kubernetes.io/projected/f35fbbaf-13e2-4a63-865d-00375eb46e97-kube-api-access-p8sv5\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 16:58:28.304519 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:28.304513 2549 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f35fbbaf-13e2-4a63-865d-00375eb46e97-installation-pull-secrets\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 16:58:28.304519 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:28.304525 2549 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f35fbbaf-13e2-4a63-865d-00375eb46e97-registry-tls\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 16:58:28.304705 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:28.304535 2549 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f35fbbaf-13e2-4a63-865d-00375eb46e97-image-registry-private-configuration\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 16:58:28.304705 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:28.304544 2549 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f35fbbaf-13e2-4a63-865d-00375eb46e97-registry-certificates\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 16:58:28.304705 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:28.304553 2549 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f35fbbaf-13e2-4a63-865d-00375eb46e97-bound-sa-token\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 16:58:28.304705 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:28.304561 2549 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f35fbbaf-13e2-4a63-865d-00375eb46e97-trusted-ca\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 16:58:28.304705 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:28.304569 2549 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f35fbbaf-13e2-4a63-865d-00375eb46e97-ca-trust-extracted\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 16:58:28.457042 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:28.456963 2549 generic.go:358] "Generic (PLEG): container finished" podID="f35fbbaf-13e2-4a63-865d-00375eb46e97" containerID="a9320ff79397aca4caa084e2f7f63c4a15e53e0ffe87b8ef993dce9d76d86fa9" exitCode=0 May 29 16:58:28.457042 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:28.457003 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76f776b797-lhxhm" event={"ID":"f35fbbaf-13e2-4a63-865d-00375eb46e97","Type":"ContainerDied","Data":"a9320ff79397aca4caa084e2f7f63c4a15e53e0ffe87b8ef993dce9d76d86fa9"} May 29 16:58:28.457042 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:28.457021 2549 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76f776b797-lhxhm" May 29 16:58:28.457042 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:28.457036 2549 scope.go:117] "RemoveContainer" containerID="a9320ff79397aca4caa084e2f7f63c4a15e53e0ffe87b8ef993dce9d76d86fa9" May 29 16:58:28.457297 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:28.457026 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76f776b797-lhxhm" event={"ID":"f35fbbaf-13e2-4a63-865d-00375eb46e97","Type":"ContainerDied","Data":"1d24da7d724874cb2563253a598fb38cfbf824b0cfdd215348a5945e437caa3c"} May 29 16:58:28.464882 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:28.464836 2549 scope.go:117] "RemoveContainer" containerID="a9320ff79397aca4caa084e2f7f63c4a15e53e0ffe87b8ef993dce9d76d86fa9" May 29 16:58:28.465146 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:58:28.465129 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9320ff79397aca4caa084e2f7f63c4a15e53e0ffe87b8ef993dce9d76d86fa9\": container with ID starting with a9320ff79397aca4caa084e2f7f63c4a15e53e0ffe87b8ef993dce9d76d86fa9 not found: ID does not exist" containerID="a9320ff79397aca4caa084e2f7f63c4a15e53e0ffe87b8ef993dce9d76d86fa9" May 29 16:58:28.465194 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:28.465154 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9320ff79397aca4caa084e2f7f63c4a15e53e0ffe87b8ef993dce9d76d86fa9"} err="failed to get container status \"a9320ff79397aca4caa084e2f7f63c4a15e53e0ffe87b8ef993dce9d76d86fa9\": rpc error: code = NotFound desc = could not find container \"a9320ff79397aca4caa084e2f7f63c4a15e53e0ffe87b8ef993dce9d76d86fa9\": container with ID starting with a9320ff79397aca4caa084e2f7f63c4a15e53e0ffe87b8ef993dce9d76d86fa9 not found: ID does not exist" May 29 16:58:28.476191 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:28.476164 2549 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-76f776b797-lhxhm"] May 29 16:58:28.480422 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:28.480394 2549 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-76f776b797-lhxhm"] May 29 16:58:30.082970 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:30.082933 2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f35fbbaf-13e2-4a63-865d-00375eb46e97" path="/var/lib/kubelet/pods/f35fbbaf-13e2-4a63-865d-00375eb46e97/volumes" May 29 16:58:31.374068 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:31.374030 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-crvwv" May 29 16:58:32.689179 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:32.689147 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c8c9b6868-mm6qs"] May 29 16:58:32.689609 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:32.689359 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f35fbbaf-13e2-4a63-865d-00375eb46e97" containerName="registry" May 29 16:58:32.689609 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:32.689371 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="f35fbbaf-13e2-4a63-865d-00375eb46e97" containerName="registry" May 29 16:58:32.689609 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:32.689417 2549 memory_manager.go:356] "RemoveStaleState removing state" podUID="f35fbbaf-13e2-4a63-865d-00375eb46e97" containerName="registry" May 29 16:58:32.693165 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:32.693147 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c8c9b6868-mm6qs" May 29 16:58:32.694990 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:32.694968 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" May 29 16:58:32.695446 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:32.695422 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" May 29 16:58:32.695557 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:32.695435 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" May 29 16:58:32.695557 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:32.695543 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" May 29 16:58:32.695721 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:32.695576 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" May 29 16:58:32.695721 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:32.695590 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" May 29 16:58:32.695721 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:32.695486 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5lnvn\"" May 29 16:58:32.695952 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:32.695936 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" May 29 16:58:32.700522 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:32.700231 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" May 29 16:58:32.700608 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:32.700520 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c8c9b6868-mm6qs"] May 29 16:58:32.834538 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:32.834504 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gb55w\" (UniqueName: \"kubernetes.io/projected/54db9067-d609-4bea-a927-c91c52543add-kube-api-access-gb55w\") pod \"console-7c8c9b6868-mm6qs\" (UID: \"54db9067-d609-4bea-a927-c91c52543add\") " pod="openshift-console/console-7c8c9b6868-mm6qs" May 29 16:58:32.834719 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:32.834544 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/54db9067-d609-4bea-a927-c91c52543add-console-config\") pod \"console-7c8c9b6868-mm6qs\" (UID: \"54db9067-d609-4bea-a927-c91c52543add\") " pod="openshift-console/console-7c8c9b6868-mm6qs" May 29 16:58:32.834719 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:32.834581 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/54db9067-d609-4bea-a927-c91c52543add-console-oauth-config\") pod \"console-7c8c9b6868-mm6qs\" (UID: \"54db9067-d609-4bea-a927-c91c52543add\") " pod="openshift-console/console-7c8c9b6868-mm6qs" May 29 16:58:32.834719 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:32.834605 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/54db9067-d609-4bea-a927-c91c52543add-console-serving-cert\") pod \"console-7c8c9b6868-mm6qs\" (UID: \"54db9067-d609-4bea-a927-c91c52543add\") " pod="openshift-console/console-7c8c9b6868-mm6qs" May 29 16:58:32.834719 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:32.834622 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54db9067-d609-4bea-a927-c91c52543add-trusted-ca-bundle\") pod \"console-7c8c9b6868-mm6qs\" (UID: \"54db9067-d609-4bea-a927-c91c52543add\") " pod="openshift-console/console-7c8c9b6868-mm6qs" May 29 16:58:32.834719 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:32.834658 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/54db9067-d609-4bea-a927-c91c52543add-oauth-serving-cert\") pod \"console-7c8c9b6868-mm6qs\" (UID: \"54db9067-d609-4bea-a927-c91c52543add\") " pod="openshift-console/console-7c8c9b6868-mm6qs" May 29 16:58:32.834719 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:32.834684 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/54db9067-d609-4bea-a927-c91c52543add-service-ca\") pod \"console-7c8c9b6868-mm6qs\" (UID: \"54db9067-d609-4bea-a927-c91c52543add\") " pod="openshift-console/console-7c8c9b6868-mm6qs" May 29 16:58:32.935399 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:32.935363 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gb55w\" (UniqueName: \"kubernetes.io/projected/54db9067-d609-4bea-a927-c91c52543add-kube-api-access-gb55w\") pod \"console-7c8c9b6868-mm6qs\" (UID: \"54db9067-d609-4bea-a927-c91c52543add\") " pod="openshift-console/console-7c8c9b6868-mm6qs" May 29 16:58:32.935499 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:32.935406 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/54db9067-d609-4bea-a927-c91c52543add-console-config\") pod \"console-7c8c9b6868-mm6qs\" (UID: \"54db9067-d609-4bea-a927-c91c52543add\") " pod="openshift-console/console-7c8c9b6868-mm6qs" May 29 16:58:32.935499 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:32.935428 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/54db9067-d609-4bea-a927-c91c52543add-console-oauth-config\") pod \"console-7c8c9b6868-mm6qs\" (UID: \"54db9067-d609-4bea-a927-c91c52543add\") " pod="openshift-console/console-7c8c9b6868-mm6qs" May 29 16:58:32.935499 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:32.935449 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/54db9067-d609-4bea-a927-c91c52543add-console-serving-cert\") pod \"console-7c8c9b6868-mm6qs\" (UID: \"54db9067-d609-4bea-a927-c91c52543add\") " pod="openshift-console/console-7c8c9b6868-mm6qs" May 29 16:58:32.935499 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:32.935477 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54db9067-d609-4bea-a927-c91c52543add-trusted-ca-bundle\") pod \"console-7c8c9b6868-mm6qs\" (UID: \"54db9067-d609-4bea-a927-c91c52543add\") " pod="openshift-console/console-7c8c9b6868-mm6qs" May 29 16:58:32.935662 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:32.935515 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/54db9067-d609-4bea-a927-c91c52543add-oauth-serving-cert\") pod \"console-7c8c9b6868-mm6qs\" (UID: \"54db9067-d609-4bea-a927-c91c52543add\") " pod="openshift-console/console-7c8c9b6868-mm6qs" May 29 16:58:32.935662 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:32.935545 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/54db9067-d609-4bea-a927-c91c52543add-service-ca\") pod \"console-7c8c9b6868-mm6qs\" (UID: \"54db9067-d609-4bea-a927-c91c52543add\") " pod="openshift-console/console-7c8c9b6868-mm6qs" May 29 16:58:32.936261 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:32.936236 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/54db9067-d609-4bea-a927-c91c52543add-console-config\") pod \"console-7c8c9b6868-mm6qs\" (UID: \"54db9067-d609-4bea-a927-c91c52543add\") " pod="openshift-console/console-7c8c9b6868-mm6qs" May 29 16:58:32.936386 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:32.936269 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/54db9067-d609-4bea-a927-c91c52543add-oauth-serving-cert\") pod \"console-7c8c9b6868-mm6qs\" (UID: \"54db9067-d609-4bea-a927-c91c52543add\") " pod="openshift-console/console-7c8c9b6868-mm6qs" May 29 16:58:32.936386 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:32.936314 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/54db9067-d609-4bea-a927-c91c52543add-service-ca\") pod \"console-7c8c9b6868-mm6qs\" (UID: \"54db9067-d609-4bea-a927-c91c52543add\") " pod="openshift-console/console-7c8c9b6868-mm6qs" May 29 16:58:32.936634 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:32.936612 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54db9067-d609-4bea-a927-c91c52543add-trusted-ca-bundle\") pod \"console-7c8c9b6868-mm6qs\" (UID: \"54db9067-d609-4bea-a927-c91c52543add\") " pod="openshift-console/console-7c8c9b6868-mm6qs" May 29 16:58:32.937955 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:32.937931 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/54db9067-d609-4bea-a927-c91c52543add-console-oauth-config\") pod \"console-7c8c9b6868-mm6qs\" (UID: \"54db9067-d609-4bea-a927-c91c52543add\") " pod="openshift-console/console-7c8c9b6868-mm6qs" May 29 16:58:32.938070 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:32.938052 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/54db9067-d609-4bea-a927-c91c52543add-console-serving-cert\") pod \"console-7c8c9b6868-mm6qs\" (UID: \"54db9067-d609-4bea-a927-c91c52543add\") " pod="openshift-console/console-7c8c9b6868-mm6qs" May 29 16:58:32.944878 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:32.944809 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gb55w\" (UniqueName: \"kubernetes.io/projected/54db9067-d609-4bea-a927-c91c52543add-kube-api-access-gb55w\") pod \"console-7c8c9b6868-mm6qs\" (UID: \"54db9067-d609-4bea-a927-c91c52543add\") " pod="openshift-console/console-7c8c9b6868-mm6qs" May 29 16:58:33.002665 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:33.002633 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c8c9b6868-mm6qs" May 29 16:58:33.147731 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:33.147700 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c8c9b6868-mm6qs"] May 29 16:58:33.150451 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:58:33.150421 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54db9067_d609_4bea_a927_c91c52543add.slice/crio-ef1b94cc3ff0dc30402234788a647e9b4cb18af7769620d6ad65e0e2f9ef1ac1 WatchSource:0}: Error finding container ef1b94cc3ff0dc30402234788a647e9b4cb18af7769620d6ad65e0e2f9ef1ac1: Status 404 returned error can't find the container with id ef1b94cc3ff0dc30402234788a647e9b4cb18af7769620d6ad65e0e2f9ef1ac1 May 29 16:58:33.470047 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:33.470005 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c8c9b6868-mm6qs" event={"ID":"54db9067-d609-4bea-a927-c91c52543add","Type":"ContainerStarted","Data":"ef1b94cc3ff0dc30402234788a647e9b4cb18af7769620d6ad65e0e2f9ef1ac1"} May 29 16:58:36.478721 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:36.478682 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c8c9b6868-mm6qs" event={"ID":"54db9067-d609-4bea-a927-c91c52543add","Type":"ContainerStarted","Data":"c42f0df2a6ac4796b1b410cd9081bd6235ae7f23706e375e2c20467c90938721"} May 29 16:58:36.495089 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:36.495039 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-7c8c9b6868-mm6qs" podStartSLOduration=2.020645913 podStartE2EDuration="4.495021098s" podCreationTimestamp="2026-05-29 16:58:32 +0000 UTC" firstStartedPulling="2026-05-29 16:58:33.152299422 +0000 UTC m=+103.638728602" lastFinishedPulling="2026-05-29 16:58:35.626674607 +0000 UTC m=+106.113103787" observedRunningTime="2026-05-29 16:58:36.49466725 +0000 UTC m=+106.981096455" watchObservedRunningTime="2026-05-29 16:58:36.495021098 +0000 UTC m=+106.981450301" May 29 16:58:38.994963 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:38.994767 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-zn8jx"] May 29 16:58:39.000005 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:38.999979 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-8469c486bb-jpxcp"] May 29 16:58:39.000170 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.000150 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-85658796db-zn8jx" May 29 16:58:39.002931 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.002911 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-8469c486bb-jpxcp" May 29 16:58:39.005142 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.005124 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" May 29 16:58:39.005296 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.005146 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" May 29 16:58:39.005640 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.005619 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" May 29 16:58:39.005735 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.005676 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-wvlzb\"" May 29 16:58:39.006164 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.006146 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" May 29 16:58:39.006437 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.006416 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" May 29 16:58:39.006517 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.006186 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" May 29 16:58:39.006576 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.006216 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" May 29 16:58:39.006632 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.006388 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" May 29 16:58:39.006632 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.006177 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-5zjgd\"" May 29 16:58:39.007811 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.007569 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-zn8jx"] May 29 16:58:39.010072 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.010053 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-8469c486bb-jpxcp"] May 29 16:58:39.013033 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.013011 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-9ddvs"] May 29 16:58:39.016037 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.016022 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9ddvs" May 29 16:58:39.018028 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.018008 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-dt92w\"" May 29 16:58:39.018121 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.018014 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" May 29 16:58:39.018121 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.018057 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" May 29 16:58:39.018121 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.018062 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" May 29 16:58:39.182647 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.182608 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67c4e63f-21b5-4b12-8a2c-915a4d64852b-metrics-client-ca\") pod \"kube-state-metrics-8469c486bb-jpxcp\" (UID: \"67c4e63f-21b5-4b12-8a2c-915a4d64852b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jpxcp" May 29 16:58:39.182647 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.182645 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0607511d-a31e-4794-b8b8-6a58179aabd0-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-zn8jx\" (UID: \"0607511d-a31e-4794-b8b8-6a58179aabd0\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-zn8jx" May 29 16:58:39.182950 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.182665 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mm8hn\" (UniqueName: \"kubernetes.io/projected/67c4e63f-21b5-4b12-8a2c-915a4d64852b-kube-api-access-mm8hn\") pod \"kube-state-metrics-8469c486bb-jpxcp\" (UID: \"67c4e63f-21b5-4b12-8a2c-915a4d64852b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jpxcp" May 29 16:58:39.182950 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.182727 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dfd131b4-94e6-4910-8548-de6c489f64ac-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9ddvs\" (UID: \"dfd131b4-94e6-4910-8548-de6c489f64ac\") " pod="openshift-monitoring/node-exporter-9ddvs" May 29 16:58:39.182950 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.182772 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhj4t\" (UniqueName: \"kubernetes.io/projected/dfd131b4-94e6-4910-8548-de6c489f64ac-kube-api-access-xhj4t\") pod \"node-exporter-9ddvs\" (UID: \"dfd131b4-94e6-4910-8548-de6c489f64ac\") " pod="openshift-monitoring/node-exporter-9ddvs" May 29 16:58:39.182950 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.182802 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddbh4\" (UniqueName: \"kubernetes.io/projected/0607511d-a31e-4794-b8b8-6a58179aabd0-kube-api-access-ddbh4\") pod \"openshift-state-metrics-85658796db-zn8jx\" (UID: \"0607511d-a31e-4794-b8b8-6a58179aabd0\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-zn8jx" May 29 16:58:39.182950 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.182827 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/67c4e63f-21b5-4b12-8a2c-915a4d64852b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-8469c486bb-jpxcp\" (UID: \"67c4e63f-21b5-4b12-8a2c-915a4d64852b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jpxcp" May 29 16:58:39.182950 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.182883 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/67c4e63f-21b5-4b12-8a2c-915a4d64852b-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-jpxcp\" (UID: \"67c4e63f-21b5-4b12-8a2c-915a4d64852b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jpxcp" May 29 16:58:39.182950 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.182911 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dfd131b4-94e6-4910-8548-de6c489f64ac-node-exporter-textfile\") pod \"node-exporter-9ddvs\" (UID: \"dfd131b4-94e6-4910-8548-de6c489f64ac\") " pod="openshift-monitoring/node-exporter-9ddvs" May 29 16:58:39.183274 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.182949 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0607511d-a31e-4794-b8b8-6a58179aabd0-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-zn8jx\" (UID: \"0607511d-a31e-4794-b8b8-6a58179aabd0\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-zn8jx" May 29 16:58:39.183274 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.182978 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dfd131b4-94e6-4910-8548-de6c489f64ac-node-exporter-tls\") pod \"node-exporter-9ddvs\" (UID: \"dfd131b4-94e6-4910-8548-de6c489f64ac\") " pod="openshift-monitoring/node-exporter-9ddvs" May 29 16:58:39.183274 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.183014 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/67c4e63f-21b5-4b12-8a2c-915a4d64852b-volume-directive-shadow\") pod \"kube-state-metrics-8469c486bb-jpxcp\" (UID: \"67c4e63f-21b5-4b12-8a2c-915a4d64852b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jpxcp" May 29 16:58:39.183274 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.183043 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0607511d-a31e-4794-b8b8-6a58179aabd0-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-zn8jx\" (UID: \"0607511d-a31e-4794-b8b8-6a58179aabd0\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-zn8jx" May 29 16:58:39.183274 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.183102 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dfd131b4-94e6-4910-8548-de6c489f64ac-node-exporter-wtmp\") pod \"node-exporter-9ddvs\" (UID: \"dfd131b4-94e6-4910-8548-de6c489f64ac\") " pod="openshift-monitoring/node-exporter-9ddvs" May 29 16:58:39.183274 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.183134 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dfd131b4-94e6-4910-8548-de6c489f64ac-sys\") pod \"node-exporter-9ddvs\" (UID: \"dfd131b4-94e6-4910-8548-de6c489f64ac\") " pod="openshift-monitoring/node-exporter-9ddvs" May 29 16:58:39.183274 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.183155 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/67c4e63f-21b5-4b12-8a2c-915a4d64852b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-8469c486bb-jpxcp\" (UID: \"67c4e63f-21b5-4b12-8a2c-915a4d64852b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jpxcp" May 29 16:58:39.183274 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.183204 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dfd131b4-94e6-4910-8548-de6c489f64ac-node-exporter-accelerators-collector-config\") pod \"node-exporter-9ddvs\" (UID: \"dfd131b4-94e6-4910-8548-de6c489f64ac\") " pod="openshift-monitoring/node-exporter-9ddvs" May 29 16:58:39.183274 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.183238 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dfd131b4-94e6-4910-8548-de6c489f64ac-root\") pod \"node-exporter-9ddvs\" (UID: \"dfd131b4-94e6-4910-8548-de6c489f64ac\") " pod="openshift-monitoring/node-exporter-9ddvs" May 29 16:58:39.183274 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.183259 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dfd131b4-94e6-4910-8548-de6c489f64ac-metrics-client-ca\") pod \"node-exporter-9ddvs\" (UID: \"dfd131b4-94e6-4910-8548-de6c489f64ac\") " pod="openshift-monitoring/node-exporter-9ddvs" May 29 16:58:39.284566 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.284530 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67c4e63f-21b5-4b12-8a2c-915a4d64852b-metrics-client-ca\") pod \"kube-state-metrics-8469c486bb-jpxcp\" (UID: \"67c4e63f-21b5-4b12-8a2c-915a4d64852b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jpxcp" May 29 16:58:39.284745 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.284575 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0607511d-a31e-4794-b8b8-6a58179aabd0-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-zn8jx\" (UID: \"0607511d-a31e-4794-b8b8-6a58179aabd0\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-zn8jx" May 29 16:58:39.284745 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.284603 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mm8hn\" (UniqueName: \"kubernetes.io/projected/67c4e63f-21b5-4b12-8a2c-915a4d64852b-kube-api-access-mm8hn\") pod \"kube-state-metrics-8469c486bb-jpxcp\" (UID: \"67c4e63f-21b5-4b12-8a2c-915a4d64852b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jpxcp" May 29 16:58:39.284745 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.284642 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dfd131b4-94e6-4910-8548-de6c489f64ac-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9ddvs\" (UID: \"dfd131b4-94e6-4910-8548-de6c489f64ac\") " pod="openshift-monitoring/node-exporter-9ddvs" May 29 16:58:39.284745 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.284665 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xhj4t\" (UniqueName: \"kubernetes.io/projected/dfd131b4-94e6-4910-8548-de6c489f64ac-kube-api-access-xhj4t\") pod \"node-exporter-9ddvs\" (UID: \"dfd131b4-94e6-4910-8548-de6c489f64ac\") " pod="openshift-monitoring/node-exporter-9ddvs" May 29 16:58:39.284745 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:58:39.284682 2549 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found May 29 16:58:39.285027 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.284691 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ddbh4\" (UniqueName: \"kubernetes.io/projected/0607511d-a31e-4794-b8b8-6a58179aabd0-kube-api-access-ddbh4\") pod \"openshift-state-metrics-85658796db-zn8jx\" (UID: \"0607511d-a31e-4794-b8b8-6a58179aabd0\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-zn8jx" May 29 16:58:39.285027 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:58:39.284762 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0607511d-a31e-4794-b8b8-6a58179aabd0-openshift-state-metrics-tls podName:0607511d-a31e-4794-b8b8-6a58179aabd0 nodeName:}" failed. No retries permitted until 2026-05-29 16:58:39.784741243 +0000 UTC m=+110.271170425 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/0607511d-a31e-4794-b8b8-6a58179aabd0-openshift-state-metrics-tls") pod "openshift-state-metrics-85658796db-zn8jx" (UID: "0607511d-a31e-4794-b8b8-6a58179aabd0") : secret "openshift-state-metrics-tls" not found May 29 16:58:39.285027 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.284805 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/67c4e63f-21b5-4b12-8a2c-915a4d64852b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-8469c486bb-jpxcp\" (UID: \"67c4e63f-21b5-4b12-8a2c-915a4d64852b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jpxcp" May 29 16:58:39.285027 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.284870 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/67c4e63f-21b5-4b12-8a2c-915a4d64852b-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-jpxcp\" (UID: \"67c4e63f-21b5-4b12-8a2c-915a4d64852b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jpxcp" May 29 16:58:39.285027 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.284899 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dfd131b4-94e6-4910-8548-de6c489f64ac-node-exporter-textfile\") pod \"node-exporter-9ddvs\" (UID: \"dfd131b4-94e6-4910-8548-de6c489f64ac\") " pod="openshift-monitoring/node-exporter-9ddvs" May 29 16:58:39.285027 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.285016 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0607511d-a31e-4794-b8b8-6a58179aabd0-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-zn8jx\" (UID: \"0607511d-a31e-4794-b8b8-6a58179aabd0\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-zn8jx" May 29 16:58:39.285378 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.285049 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dfd131b4-94e6-4910-8548-de6c489f64ac-node-exporter-tls\") pod \"node-exporter-9ddvs\" (UID: \"dfd131b4-94e6-4910-8548-de6c489f64ac\") " pod="openshift-monitoring/node-exporter-9ddvs" May 29 16:58:39.285378 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.285089 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/67c4e63f-21b5-4b12-8a2c-915a4d64852b-volume-directive-shadow\") pod \"kube-state-metrics-8469c486bb-jpxcp\" (UID: \"67c4e63f-21b5-4b12-8a2c-915a4d64852b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jpxcp" May 29 16:58:39.285378 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.285114 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0607511d-a31e-4794-b8b8-6a58179aabd0-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-zn8jx\" (UID: \"0607511d-a31e-4794-b8b8-6a58179aabd0\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-zn8jx" May 29 16:58:39.285378 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.285143 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dfd131b4-94e6-4910-8548-de6c489f64ac-node-exporter-wtmp\") pod \"node-exporter-9ddvs\" (UID: \"dfd131b4-94e6-4910-8548-de6c489f64ac\") " pod="openshift-monitoring/node-exporter-9ddvs" May 29 16:58:39.285378 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.285184 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dfd131b4-94e6-4910-8548-de6c489f64ac-node-exporter-textfile\") pod \"node-exporter-9ddvs\" (UID: \"dfd131b4-94e6-4910-8548-de6c489f64ac\") " pod="openshift-monitoring/node-exporter-9ddvs" May 29 16:58:39.285378 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.285230 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dfd131b4-94e6-4910-8548-de6c489f64ac-sys\") pod \"node-exporter-9ddvs\" (UID: \"dfd131b4-94e6-4910-8548-de6c489f64ac\") " pod="openshift-monitoring/node-exporter-9ddvs" May 29 16:58:39.285378 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.285269 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/67c4e63f-21b5-4b12-8a2c-915a4d64852b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-8469c486bb-jpxcp\" (UID: \"67c4e63f-21b5-4b12-8a2c-915a4d64852b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jpxcp" May 29 16:58:39.285378 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:58:39.285356 2549 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found May 29 16:58:39.285768 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:58:39.285421 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfd131b4-94e6-4910-8548-de6c489f64ac-node-exporter-tls podName:dfd131b4-94e6-4910-8548-de6c489f64ac nodeName:}" failed. No retries permitted until 2026-05-29 16:58:39.785402772 +0000 UTC m=+110.271831960 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/dfd131b4-94e6-4910-8548-de6c489f64ac-node-exporter-tls") pod "node-exporter-9ddvs" (UID: "dfd131b4-94e6-4910-8548-de6c489f64ac") : secret "node-exporter-tls" not found May 29 16:58:39.285768 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.285454 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dfd131b4-94e6-4910-8548-de6c489f64ac-node-exporter-accelerators-collector-config\") pod \"node-exporter-9ddvs\" (UID: \"dfd131b4-94e6-4910-8548-de6c489f64ac\") " pod="openshift-monitoring/node-exporter-9ddvs" May 29 16:58:39.285768 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.285488 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dfd131b4-94e6-4910-8548-de6c489f64ac-root\") pod \"node-exporter-9ddvs\" (UID: \"dfd131b4-94e6-4910-8548-de6c489f64ac\") " pod="openshift-monitoring/node-exporter-9ddvs" May 29 16:58:39.285768 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.285489 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/67c4e63f-21b5-4b12-8a2c-915a4d64852b-volume-directive-shadow\") pod \"kube-state-metrics-8469c486bb-jpxcp\" (UID: \"67c4e63f-21b5-4b12-8a2c-915a4d64852b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jpxcp" May 29 16:58:39.285768 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.285511 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dfd131b4-94e6-4910-8548-de6c489f64ac-node-exporter-wtmp\") pod \"node-exporter-9ddvs\" (UID: \"dfd131b4-94e6-4910-8548-de6c489f64ac\") " pod="openshift-monitoring/node-exporter-9ddvs" May 29 16:58:39.285768 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.285532 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dfd131b4-94e6-4910-8548-de6c489f64ac-metrics-client-ca\") pod \"node-exporter-9ddvs\" (UID: \"dfd131b4-94e6-4910-8548-de6c489f64ac\") " pod="openshift-monitoring/node-exporter-9ddvs" May 29 16:58:39.285768 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.285572 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dfd131b4-94e6-4910-8548-de6c489f64ac-sys\") pod \"node-exporter-9ddvs\" (UID: \"dfd131b4-94e6-4910-8548-de6c489f64ac\") " pod="openshift-monitoring/node-exporter-9ddvs" May 29 16:58:39.285768 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.285617 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dfd131b4-94e6-4910-8548-de6c489f64ac-root\") pod \"node-exporter-9ddvs\" (UID: \"dfd131b4-94e6-4910-8548-de6c489f64ac\") " pod="openshift-monitoring/node-exporter-9ddvs" May 29 16:58:39.285768 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.285378 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67c4e63f-21b5-4b12-8a2c-915a4d64852b-metrics-client-ca\") pod \"kube-state-metrics-8469c486bb-jpxcp\" (UID: \"67c4e63f-21b5-4b12-8a2c-915a4d64852b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jpxcp" May 29 16:58:39.286220 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.286057 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/67c4e63f-21b5-4b12-8a2c-915a4d64852b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-8469c486bb-jpxcp\" (UID: \"67c4e63f-21b5-4b12-8a2c-915a4d64852b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jpxcp" May 29 16:58:39.286220 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.286103 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dfd131b4-94e6-4910-8548-de6c489f64ac-node-exporter-accelerators-collector-config\") pod \"node-exporter-9ddvs\" (UID: \"dfd131b4-94e6-4910-8548-de6c489f64ac\") " pod="openshift-monitoring/node-exporter-9ddvs" May 29 16:58:39.286220 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.286108 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dfd131b4-94e6-4910-8548-de6c489f64ac-metrics-client-ca\") pod \"node-exporter-9ddvs\" (UID: \"dfd131b4-94e6-4910-8548-de6c489f64ac\") " pod="openshift-monitoring/node-exporter-9ddvs" May 29 16:58:39.286355 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.286245 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0607511d-a31e-4794-b8b8-6a58179aabd0-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-zn8jx\" (UID: \"0607511d-a31e-4794-b8b8-6a58179aabd0\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-zn8jx" May 29 16:58:39.287278 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.287248 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dfd131b4-94e6-4910-8548-de6c489f64ac-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9ddvs\" (UID: \"dfd131b4-94e6-4910-8548-de6c489f64ac\") " pod="openshift-monitoring/node-exporter-9ddvs" May 29 16:58:39.287746 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.287726 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/67c4e63f-21b5-4b12-8a2c-915a4d64852b-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-jpxcp\" (UID: \"67c4e63f-21b5-4b12-8a2c-915a4d64852b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jpxcp" May 29 16:58:39.287928 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.287912 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0607511d-a31e-4794-b8b8-6a58179aabd0-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-zn8jx\" (UID: \"0607511d-a31e-4794-b8b8-6a58179aabd0\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-zn8jx" May 29 16:58:39.288107 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.288089 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/67c4e63f-21b5-4b12-8a2c-915a4d64852b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-8469c486bb-jpxcp\" (UID: \"67c4e63f-21b5-4b12-8a2c-915a4d64852b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jpxcp" May 29 16:58:39.293430 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.293403 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddbh4\" (UniqueName: \"kubernetes.io/projected/0607511d-a31e-4794-b8b8-6a58179aabd0-kube-api-access-ddbh4\") pod \"openshift-state-metrics-85658796db-zn8jx\" (UID: \"0607511d-a31e-4794-b8b8-6a58179aabd0\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-zn8jx" May 29 16:58:39.294019 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.293960 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhj4t\" (UniqueName: \"kubernetes.io/projected/dfd131b4-94e6-4910-8548-de6c489f64ac-kube-api-access-xhj4t\") pod \"node-exporter-9ddvs\" (UID: \"dfd131b4-94e6-4910-8548-de6c489f64ac\") " pod="openshift-monitoring/node-exporter-9ddvs" May 29 16:58:39.294462 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.294440 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mm8hn\" (UniqueName: \"kubernetes.io/projected/67c4e63f-21b5-4b12-8a2c-915a4d64852b-kube-api-access-mm8hn\") pod \"kube-state-metrics-8469c486bb-jpxcp\" (UID: \"67c4e63f-21b5-4b12-8a2c-915a4d64852b\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jpxcp" May 29 16:58:39.317641 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.317615 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-8469c486bb-jpxcp" May 29 16:58:39.434338 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.434308 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-8469c486bb-jpxcp"] May 29 16:58:39.437229 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:58:39.437199 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67c4e63f_21b5_4b12_8a2c_915a4d64852b.slice/crio-2e5f30ce145f5251f33333c338451bdc126404e245c330e5adda568b90b880e3 WatchSource:0}: Error finding container 2e5f30ce145f5251f33333c338451bdc126404e245c330e5adda568b90b880e3: Status 404 returned error can't find the container with id 2e5f30ce145f5251f33333c338451bdc126404e245c330e5adda568b90b880e3 May 29 16:58:39.486238 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.486202 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-jpxcp" event={"ID":"67c4e63f-21b5-4b12-8a2c-915a4d64852b","Type":"ContainerStarted","Data":"2e5f30ce145f5251f33333c338451bdc126404e245c330e5adda568b90b880e3"} May 29 16:58:39.791199 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.791168 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0607511d-a31e-4794-b8b8-6a58179aabd0-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-zn8jx\" (UID: \"0607511d-a31e-4794-b8b8-6a58179aabd0\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-zn8jx" May 29 16:58:39.791353 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.791242 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dfd131b4-94e6-4910-8548-de6c489f64ac-node-exporter-tls\") pod \"node-exporter-9ddvs\" (UID: \"dfd131b4-94e6-4910-8548-de6c489f64ac\") " pod="openshift-monitoring/node-exporter-9ddvs" May 29 16:58:39.793453 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.793426 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dfd131b4-94e6-4910-8548-de6c489f64ac-node-exporter-tls\") pod \"node-exporter-9ddvs\" (UID: \"dfd131b4-94e6-4910-8548-de6c489f64ac\") " pod="openshift-monitoring/node-exporter-9ddvs" May 29 16:58:39.793580 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.793562 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0607511d-a31e-4794-b8b8-6a58179aabd0-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-zn8jx\" (UID: \"0607511d-a31e-4794-b8b8-6a58179aabd0\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-zn8jx" May 29 16:58:39.911311 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.911283 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-85658796db-zn8jx" May 29 16:58:39.925967 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:39.925945 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9ddvs" May 29 16:58:39.935246 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:58:39.935199 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfd131b4_94e6_4910_8548_de6c489f64ac.slice/crio-9ac484b7a8ab3881eff211d5bcacd3be53af8c84a2a0c0abe97a02fe7e3333d5 WatchSource:0}: Error finding container 9ac484b7a8ab3881eff211d5bcacd3be53af8c84a2a0c0abe97a02fe7e3333d5: Status 404 returned error can't find the container with id 9ac484b7a8ab3881eff211d5bcacd3be53af8c84a2a0c0abe97a02fe7e3333d5 May 29 16:58:40.058205 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:40.058105 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-zn8jx"] May 29 16:58:40.062187 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:58:40.062159 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0607511d_a31e_4794_b8b8_6a58179aabd0.slice/crio-1f6380244575dca1fa730bda1f70e19aa9f179bd2cf1f9d45b6dcd82b3d54767 WatchSource:0}: Error finding container 1f6380244575dca1fa730bda1f70e19aa9f179bd2cf1f9d45b6dcd82b3d54767: Status 404 returned error can't find the container with id 1f6380244575dca1fa730bda1f70e19aa9f179bd2cf1f9d45b6dcd82b3d54767 May 29 16:58:40.491202 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:40.491112 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-zn8jx" event={"ID":"0607511d-a31e-4794-b8b8-6a58179aabd0","Type":"ContainerStarted","Data":"0e202b6c6f93fd9f61c289a9e6a3b920af2f1c4d47fb5c428d5a578822dd6fc3"} May 29 16:58:40.491202 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:40.491159 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-zn8jx" event={"ID":"0607511d-a31e-4794-b8b8-6a58179aabd0","Type":"ContainerStarted","Data":"8ea420ba5224a45fd19b90562561f98a4985d6afa45ffba959fa44ae1b5f47d7"} May 29 16:58:40.491202 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:40.491172 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-zn8jx" event={"ID":"0607511d-a31e-4794-b8b8-6a58179aabd0","Type":"ContainerStarted","Data":"1f6380244575dca1fa730bda1f70e19aa9f179bd2cf1f9d45b6dcd82b3d54767"} May 29 16:58:40.492243 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:40.492218 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9ddvs" event={"ID":"dfd131b4-94e6-4910-8548-de6c489f64ac","Type":"ContainerStarted","Data":"9ac484b7a8ab3881eff211d5bcacd3be53af8c84a2a0c0abe97a02fe7e3333d5"} May 29 16:58:41.074493 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:41.074433 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7594d44679-mkqlc"] May 29 16:58:41.078585 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:41.078563 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7594d44679-mkqlc" May 29 16:58:41.080838 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:41.080811 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" May 29 16:58:41.081498 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:41.081467 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" May 29 16:58:41.081610 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:41.081587 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" May 29 16:58:41.081909 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:41.081888 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" May 29 16:58:41.082020 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:41.081944 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" May 29 16:58:41.082020 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:41.082002 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-fwf5h\"" May 29 16:58:41.082020 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:41.081947 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-b8begor8ecd55\"" May 29 16:58:41.088724 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:41.088701 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7594d44679-mkqlc"] May 29 16:58:41.202493 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:41.202456 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3686b4d1-9720-4c73-95ce-177c869d1c5f-metrics-client-ca\") pod \"thanos-querier-7594d44679-mkqlc\" (UID: \"3686b4d1-9720-4c73-95ce-177c869d1c5f\") " pod="openshift-monitoring/thanos-querier-7594d44679-mkqlc" May 29 16:58:41.202641 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:41.202494 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3686b4d1-9720-4c73-95ce-177c869d1c5f-secret-grpc-tls\") pod \"thanos-querier-7594d44679-mkqlc\" (UID: \"3686b4d1-9720-4c73-95ce-177c869d1c5f\") " pod="openshift-monitoring/thanos-querier-7594d44679-mkqlc" May 29 16:58:41.202641 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:41.202565 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdq6l\" (UniqueName: \"kubernetes.io/projected/3686b4d1-9720-4c73-95ce-177c869d1c5f-kube-api-access-xdq6l\") pod \"thanos-querier-7594d44679-mkqlc\" (UID: \"3686b4d1-9720-4c73-95ce-177c869d1c5f\") " pod="openshift-monitoring/thanos-querier-7594d44679-mkqlc" May 29 16:58:41.202641 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:41.202586 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3686b4d1-9720-4c73-95ce-177c869d1c5f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7594d44679-mkqlc\" (UID: \"3686b4d1-9720-4c73-95ce-177c869d1c5f\") " pod="openshift-monitoring/thanos-querier-7594d44679-mkqlc" May 29 16:58:41.202781 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:41.202662 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3686b4d1-9720-4c73-95ce-177c869d1c5f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7594d44679-mkqlc\" (UID: \"3686b4d1-9720-4c73-95ce-177c869d1c5f\") " pod="openshift-monitoring/thanos-querier-7594d44679-mkqlc" May 29 16:58:41.202781 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:41.202705 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3686b4d1-9720-4c73-95ce-177c869d1c5f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7594d44679-mkqlc\" (UID: \"3686b4d1-9720-4c73-95ce-177c869d1c5f\") " pod="openshift-monitoring/thanos-querier-7594d44679-mkqlc" May 29 16:58:41.202781 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:41.202736 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3686b4d1-9720-4c73-95ce-177c869d1c5f-secret-thanos-querier-tls\") pod \"thanos-querier-7594d44679-mkqlc\" (UID: \"3686b4d1-9720-4c73-95ce-177c869d1c5f\") " pod="openshift-monitoring/thanos-querier-7594d44679-mkqlc" May 29 16:58:41.202912 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:41.202789 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3686b4d1-9720-4c73-95ce-177c869d1c5f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7594d44679-mkqlc\" (UID: \"3686b4d1-9720-4c73-95ce-177c869d1c5f\") " pod="openshift-monitoring/thanos-querier-7594d44679-mkqlc" May 29 16:58:41.303990 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:41.303951 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xdq6l\" (UniqueName: \"kubernetes.io/projected/3686b4d1-9720-4c73-95ce-177c869d1c5f-kube-api-access-xdq6l\") pod \"thanos-querier-7594d44679-mkqlc\" (UID: \"3686b4d1-9720-4c73-95ce-177c869d1c5f\") " pod="openshift-monitoring/thanos-querier-7594d44679-mkqlc" May 29 16:58:41.303990 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:41.303991 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3686b4d1-9720-4c73-95ce-177c869d1c5f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7594d44679-mkqlc\" (UID: \"3686b4d1-9720-4c73-95ce-177c869d1c5f\") " pod="openshift-monitoring/thanos-querier-7594d44679-mkqlc" May 29 16:58:41.304224 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:41.304017 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3686b4d1-9720-4c73-95ce-177c869d1c5f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7594d44679-mkqlc\" (UID: \"3686b4d1-9720-4c73-95ce-177c869d1c5f\") " pod="openshift-monitoring/thanos-querier-7594d44679-mkqlc" May 29 16:58:41.304224 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:41.304043 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3686b4d1-9720-4c73-95ce-177c869d1c5f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7594d44679-mkqlc\" (UID: \"3686b4d1-9720-4c73-95ce-177c869d1c5f\") " pod="openshift-monitoring/thanos-querier-7594d44679-mkqlc" May 29 16:58:41.304224 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:41.304195 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3686b4d1-9720-4c73-95ce-177c869d1c5f-secret-thanos-querier-tls\") pod \"thanos-querier-7594d44679-mkqlc\" (UID: \"3686b4d1-9720-4c73-95ce-177c869d1c5f\") " pod="openshift-monitoring/thanos-querier-7594d44679-mkqlc" May 29 16:58:41.304378 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:41.304276 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3686b4d1-9720-4c73-95ce-177c869d1c5f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7594d44679-mkqlc\" (UID: \"3686b4d1-9720-4c73-95ce-177c869d1c5f\") " pod="openshift-monitoring/thanos-querier-7594d44679-mkqlc" May 29 16:58:41.304378 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:41.304354 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3686b4d1-9720-4c73-95ce-177c869d1c5f-metrics-client-ca\") pod \"thanos-querier-7594d44679-mkqlc\" (UID: \"3686b4d1-9720-4c73-95ce-177c869d1c5f\") " pod="openshift-monitoring/thanos-querier-7594d44679-mkqlc" May 29 16:58:41.304477 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:41.304383 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3686b4d1-9720-4c73-95ce-177c869d1c5f-secret-grpc-tls\") pod \"thanos-querier-7594d44679-mkqlc\" (UID: \"3686b4d1-9720-4c73-95ce-177c869d1c5f\") " pod="openshift-monitoring/thanos-querier-7594d44679-mkqlc" May 29 16:58:41.305361 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:41.305329 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3686b4d1-9720-4c73-95ce-177c869d1c5f-metrics-client-ca\") pod \"thanos-querier-7594d44679-mkqlc\" (UID: \"3686b4d1-9720-4c73-95ce-177c869d1c5f\") " pod="openshift-monitoring/thanos-querier-7594d44679-mkqlc" May 29 16:58:41.307089 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:41.307040 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3686b4d1-9720-4c73-95ce-177c869d1c5f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7594d44679-mkqlc\" (UID: \"3686b4d1-9720-4c73-95ce-177c869d1c5f\") " pod="openshift-monitoring/thanos-querier-7594d44679-mkqlc" May 29 16:58:41.307089 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:41.307058 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3686b4d1-9720-4c73-95ce-177c869d1c5f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7594d44679-mkqlc\" (UID: \"3686b4d1-9720-4c73-95ce-177c869d1c5f\") " pod="openshift-monitoring/thanos-querier-7594d44679-mkqlc" May 29 16:58:41.307290 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:41.307176 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3686b4d1-9720-4c73-95ce-177c869d1c5f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7594d44679-mkqlc\" (UID: \"3686b4d1-9720-4c73-95ce-177c869d1c5f\") " pod="openshift-monitoring/thanos-querier-7594d44679-mkqlc" May 29 16:58:41.307419 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:41.307391 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3686b4d1-9720-4c73-95ce-177c869d1c5f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7594d44679-mkqlc\" (UID: \"3686b4d1-9720-4c73-95ce-177c869d1c5f\") " pod="openshift-monitoring/thanos-querier-7594d44679-mkqlc" May 29 16:58:41.308248 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:41.308229 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3686b4d1-9720-4c73-95ce-177c869d1c5f-secret-thanos-querier-tls\") pod \"thanos-querier-7594d44679-mkqlc\" (UID: \"3686b4d1-9720-4c73-95ce-177c869d1c5f\") " pod="openshift-monitoring/thanos-querier-7594d44679-mkqlc" May 29 16:58:41.308922 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:41.308900 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3686b4d1-9720-4c73-95ce-177c869d1c5f-secret-grpc-tls\") pod \"thanos-querier-7594d44679-mkqlc\" (UID: \"3686b4d1-9720-4c73-95ce-177c869d1c5f\") " pod="openshift-monitoring/thanos-querier-7594d44679-mkqlc" May 29 16:58:41.312377 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:41.312354 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdq6l\" (UniqueName: \"kubernetes.io/projected/3686b4d1-9720-4c73-95ce-177c869d1c5f-kube-api-access-xdq6l\") pod \"thanos-querier-7594d44679-mkqlc\" (UID: \"3686b4d1-9720-4c73-95ce-177c869d1c5f\") " pod="openshift-monitoring/thanos-querier-7594d44679-mkqlc" May 29 16:58:41.393501 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:41.393422 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7594d44679-mkqlc" May 29 16:58:41.497227 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:41.497188 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-jpxcp" event={"ID":"67c4e63f-21b5-4b12-8a2c-915a4d64852b","Type":"ContainerStarted","Data":"0ace387fb81ec92332c7d9fda5e2fb19c1aa2570f2442690aecdfc8c8cf661da"} May 29 16:58:41.497347 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:41.497233 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-jpxcp" event={"ID":"67c4e63f-21b5-4b12-8a2c-915a4d64852b","Type":"ContainerStarted","Data":"fc90b77d9c46479b932b2d77b01ba862bc990505c43de5d5bce3894ba078749f"} May 29 16:58:41.497347 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:41.497249 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-jpxcp" event={"ID":"67c4e63f-21b5-4b12-8a2c-915a4d64852b","Type":"ContainerStarted","Data":"c2d9a38d1e4c5349a93dad1078b8909d78100d5c554104bf3ec79f4b01ff0529"} May 29 16:58:41.516645 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:41.516590 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-8469c486bb-jpxcp" podStartSLOduration=2.31777149 podStartE2EDuration="3.516571688s" podCreationTimestamp="2026-05-29 16:58:38 +0000 UTC" firstStartedPulling="2026-05-29 16:58:39.438997092 +0000 UTC m=+109.925426271" lastFinishedPulling="2026-05-29 16:58:40.637797271 +0000 UTC m=+111.124226469" observedRunningTime="2026-05-29 16:58:41.51479806 +0000 UTC m=+112.001227263" watchObservedRunningTime="2026-05-29 16:58:41.516571688 +0000 UTC m=+112.003000893" May 29 16:58:41.607881 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:41.606776 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7594d44679-mkqlc"] May 29 16:58:41.612694 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:58:41.612660 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3686b4d1_9720_4c73_95ce_177c869d1c5f.slice/crio-67697488f67259477d58c8b0acbac2ebde3474a5aeb4ddeb5ba35c3d6ae8911a WatchSource:0}: Error finding container 67697488f67259477d58c8b0acbac2ebde3474a5aeb4ddeb5ba35c3d6ae8911a: Status 404 returned error can't find the container with id 67697488f67259477d58c8b0acbac2ebde3474a5aeb4ddeb5ba35c3d6ae8911a May 29 16:58:42.503319 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:42.503275 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-zn8jx" event={"ID":"0607511d-a31e-4794-b8b8-6a58179aabd0","Type":"ContainerStarted","Data":"6d55140183bf21fb3ef0824880311798d09f52704108a3bf79f67038cb698bf7"} May 29 16:58:42.504782 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:42.504749 2549 generic.go:358] "Generic (PLEG): container finished" podID="dfd131b4-94e6-4910-8548-de6c489f64ac" containerID="f63db06a9fc3f81a5c6eef3159e8948088c8e5b98b3e74c4ea42b10076122f21" exitCode=0 May 29 16:58:42.504919 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:42.504856 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9ddvs" event={"ID":"dfd131b4-94e6-4910-8548-de6c489f64ac","Type":"ContainerDied","Data":"f63db06a9fc3f81a5c6eef3159e8948088c8e5b98b3e74c4ea42b10076122f21"} May 29 16:58:42.506027 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:42.506002 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7594d44679-mkqlc" event={"ID":"3686b4d1-9720-4c73-95ce-177c869d1c5f","Type":"ContainerStarted","Data":"67697488f67259477d58c8b0acbac2ebde3474a5aeb4ddeb5ba35c3d6ae8911a"} May 29 16:58:42.520264 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:42.520211 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-85658796db-zn8jx" podStartSLOduration=3.239460018 podStartE2EDuration="4.520194995s" podCreationTimestamp="2026-05-29 16:58:38 +0000 UTC" firstStartedPulling="2026-05-29 16:58:40.197334727 +0000 UTC m=+110.683764046" lastFinishedPulling="2026-05-29 16:58:41.478069838 +0000 UTC m=+111.964499023" observedRunningTime="2026-05-29 16:58:42.517802947 +0000 UTC m=+113.004232150" watchObservedRunningTime="2026-05-29 16:58:42.520194995 +0000 UTC m=+113.006624198" May 29 16:58:43.003275 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:43.003182 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c8c9b6868-mm6qs" May 29 16:58:43.003275 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:43.003247 2549 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c8c9b6868-mm6qs" May 29 16:58:43.007942 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:43.007914 2549 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c8c9b6868-mm6qs" May 29 16:58:43.510506 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:43.510465 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9ddvs" event={"ID":"dfd131b4-94e6-4910-8548-de6c489f64ac","Type":"ContainerStarted","Data":"0418a29339333c72601b49bc3f3a420ac26ae55b13f8482566dddcbd021365d6"} May 29 16:58:43.510506 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:43.510509 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9ddvs" event={"ID":"dfd131b4-94e6-4910-8548-de6c489f64ac","Type":"ContainerStarted","Data":"7e90791ccb421fe5f426806732a344458067167516a2ab93ce8666d424e96d68"} May 29 16:58:43.515111 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:43.515082 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c8c9b6868-mm6qs" May 29 16:58:43.528468 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:43.528413 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-9ddvs" podStartSLOduration=3.988711094 podStartE2EDuration="5.528395281s" podCreationTimestamp="2026-05-29 16:58:38 +0000 UTC" firstStartedPulling="2026-05-29 16:58:39.937220461 +0000 UTC m=+110.423649647" lastFinishedPulling="2026-05-29 16:58:41.476904642 +0000 UTC m=+111.963333834" observedRunningTime="2026-05-29 16:58:43.526747146 +0000 UTC m=+114.013176348" watchObservedRunningTime="2026-05-29 16:58:43.528395281 +0000 UTC m=+114.014824487" May 29 16:58:44.096059 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:44.096033 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f7dcffc48-cbxdt"] May 29 16:58:44.099024 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:44.099009 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f7dcffc48-cbxdt" May 29 16:58:44.113435 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:44.111860 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f7dcffc48-cbxdt"] May 29 16:58:44.230589 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:44.230502 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d54083d9-f32e-43ff-9202-598c7d31a427-console-serving-cert\") pod \"console-5f7dcffc48-cbxdt\" (UID: \"d54083d9-f32e-43ff-9202-598c7d31a427\") " pod="openshift-console/console-5f7dcffc48-cbxdt" May 29 16:58:44.230589 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:44.230570 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwrjm\" (UniqueName: \"kubernetes.io/projected/d54083d9-f32e-43ff-9202-598c7d31a427-kube-api-access-xwrjm\") pod \"console-5f7dcffc48-cbxdt\" (UID: \"d54083d9-f32e-43ff-9202-598c7d31a427\") " pod="openshift-console/console-5f7dcffc48-cbxdt" May 29 16:58:44.230786 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:44.230599 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d54083d9-f32e-43ff-9202-598c7d31a427-console-config\") pod \"console-5f7dcffc48-cbxdt\" (UID: \"d54083d9-f32e-43ff-9202-598c7d31a427\") " pod="openshift-console/console-5f7dcffc48-cbxdt" May 29 16:58:44.230786 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:44.230699 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d54083d9-f32e-43ff-9202-598c7d31a427-trusted-ca-bundle\") pod \"console-5f7dcffc48-cbxdt\" (UID: \"d54083d9-f32e-43ff-9202-598c7d31a427\") " pod="openshift-console/console-5f7dcffc48-cbxdt" May 29 16:58:44.230786 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:44.230727 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d54083d9-f32e-43ff-9202-598c7d31a427-oauth-serving-cert\") pod \"console-5f7dcffc48-cbxdt\" (UID: \"d54083d9-f32e-43ff-9202-598c7d31a427\") " pod="openshift-console/console-5f7dcffc48-cbxdt" May 29 16:58:44.230786 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:44.230760 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d54083d9-f32e-43ff-9202-598c7d31a427-console-oauth-config\") pod \"console-5f7dcffc48-cbxdt\" (UID: \"d54083d9-f32e-43ff-9202-598c7d31a427\") " pod="openshift-console/console-5f7dcffc48-cbxdt" May 29 16:58:44.230786 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:44.230781 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d54083d9-f32e-43ff-9202-598c7d31a427-service-ca\") pod \"console-5f7dcffc48-cbxdt\" (UID: \"d54083d9-f32e-43ff-9202-598c7d31a427\") " pod="openshift-console/console-5f7dcffc48-cbxdt" May 29 16:58:44.331195 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:44.331156 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xwrjm\" (UniqueName: \"kubernetes.io/projected/d54083d9-f32e-43ff-9202-598c7d31a427-kube-api-access-xwrjm\") pod \"console-5f7dcffc48-cbxdt\" (UID: \"d54083d9-f32e-43ff-9202-598c7d31a427\") " pod="openshift-console/console-5f7dcffc48-cbxdt" May 29 16:58:44.331195 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:44.331202 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d54083d9-f32e-43ff-9202-598c7d31a427-console-config\") pod \"console-5f7dcffc48-cbxdt\" (UID: \"d54083d9-f32e-43ff-9202-598c7d31a427\") " pod="openshift-console/console-5f7dcffc48-cbxdt" May 29 16:58:44.331449 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:44.331387 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d54083d9-f32e-43ff-9202-598c7d31a427-trusted-ca-bundle\") pod \"console-5f7dcffc48-cbxdt\" (UID: \"d54083d9-f32e-43ff-9202-598c7d31a427\") " pod="openshift-console/console-5f7dcffc48-cbxdt" May 29 16:58:44.331449 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:44.331432 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d54083d9-f32e-43ff-9202-598c7d31a427-oauth-serving-cert\") pod \"console-5f7dcffc48-cbxdt\" (UID: \"d54083d9-f32e-43ff-9202-598c7d31a427\") " pod="openshift-console/console-5f7dcffc48-cbxdt" May 29 16:58:44.331555 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:44.331483 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d54083d9-f32e-43ff-9202-598c7d31a427-console-oauth-config\") pod \"console-5f7dcffc48-cbxdt\" (UID: \"d54083d9-f32e-43ff-9202-598c7d31a427\") " pod="openshift-console/console-5f7dcffc48-cbxdt" May 29 16:58:44.331555 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:44.331510 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d54083d9-f32e-43ff-9202-598c7d31a427-service-ca\") pod \"console-5f7dcffc48-cbxdt\" (UID: \"d54083d9-f32e-43ff-9202-598c7d31a427\") " pod="openshift-console/console-5f7dcffc48-cbxdt" May 29 16:58:44.331646 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:44.331556 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d54083d9-f32e-43ff-9202-598c7d31a427-console-serving-cert\") pod \"console-5f7dcffc48-cbxdt\" (UID: \"d54083d9-f32e-43ff-9202-598c7d31a427\") " pod="openshift-console/console-5f7dcffc48-cbxdt" May 29 16:58:44.331956 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:44.331931 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d54083d9-f32e-43ff-9202-598c7d31a427-console-config\") pod \"console-5f7dcffc48-cbxdt\" (UID: \"d54083d9-f32e-43ff-9202-598c7d31a427\") " pod="openshift-console/console-5f7dcffc48-cbxdt" May 29 16:58:44.332205 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:44.332179 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d54083d9-f32e-43ff-9202-598c7d31a427-trusted-ca-bundle\") pod \"console-5f7dcffc48-cbxdt\" (UID: \"d54083d9-f32e-43ff-9202-598c7d31a427\") " pod="openshift-console/console-5f7dcffc48-cbxdt" May 29 16:58:44.332283 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:44.332253 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d54083d9-f32e-43ff-9202-598c7d31a427-service-ca\") pod \"console-5f7dcffc48-cbxdt\" (UID: \"d54083d9-f32e-43ff-9202-598c7d31a427\") " pod="openshift-console/console-5f7dcffc48-cbxdt" May 29 16:58:44.332283 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:44.332253 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d54083d9-f32e-43ff-9202-598c7d31a427-oauth-serving-cert\") pod \"console-5f7dcffc48-cbxdt\" (UID: \"d54083d9-f32e-43ff-9202-598c7d31a427\") " pod="openshift-console/console-5f7dcffc48-cbxdt" May 29 16:58:44.333964 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:44.333944 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d54083d9-f32e-43ff-9202-598c7d31a427-console-oauth-config\") pod \"console-5f7dcffc48-cbxdt\" (UID: \"d54083d9-f32e-43ff-9202-598c7d31a427\") " pod="openshift-console/console-5f7dcffc48-cbxdt" May 29 16:58:44.334080 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:44.334064 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d54083d9-f32e-43ff-9202-598c7d31a427-console-serving-cert\") pod \"console-5f7dcffc48-cbxdt\" (UID: \"d54083d9-f32e-43ff-9202-598c7d31a427\") " pod="openshift-console/console-5f7dcffc48-cbxdt" May 29 16:58:44.340662 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:44.340634 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwrjm\" (UniqueName: \"kubernetes.io/projected/d54083d9-f32e-43ff-9202-598c7d31a427-kube-api-access-xwrjm\") pod \"console-5f7dcffc48-cbxdt\" (UID: \"d54083d9-f32e-43ff-9202-598c7d31a427\") " pod="openshift-console/console-5f7dcffc48-cbxdt" May 29 16:58:44.409716 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:44.409676 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f7dcffc48-cbxdt" May 29 16:58:44.515237 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:44.515193 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7594d44679-mkqlc" event={"ID":"3686b4d1-9720-4c73-95ce-177c869d1c5f","Type":"ContainerStarted","Data":"a3c4d91321c0795d0fb318c8ec00bf56f0505747d1ca2db656209f2d13bf4dbd"} May 29 16:58:44.515237 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:44.515236 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7594d44679-mkqlc" event={"ID":"3686b4d1-9720-4c73-95ce-177c869d1c5f","Type":"ContainerStarted","Data":"a5e13ffbf32d302c73b5454caa46e4dcd257b61ff41deec94fb39d297a6c36a8"} May 29 16:58:44.515602 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:44.515248 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7594d44679-mkqlc" event={"ID":"3686b4d1-9720-4c73-95ce-177c869d1c5f","Type":"ContainerStarted","Data":"7a1c8e8cd70e0bdfa5e5f945b0b6a2d9a08c402a9a3f364b67ad3176d6419d9f"} May 29 16:58:44.529594 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:44.529558 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f7dcffc48-cbxdt"] May 29 16:58:44.532689 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:58:44.532658 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd54083d9_f32e_43ff_9202_598c7d31a427.slice/crio-3c5e92a0b631d63c405efe1bf12ea256afe90fda7e321d6e8e02d76fa3ca87b8 WatchSource:0}: Error finding container 3c5e92a0b631d63c405efe1bf12ea256afe90fda7e321d6e8e02d76fa3ca87b8: Status 404 returned error can't find the container with id 3c5e92a0b631d63c405efe1bf12ea256afe90fda7e321d6e8e02d76fa3ca87b8 May 29 16:58:45.307916 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.307891 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] May 29 16:58:45.312899 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.312876 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.314880 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.314858 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" May 29 16:58:45.315286 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.315261 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" May 29 16:58:45.315383 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.315358 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" May 29 16:58:45.315383 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.315371 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" May 29 16:58:45.315524 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.315370 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" May 29 16:58:45.315524 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.315274 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" May 29 16:58:45.315524 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.315482 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" May 29 16:58:45.315770 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.315736 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" May 29 16:58:45.315770 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.315760 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" May 29 16:58:45.315917 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.315901 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-62ssvscooh466\"" May 29 16:58:45.316036 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.316016 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" May 29 16:58:45.316332 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.316314 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-v78cc\"" May 29 16:58:45.316413 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.316334 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" May 29 16:58:45.318484 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.318469 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" May 29 16:58:45.321798 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.321781 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" May 29 16:58:45.324033 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.324012 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] May 29 16:58:45.441728 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.441630 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.441728 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.441682 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.441728 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.441709 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.441990 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.441754 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.441990 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.441777 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.441990 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.441831 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhnpp\" (UniqueName: \"kubernetes.io/projected/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-kube-api-access-vhnpp\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.441990 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.441894 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.441990 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.441921 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-config\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.441990 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.441963 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.441990 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.441982 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.442215 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.442010 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-config-out\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.442215 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.442035 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.442215 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.442064 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.442215 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.442080 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.442215 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.442099 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.442215 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.442116 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.442215 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.442185 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-web-config\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.442439 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.442216 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.519829 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.519795 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f7dcffc48-cbxdt" event={"ID":"d54083d9-f32e-43ff-9202-598c7d31a427","Type":"ContainerStarted","Data":"32642da806334d521121851d1eaaacb709ebb9cd11116ce987081f3413be9683"} May 29 16:58:45.519829 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.519830 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f7dcffc48-cbxdt" event={"ID":"d54083d9-f32e-43ff-9202-598c7d31a427","Type":"ContainerStarted","Data":"3c5e92a0b631d63c405efe1bf12ea256afe90fda7e321d6e8e02d76fa3ca87b8"} May 29 16:58:45.522353 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.522319 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7594d44679-mkqlc" event={"ID":"3686b4d1-9720-4c73-95ce-177c869d1c5f","Type":"ContainerStarted","Data":"4fb640980cc0c35bf525d5a5f98970b3e2004170783f1cb64300133373930041"} May 29 16:58:45.522353 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.522347 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7594d44679-mkqlc" event={"ID":"3686b4d1-9720-4c73-95ce-177c869d1c5f","Type":"ContainerStarted","Data":"ae963b5dd0852bae89894273c77e915b7b929ca8dd805aba43599cb83293b540"} May 29 16:58:45.522353 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.522356 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7594d44679-mkqlc" event={"ID":"3686b4d1-9720-4c73-95ce-177c869d1c5f","Type":"ContainerStarted","Data":"6aa64c540e942ba9ff77d7c27ce68cdd5e92e5751a9dcd042ee604b6e5088384"} May 29 16:58:45.522562 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.522546 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7594d44679-mkqlc" May 29 16:58:45.536173 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.536133 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-5f7dcffc48-cbxdt" podStartSLOduration=1.536121436 podStartE2EDuration="1.536121436s" podCreationTimestamp="2026-05-29 16:58:44 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 16:58:45.534787293 +0000 UTC m=+116.021216496" watchObservedRunningTime="2026-05-29 16:58:45.536121436 +0000 UTC m=+116.022550635" May 29 16:58:45.543520 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.543494 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.543610 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.543523 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.543610 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.543573 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-web-config\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.543696 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.543606 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.543696 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.543646 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.543696 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.543675 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.544030 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.543701 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.544030 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.543728 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.544030 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.543775 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.544030 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.543805 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vhnpp\" (UniqueName: \"kubernetes.io/projected/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-kube-api-access-vhnpp\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.544030 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.543834 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.544030 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.543891 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-config\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.544030 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.543929 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.544030 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.543958 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.544030 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.544000 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-config-out\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.544030 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.544020 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.544536 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.544045 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.544536 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.544074 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.544536 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.544114 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.544685 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.544648 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.544799 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.544774 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.545401 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.545375 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.545401 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.545389 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.547061 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.547033 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.547706 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.547679 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.548212 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.548191 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.548516 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.548496 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.548580 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.548559 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.549012 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.548986 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.549149 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.549128 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-web-config\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.549218 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.549196 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.549475 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.549448 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-config-out\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.549475 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.549466 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-config\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.549740 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.549723 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.550538 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.550516 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.551506 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.551486 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhnpp\" (UniqueName: \"kubernetes.io/projected/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-kube-api-access-vhnpp\") pod \"prometheus-k8s-0\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.556608 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.556570 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7594d44679-mkqlc" podStartSLOduration=1.00165507 podStartE2EDuration="4.556558648s" podCreationTimestamp="2026-05-29 16:58:41 +0000 UTC" firstStartedPulling="2026-05-29 16:58:41.614832288 +0000 UTC m=+112.101261473" lastFinishedPulling="2026-05-29 16:58:45.169735871 +0000 UTC m=+115.656165051" observedRunningTime="2026-05-29 16:58:45.554666588 +0000 UTC m=+116.041095789" watchObservedRunningTime="2026-05-29 16:58:45.556558648 +0000 UTC m=+116.042987850" May 29 16:58:45.639531 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.639497 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:45.770905 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:45.770878 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] May 29 16:58:45.772751 ip-10-0-141-197 kubenswrapper[2549]: W0529 16:58:45.772721 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb595abfc_f4c3_43b7_94b9_b6eb00520ca8.slice/crio-193143247b16268f8af2ccf51b9a9f1267796c5a24ac63fa2399d3b979854b88 WatchSource:0}: Error finding container 193143247b16268f8af2ccf51b9a9f1267796c5a24ac63fa2399d3b979854b88: Status 404 returned error can't find the container with id 193143247b16268f8af2ccf51b9a9f1267796c5a24ac63fa2399d3b979854b88 May 29 16:58:46.526913 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:46.526864 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b595abfc-f4c3-43b7-94b9-b6eb00520ca8","Type":"ContainerStarted","Data":"193143247b16268f8af2ccf51b9a9f1267796c5a24ac63fa2399d3b979854b88"} May 29 16:58:47.531291 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:47.531249 2549 generic.go:358] "Generic (PLEG): container finished" podID="b595abfc-f4c3-43b7-94b9-b6eb00520ca8" containerID="ddc7d497a54ac3be13465e507ce387e46bef9ea748aebb3eded5bbe472c5ddb8" exitCode=0 May 29 16:58:47.531634 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:47.531335 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b595abfc-f4c3-43b7-94b9-b6eb00520ca8","Type":"ContainerDied","Data":"ddc7d497a54ac3be13465e507ce387e46bef9ea748aebb3eded5bbe472c5ddb8"} May 29 16:58:50.648660 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:50.648581 2549 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f7dcffc48-cbxdt"] May 29 16:58:51.534211 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:51.534180 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7594d44679-mkqlc" May 29 16:58:51.545134 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:51.545103 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b595abfc-f4c3-43b7-94b9-b6eb00520ca8","Type":"ContainerStarted","Data":"d4afe07b4abd51548797adc00a40b267f62379b7139fa8569aee6c5a98c4f1f4"} May 29 16:58:51.545134 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:51.545136 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b595abfc-f4c3-43b7-94b9-b6eb00520ca8","Type":"ContainerStarted","Data":"6d8b6580d1d20aba4363be3f701ad3d3db74f1186745d56e714f5678e4de440f"} May 29 16:58:51.545313 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:51.545146 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b595abfc-f4c3-43b7-94b9-b6eb00520ca8","Type":"ContainerStarted","Data":"d295027d80b4a8cbfc3cbc5a627a6e7f1b172228c8e8e5eef967d194590b7968"} May 29 16:58:51.545313 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:51.545155 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b595abfc-f4c3-43b7-94b9-b6eb00520ca8","Type":"ContainerStarted","Data":"f1a278f7d38c84c4614b922dc3772442d3e0ebe391f9f05fc05e66d03efecb66"} May 29 16:58:51.545313 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:51.545164 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b595abfc-f4c3-43b7-94b9-b6eb00520ca8","Type":"ContainerStarted","Data":"a84f0bdc2a0fe70215697f108054fe50419efb6b8fc2295e28b6d9b4d9fe78b0"} May 29 16:58:51.545313 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:51.545171 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b595abfc-f4c3-43b7-94b9-b6eb00520ca8","Type":"ContainerStarted","Data":"0198c250e9fdde86bd791e4add6349098d34bd5257b132452d885c4f970a12db"} May 29 16:58:51.579802 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:51.579740 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=1.829266133 podStartE2EDuration="6.579719944s" podCreationTimestamp="2026-05-29 16:58:45 +0000 UTC" firstStartedPulling="2026-05-29 16:58:45.77464236 +0000 UTC m=+116.261071543" lastFinishedPulling="2026-05-29 16:58:50.525096172 +0000 UTC m=+121.011525354" observedRunningTime="2026-05-29 16:58:51.577594679 +0000 UTC m=+122.064023881" watchObservedRunningTime="2026-05-29 16:58:51.579719944 +0000 UTC m=+122.066149147" May 29 16:58:54.410093 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:54.410057 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f7dcffc48-cbxdt" May 29 16:58:55.640579 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:55.640536 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" May 29 16:58:59.844985 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:58:59.844957 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wnqbv_17e14531-cc91-412b-9804-9c1c9fe52eb8/serve-healthcheck-canary/0.log" May 29 16:59:01.832623 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:01.832590 2549 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c8c9b6868-mm6qs"] May 29 16:59:15.675934 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:15.675876 2549 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-5f7dcffc48-cbxdt" podUID="d54083d9-f32e-43ff-9202-598c7d31a427" containerName="console" containerID="cri-o://32642da806334d521121851d1eaaacb709ebb9cd11116ce987081f3413be9683" gracePeriod=15 May 29 16:59:15.913003 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:15.912983 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5f7dcffc48-cbxdt_d54083d9-f32e-43ff-9202-598c7d31a427/console/0.log" May 29 16:59:15.913100 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:15.913039 2549 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f7dcffc48-cbxdt" May 29 16:59:16.009783 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:16.009705 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d54083d9-f32e-43ff-9202-598c7d31a427-oauth-serving-cert\") pod \"d54083d9-f32e-43ff-9202-598c7d31a427\" (UID: \"d54083d9-f32e-43ff-9202-598c7d31a427\") " May 29 16:59:16.009783 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:16.009755 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d54083d9-f32e-43ff-9202-598c7d31a427-console-serving-cert\") pod \"d54083d9-f32e-43ff-9202-598c7d31a427\" (UID: \"d54083d9-f32e-43ff-9202-598c7d31a427\") " May 29 16:59:16.009783 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:16.009776 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d54083d9-f32e-43ff-9202-598c7d31a427-console-config\") pod \"d54083d9-f32e-43ff-9202-598c7d31a427\" (UID: \"d54083d9-f32e-43ff-9202-598c7d31a427\") " May 29 16:59:16.010055 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:16.009862 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d54083d9-f32e-43ff-9202-598c7d31a427-service-ca\") pod \"d54083d9-f32e-43ff-9202-598c7d31a427\" (UID: \"d54083d9-f32e-43ff-9202-598c7d31a427\") " May 29 16:59:16.010055 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:16.009947 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xwrjm\" (UniqueName: \"kubernetes.io/projected/d54083d9-f32e-43ff-9202-598c7d31a427-kube-api-access-xwrjm\") pod \"d54083d9-f32e-43ff-9202-598c7d31a427\" (UID: \"d54083d9-f32e-43ff-9202-598c7d31a427\") " May 29 16:59:16.010055 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:16.009973 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d54083d9-f32e-43ff-9202-598c7d31a427-console-oauth-config\") pod \"d54083d9-f32e-43ff-9202-598c7d31a427\" (UID: \"d54083d9-f32e-43ff-9202-598c7d31a427\") " May 29 16:59:16.010055 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:16.010001 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d54083d9-f32e-43ff-9202-598c7d31a427-trusted-ca-bundle\") pod \"d54083d9-f32e-43ff-9202-598c7d31a427\" (UID: \"d54083d9-f32e-43ff-9202-598c7d31a427\") " May 29 16:59:16.010257 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:16.010203 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d54083d9-f32e-43ff-9202-598c7d31a427-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d54083d9-f32e-43ff-9202-598c7d31a427" (UID: "d54083d9-f32e-43ff-9202-598c7d31a427"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 29 16:59:16.010257 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:16.010215 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d54083d9-f32e-43ff-9202-598c7d31a427-console-config" (OuterVolumeSpecName: "console-config") pod "d54083d9-f32e-43ff-9202-598c7d31a427" (UID: "d54083d9-f32e-43ff-9202-598c7d31a427"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 29 16:59:16.010361 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:16.010305 2549 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d54083d9-f32e-43ff-9202-598c7d31a427-console-config\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 16:59:16.010361 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:16.010302 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d54083d9-f32e-43ff-9202-598c7d31a427-service-ca" (OuterVolumeSpecName: "service-ca") pod "d54083d9-f32e-43ff-9202-598c7d31a427" (UID: "d54083d9-f32e-43ff-9202-598c7d31a427"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 29 16:59:16.010361 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:16.010321 2549 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d54083d9-f32e-43ff-9202-598c7d31a427-oauth-serving-cert\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 16:59:16.010589 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:16.010567 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d54083d9-f32e-43ff-9202-598c7d31a427-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d54083d9-f32e-43ff-9202-598c7d31a427" (UID: "d54083d9-f32e-43ff-9202-598c7d31a427"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 29 16:59:16.012090 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:16.012060 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d54083d9-f32e-43ff-9202-598c7d31a427-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d54083d9-f32e-43ff-9202-598c7d31a427" (UID: "d54083d9-f32e-43ff-9202-598c7d31a427"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 29 16:59:16.012174 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:16.012118 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d54083d9-f32e-43ff-9202-598c7d31a427-kube-api-access-xwrjm" (OuterVolumeSpecName: "kube-api-access-xwrjm") pod "d54083d9-f32e-43ff-9202-598c7d31a427" (UID: "d54083d9-f32e-43ff-9202-598c7d31a427"). InnerVolumeSpecName "kube-api-access-xwrjm". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 29 16:59:16.012174 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:16.012168 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d54083d9-f32e-43ff-9202-598c7d31a427-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d54083d9-f32e-43ff-9202-598c7d31a427" (UID: "d54083d9-f32e-43ff-9202-598c7d31a427"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 29 16:59:16.112225 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:16.112195 2549 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d54083d9-f32e-43ff-9202-598c7d31a427-console-serving-cert\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 16:59:16.112225 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:16.112224 2549 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d54083d9-f32e-43ff-9202-598c7d31a427-service-ca\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 16:59:16.112371 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:16.112235 2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xwrjm\" (UniqueName: \"kubernetes.io/projected/d54083d9-f32e-43ff-9202-598c7d31a427-kube-api-access-xwrjm\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 16:59:16.112371 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:16.112245 2549 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d54083d9-f32e-43ff-9202-598c7d31a427-console-oauth-config\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 16:59:16.112371 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:16.112255 2549 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d54083d9-f32e-43ff-9202-598c7d31a427-trusted-ca-bundle\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 16:59:16.624434 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:16.624405 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5f7dcffc48-cbxdt_d54083d9-f32e-43ff-9202-598c7d31a427/console/0.log" May 29 16:59:16.624603 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:16.624443 2549 generic.go:358] "Generic (PLEG): container finished" podID="d54083d9-f32e-43ff-9202-598c7d31a427" containerID="32642da806334d521121851d1eaaacb709ebb9cd11116ce987081f3413be9683" exitCode=2 May 29 16:59:16.624603 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:16.624509 2549 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f7dcffc48-cbxdt" May 29 16:59:16.624603 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:16.624519 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f7dcffc48-cbxdt" event={"ID":"d54083d9-f32e-43ff-9202-598c7d31a427","Type":"ContainerDied","Data":"32642da806334d521121851d1eaaacb709ebb9cd11116ce987081f3413be9683"} May 29 16:59:16.624603 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:16.624545 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f7dcffc48-cbxdt" event={"ID":"d54083d9-f32e-43ff-9202-598c7d31a427","Type":"ContainerDied","Data":"3c5e92a0b631d63c405efe1bf12ea256afe90fda7e321d6e8e02d76fa3ca87b8"} May 29 16:59:16.624603 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:16.624558 2549 scope.go:117] "RemoveContainer" containerID="32642da806334d521121851d1eaaacb709ebb9cd11116ce987081f3413be9683" May 29 16:59:16.653429 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:16.653409 2549 scope.go:117] "RemoveContainer" containerID="32642da806334d521121851d1eaaacb709ebb9cd11116ce987081f3413be9683" May 29 16:59:16.653710 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:59:16.653685 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32642da806334d521121851d1eaaacb709ebb9cd11116ce987081f3413be9683\": container with ID starting with 32642da806334d521121851d1eaaacb709ebb9cd11116ce987081f3413be9683 not found: ID does not exist" containerID="32642da806334d521121851d1eaaacb709ebb9cd11116ce987081f3413be9683" May 29 16:59:16.653810 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:16.653713 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32642da806334d521121851d1eaaacb709ebb9cd11116ce987081f3413be9683"} err="failed to get container status \"32642da806334d521121851d1eaaacb709ebb9cd11116ce987081f3413be9683\": rpc error: code = NotFound desc = could not find container \"32642da806334d521121851d1eaaacb709ebb9cd11116ce987081f3413be9683\": container with ID starting with 32642da806334d521121851d1eaaacb709ebb9cd11116ce987081f3413be9683 not found: ID does not exist" May 29 16:59:16.654050 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:16.654033 2549 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f7dcffc48-cbxdt"] May 29 16:59:16.659333 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:16.659309 2549 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5f7dcffc48-cbxdt"] May 29 16:59:18.084608 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:18.084573 2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d54083d9-f32e-43ff-9202-598c7d31a427" path="/var/lib/kubelet/pods/d54083d9-f32e-43ff-9202-598c7d31a427/volumes" May 29 16:59:26.852017 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:26.851971 2549 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-7c8c9b6868-mm6qs" podUID="54db9067-d609-4bea-a927-c91c52543add" containerName="console" containerID="cri-o://c42f0df2a6ac4796b1b410cd9081bd6235ae7f23706e375e2c20467c90938721" gracePeriod=15 May 29 16:59:27.094919 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:27.094897 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7c8c9b6868-mm6qs_54db9067-d609-4bea-a927-c91c52543add/console/0.log" May 29 16:59:27.095018 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:27.094957 2549 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c8c9b6868-mm6qs" May 29 16:59:27.203947 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:27.203864 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/54db9067-d609-4bea-a927-c91c52543add-service-ca\") pod \"54db9067-d609-4bea-a927-c91c52543add\" (UID: \"54db9067-d609-4bea-a927-c91c52543add\") " May 29 16:59:27.203947 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:27.203904 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/54db9067-d609-4bea-a927-c91c52543add-console-config\") pod \"54db9067-d609-4bea-a927-c91c52543add\" (UID: \"54db9067-d609-4bea-a927-c91c52543add\") " May 29 16:59:27.204139 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:27.203959 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gb55w\" (UniqueName: \"kubernetes.io/projected/54db9067-d609-4bea-a927-c91c52543add-kube-api-access-gb55w\") pod \"54db9067-d609-4bea-a927-c91c52543add\" (UID: \"54db9067-d609-4bea-a927-c91c52543add\") " May 29 16:59:27.204139 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:27.203987 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/54db9067-d609-4bea-a927-c91c52543add-console-oauth-config\") pod \"54db9067-d609-4bea-a927-c91c52543add\" (UID: \"54db9067-d609-4bea-a927-c91c52543add\") " May 29 16:59:27.204139 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:27.204030 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54db9067-d609-4bea-a927-c91c52543add-trusted-ca-bundle\") pod \"54db9067-d609-4bea-a927-c91c52543add\" (UID: \"54db9067-d609-4bea-a927-c91c52543add\") " May 29 16:59:27.204139 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:27.204067 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/54db9067-d609-4bea-a927-c91c52543add-console-serving-cert\") pod \"54db9067-d609-4bea-a927-c91c52543add\" (UID: \"54db9067-d609-4bea-a927-c91c52543add\") " May 29 16:59:27.204139 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:27.204100 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/54db9067-d609-4bea-a927-c91c52543add-oauth-serving-cert\") pod \"54db9067-d609-4bea-a927-c91c52543add\" (UID: \"54db9067-d609-4bea-a927-c91c52543add\") " May 29 16:59:27.204383 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:27.204206 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54db9067-d609-4bea-a927-c91c52543add-service-ca" (OuterVolumeSpecName: "service-ca") pod "54db9067-d609-4bea-a927-c91c52543add" (UID: "54db9067-d609-4bea-a927-c91c52543add"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 29 16:59:27.204383 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:27.204348 2549 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/54db9067-d609-4bea-a927-c91c52543add-service-ca\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 16:59:27.204495 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:27.204355 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54db9067-d609-4bea-a927-c91c52543add-console-config" (OuterVolumeSpecName: "console-config") pod "54db9067-d609-4bea-a927-c91c52543add" (UID: "54db9067-d609-4bea-a927-c91c52543add"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 29 16:59:27.204550 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:27.204492 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54db9067-d609-4bea-a927-c91c52543add-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "54db9067-d609-4bea-a927-c91c52543add" (UID: "54db9067-d609-4bea-a927-c91c52543add"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 29 16:59:27.204550 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:27.204535 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54db9067-d609-4bea-a927-c91c52543add-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "54db9067-d609-4bea-a927-c91c52543add" (UID: "54db9067-d609-4bea-a927-c91c52543add"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 29 16:59:27.206121 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:27.206095 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54db9067-d609-4bea-a927-c91c52543add-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "54db9067-d609-4bea-a927-c91c52543add" (UID: "54db9067-d609-4bea-a927-c91c52543add"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 29 16:59:27.206230 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:27.206117 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54db9067-d609-4bea-a927-c91c52543add-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "54db9067-d609-4bea-a927-c91c52543add" (UID: "54db9067-d609-4bea-a927-c91c52543add"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 29 16:59:27.206230 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:27.206177 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54db9067-d609-4bea-a927-c91c52543add-kube-api-access-gb55w" (OuterVolumeSpecName: "kube-api-access-gb55w") pod "54db9067-d609-4bea-a927-c91c52543add" (UID: "54db9067-d609-4bea-a927-c91c52543add"). InnerVolumeSpecName "kube-api-access-gb55w". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 29 16:59:27.305742 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:27.305710 2549 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/54db9067-d609-4bea-a927-c91c52543add-console-config\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 16:59:27.305742 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:27.305736 2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gb55w\" (UniqueName: \"kubernetes.io/projected/54db9067-d609-4bea-a927-c91c52543add-kube-api-access-gb55w\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 16:59:27.305742 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:27.305747 2549 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/54db9067-d609-4bea-a927-c91c52543add-console-oauth-config\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 16:59:27.305967 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:27.305756 2549 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54db9067-d609-4bea-a927-c91c52543add-trusted-ca-bundle\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 16:59:27.305967 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:27.305765 2549 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/54db9067-d609-4bea-a927-c91c52543add-console-serving-cert\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 16:59:27.305967 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:27.305773 2549 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/54db9067-d609-4bea-a927-c91c52543add-oauth-serving-cert\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 16:59:27.671189 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:27.667360 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7c8c9b6868-mm6qs_54db9067-d609-4bea-a927-c91c52543add/console/0.log" May 29 16:59:27.671189 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:27.667412 2549 generic.go:358] "Generic (PLEG): container finished" podID="54db9067-d609-4bea-a927-c91c52543add" containerID="c42f0df2a6ac4796b1b410cd9081bd6235ae7f23706e375e2c20467c90938721" exitCode=2 May 29 16:59:27.671189 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:27.667499 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c8c9b6868-mm6qs" event={"ID":"54db9067-d609-4bea-a927-c91c52543add","Type":"ContainerDied","Data":"c42f0df2a6ac4796b1b410cd9081bd6235ae7f23706e375e2c20467c90938721"} May 29 16:59:27.671189 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:27.667531 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c8c9b6868-mm6qs" event={"ID":"54db9067-d609-4bea-a927-c91c52543add","Type":"ContainerDied","Data":"ef1b94cc3ff0dc30402234788a647e9b4cb18af7769620d6ad65e0e2f9ef1ac1"} May 29 16:59:27.671189 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:27.667552 2549 scope.go:117] "RemoveContainer" containerID="c42f0df2a6ac4796b1b410cd9081bd6235ae7f23706e375e2c20467c90938721" May 29 16:59:27.671189 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:27.667738 2549 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c8c9b6868-mm6qs" May 29 16:59:27.681306 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:27.681287 2549 scope.go:117] "RemoveContainer" containerID="c42f0df2a6ac4796b1b410cd9081bd6235ae7f23706e375e2c20467c90938721" May 29 16:59:27.681546 ip-10-0-141-197 kubenswrapper[2549]: E0529 16:59:27.681527 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c42f0df2a6ac4796b1b410cd9081bd6235ae7f23706e375e2c20467c90938721\": container with ID starting with c42f0df2a6ac4796b1b410cd9081bd6235ae7f23706e375e2c20467c90938721 not found: ID does not exist" containerID="c42f0df2a6ac4796b1b410cd9081bd6235ae7f23706e375e2c20467c90938721" May 29 16:59:27.681610 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:27.681553 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c42f0df2a6ac4796b1b410cd9081bd6235ae7f23706e375e2c20467c90938721"} err="failed to get container status \"c42f0df2a6ac4796b1b410cd9081bd6235ae7f23706e375e2c20467c90938721\": rpc error: code = NotFound desc = could not find container \"c42f0df2a6ac4796b1b410cd9081bd6235ae7f23706e375e2c20467c90938721\": container with ID starting with c42f0df2a6ac4796b1b410cd9081bd6235ae7f23706e375e2c20467c90938721 not found: ID does not exist" May 29 16:59:27.692266 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:27.692241 2549 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c8c9b6868-mm6qs"] May 29 16:59:27.695479 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:27.695455 2549 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c8c9b6868-mm6qs"] May 29 16:59:28.084130 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:28.084090 2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54db9067-d609-4bea-a927-c91c52543add" path="/var/lib/kubelet/pods/54db9067-d609-4bea-a927-c91c52543add/volumes" May 29 16:59:45.640484 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:45.640445 2549 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" May 29 16:59:45.660665 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:45.660640 2549 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" May 29 16:59:45.732590 ip-10-0-141-197 kubenswrapper[2549]: I0529 16:59:45.732560 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:03.691678 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:03.691639 2549 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] May 29 17:00:03.692160 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:03.692130 2549 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b595abfc-f4c3-43b7-94b9-b6eb00520ca8" containerName="prometheus" containerID="cri-o://0198c250e9fdde86bd791e4add6349098d34bd5257b132452d885c4f970a12db" gracePeriod=600 May 29 17:00:03.692160 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:03.692154 2549 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b595abfc-f4c3-43b7-94b9-b6eb00520ca8" containerName="kube-rbac-proxy" containerID="cri-o://6d8b6580d1d20aba4363be3f701ad3d3db74f1186745d56e714f5678e4de440f" gracePeriod=600 May 29 17:00:03.692374 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:03.692189 2549 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b595abfc-f4c3-43b7-94b9-b6eb00520ca8" containerName="kube-rbac-proxy-web" containerID="cri-o://d295027d80b4a8cbfc3cbc5a627a6e7f1b172228c8e8e5eef967d194590b7968" gracePeriod=600 May 29 17:00:03.692374 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:03.692161 2549 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b595abfc-f4c3-43b7-94b9-b6eb00520ca8" containerName="thanos-sidecar" containerID="cri-o://f1a278f7d38c84c4614b922dc3772442d3e0ebe391f9f05fc05e66d03efecb66" gracePeriod=600 May 29 17:00:03.692374 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:03.692230 2549 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b595abfc-f4c3-43b7-94b9-b6eb00520ca8" containerName="kube-rbac-proxy-thanos" containerID="cri-o://d4afe07b4abd51548797adc00a40b267f62379b7139fa8569aee6c5a98c4f1f4" gracePeriod=600 May 29 17:00:03.692374 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:03.692227 2549 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b595abfc-f4c3-43b7-94b9-b6eb00520ca8" containerName="config-reloader" containerID="cri-o://a84f0bdc2a0fe70215697f108054fe50419efb6b8fc2295e28b6d9b4d9fe78b0" gracePeriod=600 May 29 17:00:03.931091 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:03.931068 2549 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:04.000191 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.000112 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " May 29 17:00:04.000191 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.000162 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " May 29 17:00:04.000388 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.000194 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-prometheus-k8s-db\") pod \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " May 29 17:00:04.000388 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.000223 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-thanos-prometheus-http-client-file\") pod \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " May 29 17:00:04.000388 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.000259 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-config-out\") pod \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " May 29 17:00:04.000388 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.000288 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-config\") pod \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " May 29 17:00:04.000388 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.000320 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-secret-metrics-client-certs\") pod \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " May 29 17:00:04.000388 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.000344 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-tls-assets\") pod \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " May 29 17:00:04.000388 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.000381 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-prometheus-trusted-ca-bundle\") pod \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " May 29 17:00:04.000728 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.000415 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-configmap-kubelet-serving-ca-bundle\") pod \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " May 29 17:00:04.000728 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.000439 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-web-config\") pod \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " May 29 17:00:04.000728 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.000496 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-prometheus-k8s-rulefiles-0\") pod \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " May 29 17:00:04.000728 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.000523 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-secret-kube-rbac-proxy\") pod \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " May 29 17:00:04.000728 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.000555 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vhnpp\" (UniqueName: \"kubernetes.io/projected/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-kube-api-access-vhnpp\") pod \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " May 29 17:00:04.000728 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.000587 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-configmap-serving-certs-ca-bundle\") pod \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " May 29 17:00:04.000728 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.000612 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-configmap-metrics-client-ca\") pod \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " May 29 17:00:04.000728 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.000654 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-secret-prometheus-k8s-tls\") pod \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " May 29 17:00:04.000728 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.000678 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-secret-grpc-tls\") pod \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\" (UID: \"b595abfc-f4c3-43b7-94b9-b6eb00520ca8\") " May 29 17:00:04.001625 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.001590 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "b595abfc-f4c3-43b7-94b9-b6eb00520ca8" (UID: "b595abfc-f4c3-43b7-94b9-b6eb00520ca8"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 29 17:00:04.002110 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.002077 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "b595abfc-f4c3-43b7-94b9-b6eb00520ca8" (UID: "b595abfc-f4c3-43b7-94b9-b6eb00520ca8"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 29 17:00:04.002110 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.002093 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "b595abfc-f4c3-43b7-94b9-b6eb00520ca8" (UID: "b595abfc-f4c3-43b7-94b9-b6eb00520ca8"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 29 17:00:04.003217 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.002579 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "b595abfc-f4c3-43b7-94b9-b6eb00520ca8" (UID: "b595abfc-f4c3-43b7-94b9-b6eb00520ca8"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 29 17:00:04.003217 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.002977 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "b595abfc-f4c3-43b7-94b9-b6eb00520ca8" (UID: "b595abfc-f4c3-43b7-94b9-b6eb00520ca8"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 29 17:00:04.003217 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.003093 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "b595abfc-f4c3-43b7-94b9-b6eb00520ca8" (UID: "b595abfc-f4c3-43b7-94b9-b6eb00520ca8"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 29 17:00:04.003559 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.003530 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "b595abfc-f4c3-43b7-94b9-b6eb00520ca8" (UID: "b595abfc-f4c3-43b7-94b9-b6eb00520ca8"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 29 17:00:04.004669 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.004626 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "b595abfc-f4c3-43b7-94b9-b6eb00520ca8" (UID: "b595abfc-f4c3-43b7-94b9-b6eb00520ca8"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 29 17:00:04.004771 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.004749 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "b595abfc-f4c3-43b7-94b9-b6eb00520ca8" (UID: "b595abfc-f4c3-43b7-94b9-b6eb00520ca8"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 29 17:00:04.004949 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.004892 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "b595abfc-f4c3-43b7-94b9-b6eb00520ca8" (UID: "b595abfc-f4c3-43b7-94b9-b6eb00520ca8"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 29 17:00:04.004949 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.004910 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-config" (OuterVolumeSpecName: "config") pod "b595abfc-f4c3-43b7-94b9-b6eb00520ca8" (UID: "b595abfc-f4c3-43b7-94b9-b6eb00520ca8"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 29 17:00:04.005497 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.005462 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "b595abfc-f4c3-43b7-94b9-b6eb00520ca8" (UID: "b595abfc-f4c3-43b7-94b9-b6eb00520ca8"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 29 17:00:04.005497 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.005483 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-config-out" (OuterVolumeSpecName: "config-out") pod "b595abfc-f4c3-43b7-94b9-b6eb00520ca8" (UID: "b595abfc-f4c3-43b7-94b9-b6eb00520ca8"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 29 17:00:04.005773 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.005737 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "b595abfc-f4c3-43b7-94b9-b6eb00520ca8" (UID: "b595abfc-f4c3-43b7-94b9-b6eb00520ca8"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 29 17:00:04.006442 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.006414 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "b595abfc-f4c3-43b7-94b9-b6eb00520ca8" (UID: "b595abfc-f4c3-43b7-94b9-b6eb00520ca8"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 29 17:00:04.006557 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.006537 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "b595abfc-f4c3-43b7-94b9-b6eb00520ca8" (UID: "b595abfc-f4c3-43b7-94b9-b6eb00520ca8"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 29 17:00:04.006669 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.006651 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-kube-api-access-vhnpp" (OuterVolumeSpecName: "kube-api-access-vhnpp") pod "b595abfc-f4c3-43b7-94b9-b6eb00520ca8" (UID: "b595abfc-f4c3-43b7-94b9-b6eb00520ca8"). InnerVolumeSpecName "kube-api-access-vhnpp". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 29 17:00:04.015460 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.015436 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-web-config" (OuterVolumeSpecName: "web-config") pod "b595abfc-f4c3-43b7-94b9-b6eb00520ca8" (UID: "b595abfc-f4c3-43b7-94b9-b6eb00520ca8"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 29 17:00:04.101914 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.101891 2549 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-prometheus-trusted-ca-bundle\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 17:00:04.101914 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.101912 2549 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 17:00:04.102025 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.101922 2549 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-web-config\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 17:00:04.102025 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.101930 2549 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 17:00:04.102025 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.101939 2549 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-secret-kube-rbac-proxy\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 17:00:04.102025 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.101948 2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vhnpp\" (UniqueName: \"kubernetes.io/projected/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-kube-api-access-vhnpp\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 17:00:04.102025 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.101956 2549 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 17:00:04.102025 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.101966 2549 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-configmap-metrics-client-ca\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 17:00:04.102025 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.101975 2549 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-secret-prometheus-k8s-tls\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 17:00:04.102025 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.101984 2549 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-secret-grpc-tls\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 17:00:04.102025 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.101992 2549 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 17:00:04.102025 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.102001 2549 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 17:00:04.102025 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.102009 2549 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-prometheus-k8s-db\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 17:00:04.102025 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.102018 2549 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-thanos-prometheus-http-client-file\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 17:00:04.102025 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.102027 2549 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-config-out\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 17:00:04.102384 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.102036 2549 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-config\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 17:00:04.102384 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.102044 2549 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-secret-metrics-client-certs\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 17:00:04.102384 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.102053 2549 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b595abfc-f4c3-43b7-94b9-b6eb00520ca8-tls-assets\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 17:00:04.776469 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.776428 2549 generic.go:358] "Generic (PLEG): container finished" podID="b595abfc-f4c3-43b7-94b9-b6eb00520ca8" containerID="d4afe07b4abd51548797adc00a40b267f62379b7139fa8569aee6c5a98c4f1f4" exitCode=0 May 29 17:00:04.776469 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.776464 2549 generic.go:358] "Generic (PLEG): container finished" podID="b595abfc-f4c3-43b7-94b9-b6eb00520ca8" containerID="6d8b6580d1d20aba4363be3f701ad3d3db74f1186745d56e714f5678e4de440f" exitCode=0 May 29 17:00:04.776914 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.776464 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b595abfc-f4c3-43b7-94b9-b6eb00520ca8","Type":"ContainerDied","Data":"d4afe07b4abd51548797adc00a40b267f62379b7139fa8569aee6c5a98c4f1f4"} May 29 17:00:04.776914 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.776528 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b595abfc-f4c3-43b7-94b9-b6eb00520ca8","Type":"ContainerDied","Data":"6d8b6580d1d20aba4363be3f701ad3d3db74f1186745d56e714f5678e4de440f"} May 29 17:00:04.776914 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.776555 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b595abfc-f4c3-43b7-94b9-b6eb00520ca8","Type":"ContainerDied","Data":"d295027d80b4a8cbfc3cbc5a627a6e7f1b172228c8e8e5eef967d194590b7968"} May 29 17:00:04.776914 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.776479 2549 generic.go:358] "Generic (PLEG): container finished" podID="b595abfc-f4c3-43b7-94b9-b6eb00520ca8" containerID="d295027d80b4a8cbfc3cbc5a627a6e7f1b172228c8e8e5eef967d194590b7968" exitCode=0 May 29 17:00:04.776914 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.776574 2549 scope.go:117] "RemoveContainer" containerID="d4afe07b4abd51548797adc00a40b267f62379b7139fa8569aee6c5a98c4f1f4" May 29 17:00:04.776914 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.776604 2549 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:04.776914 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.776629 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b595abfc-f4c3-43b7-94b9-b6eb00520ca8","Type":"ContainerDied","Data":"f1a278f7d38c84c4614b922dc3772442d3e0ebe391f9f05fc05e66d03efecb66"} May 29 17:00:04.776914 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.776600 2549 generic.go:358] "Generic (PLEG): container finished" podID="b595abfc-f4c3-43b7-94b9-b6eb00520ca8" containerID="f1a278f7d38c84c4614b922dc3772442d3e0ebe391f9f05fc05e66d03efecb66" exitCode=0 May 29 17:00:04.776914 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.776722 2549 generic.go:358] "Generic (PLEG): container finished" podID="b595abfc-f4c3-43b7-94b9-b6eb00520ca8" containerID="a84f0bdc2a0fe70215697f108054fe50419efb6b8fc2295e28b6d9b4d9fe78b0" exitCode=0 May 29 17:00:04.776914 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.776737 2549 generic.go:358] "Generic (PLEG): container finished" podID="b595abfc-f4c3-43b7-94b9-b6eb00520ca8" containerID="0198c250e9fdde86bd791e4add6349098d34bd5257b132452d885c4f970a12db" exitCode=0 May 29 17:00:04.776914 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.776805 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b595abfc-f4c3-43b7-94b9-b6eb00520ca8","Type":"ContainerDied","Data":"a84f0bdc2a0fe70215697f108054fe50419efb6b8fc2295e28b6d9b4d9fe78b0"} May 29 17:00:04.776914 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.776833 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b595abfc-f4c3-43b7-94b9-b6eb00520ca8","Type":"ContainerDied","Data":"0198c250e9fdde86bd791e4add6349098d34bd5257b132452d885c4f970a12db"} May 29 17:00:04.776914 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.776872 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b595abfc-f4c3-43b7-94b9-b6eb00520ca8","Type":"ContainerDied","Data":"193143247b16268f8af2ccf51b9a9f1267796c5a24ac63fa2399d3b979854b88"} May 29 17:00:04.783930 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.783906 2549 scope.go:117] "RemoveContainer" containerID="6d8b6580d1d20aba4363be3f701ad3d3db74f1186745d56e714f5678e4de440f" May 29 17:00:04.790106 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.790092 2549 scope.go:117] "RemoveContainer" containerID="d295027d80b4a8cbfc3cbc5a627a6e7f1b172228c8e8e5eef967d194590b7968" May 29 17:00:04.794586 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.794548 2549 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] May 29 17:00:04.796411 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.796395 2549 scope.go:117] "RemoveContainer" containerID="f1a278f7d38c84c4614b922dc3772442d3e0ebe391f9f05fc05e66d03efecb66" May 29 17:00:04.798670 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.798649 2549 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] May 29 17:00:04.803190 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.803160 2549 scope.go:117] "RemoveContainer" containerID="a84f0bdc2a0fe70215697f108054fe50419efb6b8fc2295e28b6d9b4d9fe78b0" May 29 17:00:04.809076 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.809058 2549 scope.go:117] "RemoveContainer" containerID="0198c250e9fdde86bd791e4add6349098d34bd5257b132452d885c4f970a12db" May 29 17:00:04.815515 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.815500 2549 scope.go:117] "RemoveContainer" containerID="ddc7d497a54ac3be13465e507ce387e46bef9ea748aebb3eded5bbe472c5ddb8" May 29 17:00:04.821584 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.821568 2549 scope.go:117] "RemoveContainer" containerID="d4afe07b4abd51548797adc00a40b267f62379b7139fa8569aee6c5a98c4f1f4" May 29 17:00:04.821825 ip-10-0-141-197 kubenswrapper[2549]: E0529 17:00:04.821802 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4afe07b4abd51548797adc00a40b267f62379b7139fa8569aee6c5a98c4f1f4\": container with ID starting with d4afe07b4abd51548797adc00a40b267f62379b7139fa8569aee6c5a98c4f1f4 not found: ID does not exist" containerID="d4afe07b4abd51548797adc00a40b267f62379b7139fa8569aee6c5a98c4f1f4" May 29 17:00:04.821898 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.821836 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4afe07b4abd51548797adc00a40b267f62379b7139fa8569aee6c5a98c4f1f4"} err="failed to get container status \"d4afe07b4abd51548797adc00a40b267f62379b7139fa8569aee6c5a98c4f1f4\": rpc error: code = NotFound desc = could not find container \"d4afe07b4abd51548797adc00a40b267f62379b7139fa8569aee6c5a98c4f1f4\": container with ID starting with d4afe07b4abd51548797adc00a40b267f62379b7139fa8569aee6c5a98c4f1f4 not found: ID does not exist" May 29 17:00:04.821898 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.821881 2549 scope.go:117] "RemoveContainer" containerID="6d8b6580d1d20aba4363be3f701ad3d3db74f1186745d56e714f5678e4de440f" May 29 17:00:04.821998 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.821971 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] May 29 17:00:04.822152 ip-10-0-141-197 kubenswrapper[2549]: E0529 17:00:04.822125 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d8b6580d1d20aba4363be3f701ad3d3db74f1186745d56e714f5678e4de440f\": container with ID starting with 6d8b6580d1d20aba4363be3f701ad3d3db74f1186745d56e714f5678e4de440f not found: ID does not exist" containerID="6d8b6580d1d20aba4363be3f701ad3d3db74f1186745d56e714f5678e4de440f" May 29 17:00:04.822220 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.822159 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d8b6580d1d20aba4363be3f701ad3d3db74f1186745d56e714f5678e4de440f"} err="failed to get container status \"6d8b6580d1d20aba4363be3f701ad3d3db74f1186745d56e714f5678e4de440f\": rpc error: code = NotFound desc = could not find container \"6d8b6580d1d20aba4363be3f701ad3d3db74f1186745d56e714f5678e4de440f\": container with ID starting with 6d8b6580d1d20aba4363be3f701ad3d3db74f1186745d56e714f5678e4de440f not found: ID does not exist" May 29 17:00:04.822220 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.822181 2549 scope.go:117] "RemoveContainer" containerID="d295027d80b4a8cbfc3cbc5a627a6e7f1b172228c8e8e5eef967d194590b7968" May 29 17:00:04.822313 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.822280 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b595abfc-f4c3-43b7-94b9-b6eb00520ca8" containerName="init-config-reloader" May 29 17:00:04.822313 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.822294 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="b595abfc-f4c3-43b7-94b9-b6eb00520ca8" containerName="init-config-reloader" May 29 17:00:04.822313 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.822306 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b595abfc-f4c3-43b7-94b9-b6eb00520ca8" containerName="config-reloader" May 29 17:00:04.822446 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.822315 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="b595abfc-f4c3-43b7-94b9-b6eb00520ca8" containerName="config-reloader" May 29 17:00:04.822446 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.822324 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b595abfc-f4c3-43b7-94b9-b6eb00520ca8" containerName="kube-rbac-proxy-web" May 29 17:00:04.822446 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.822330 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="b595abfc-f4c3-43b7-94b9-b6eb00520ca8" containerName="kube-rbac-proxy-web" May 29 17:00:04.822446 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.822344 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b595abfc-f4c3-43b7-94b9-b6eb00520ca8" containerName="kube-rbac-proxy-thanos" May 29 17:00:04.822446 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.822352 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="b595abfc-f4c3-43b7-94b9-b6eb00520ca8" containerName="kube-rbac-proxy-thanos" May 29 17:00:04.822446 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.822361 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b595abfc-f4c3-43b7-94b9-b6eb00520ca8" containerName="kube-rbac-proxy" May 29 17:00:04.822446 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.822366 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="b595abfc-f4c3-43b7-94b9-b6eb00520ca8" containerName="kube-rbac-proxy" May 29 17:00:04.822446 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.822373 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54db9067-d609-4bea-a927-c91c52543add" containerName="console" May 29 17:00:04.822446 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.822378 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="54db9067-d609-4bea-a927-c91c52543add" containerName="console" May 29 17:00:04.822446 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.822385 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b595abfc-f4c3-43b7-94b9-b6eb00520ca8" containerName="prometheus" May 29 17:00:04.822446 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.822390 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="b595abfc-f4c3-43b7-94b9-b6eb00520ca8" containerName="prometheus" May 29 17:00:04.822446 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.822402 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d54083d9-f32e-43ff-9202-598c7d31a427" containerName="console" May 29 17:00:04.822446 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.822407 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="d54083d9-f32e-43ff-9202-598c7d31a427" containerName="console" May 29 17:00:04.822446 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.822416 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b595abfc-f4c3-43b7-94b9-b6eb00520ca8" containerName="thanos-sidecar" May 29 17:00:04.822446 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.822421 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="b595abfc-f4c3-43b7-94b9-b6eb00520ca8" containerName="thanos-sidecar" May 29 17:00:04.823057 ip-10-0-141-197 kubenswrapper[2549]: E0529 17:00:04.822444 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d295027d80b4a8cbfc3cbc5a627a6e7f1b172228c8e8e5eef967d194590b7968\": container with ID starting with d295027d80b4a8cbfc3cbc5a627a6e7f1b172228c8e8e5eef967d194590b7968 not found: ID does not exist" containerID="d295027d80b4a8cbfc3cbc5a627a6e7f1b172228c8e8e5eef967d194590b7968" May 29 17:00:04.823057 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.822461 2549 memory_manager.go:356] "RemoveStaleState removing state" podUID="b595abfc-f4c3-43b7-94b9-b6eb00520ca8" containerName="config-reloader" May 29 17:00:04.823057 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.822468 2549 memory_manager.go:356] "RemoveStaleState removing state" podUID="b595abfc-f4c3-43b7-94b9-b6eb00520ca8" containerName="prometheus" May 29 17:00:04.823057 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.822474 2549 memory_manager.go:356] "RemoveStaleState removing state" podUID="b595abfc-f4c3-43b7-94b9-b6eb00520ca8" containerName="kube-rbac-proxy-thanos" May 29 17:00:04.823057 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.822481 2549 memory_manager.go:356] "RemoveStaleState removing state" podUID="b595abfc-f4c3-43b7-94b9-b6eb00520ca8" containerName="kube-rbac-proxy-web" May 29 17:00:04.823057 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.822487 2549 memory_manager.go:356] "RemoveStaleState removing state" podUID="b595abfc-f4c3-43b7-94b9-b6eb00520ca8" containerName="kube-rbac-proxy" May 29 17:00:04.823057 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.822494 2549 memory_manager.go:356] "RemoveStaleState removing state" podUID="d54083d9-f32e-43ff-9202-598c7d31a427" containerName="console" May 29 17:00:04.823057 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.822499 2549 memory_manager.go:356] "RemoveStaleState removing state" podUID="54db9067-d609-4bea-a927-c91c52543add" containerName="console" May 29 17:00:04.823057 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.822506 2549 memory_manager.go:356] "RemoveStaleState removing state" podUID="b595abfc-f4c3-43b7-94b9-b6eb00520ca8" containerName="thanos-sidecar" May 29 17:00:04.823057 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.822471 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d295027d80b4a8cbfc3cbc5a627a6e7f1b172228c8e8e5eef967d194590b7968"} err="failed to get container status \"d295027d80b4a8cbfc3cbc5a627a6e7f1b172228c8e8e5eef967d194590b7968\": rpc error: code = NotFound desc = could not find container \"d295027d80b4a8cbfc3cbc5a627a6e7f1b172228c8e8e5eef967d194590b7968\": container with ID starting with d295027d80b4a8cbfc3cbc5a627a6e7f1b172228c8e8e5eef967d194590b7968 not found: ID does not exist" May 29 17:00:04.823057 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.822559 2549 scope.go:117] "RemoveContainer" containerID="f1a278f7d38c84c4614b922dc3772442d3e0ebe391f9f05fc05e66d03efecb66" May 29 17:00:04.823057 ip-10-0-141-197 kubenswrapper[2549]: E0529 17:00:04.822737 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1a278f7d38c84c4614b922dc3772442d3e0ebe391f9f05fc05e66d03efecb66\": container with ID starting with f1a278f7d38c84c4614b922dc3772442d3e0ebe391f9f05fc05e66d03efecb66 not found: ID does not exist" containerID="f1a278f7d38c84c4614b922dc3772442d3e0ebe391f9f05fc05e66d03efecb66" May 29 17:00:04.823057 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.822762 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1a278f7d38c84c4614b922dc3772442d3e0ebe391f9f05fc05e66d03efecb66"} err="failed to get container status \"f1a278f7d38c84c4614b922dc3772442d3e0ebe391f9f05fc05e66d03efecb66\": rpc error: code = NotFound desc = could not find container \"f1a278f7d38c84c4614b922dc3772442d3e0ebe391f9f05fc05e66d03efecb66\": container with ID starting with f1a278f7d38c84c4614b922dc3772442d3e0ebe391f9f05fc05e66d03efecb66 not found: ID does not exist" May 29 17:00:04.823057 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.822784 2549 scope.go:117] "RemoveContainer" containerID="a84f0bdc2a0fe70215697f108054fe50419efb6b8fc2295e28b6d9b4d9fe78b0" May 29 17:00:04.823513 ip-10-0-141-197 kubenswrapper[2549]: E0529 17:00:04.823068 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a84f0bdc2a0fe70215697f108054fe50419efb6b8fc2295e28b6d9b4d9fe78b0\": container with ID starting with a84f0bdc2a0fe70215697f108054fe50419efb6b8fc2295e28b6d9b4d9fe78b0 not found: ID does not exist" containerID="a84f0bdc2a0fe70215697f108054fe50419efb6b8fc2295e28b6d9b4d9fe78b0" May 29 17:00:04.823513 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.823101 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a84f0bdc2a0fe70215697f108054fe50419efb6b8fc2295e28b6d9b4d9fe78b0"} err="failed to get container status \"a84f0bdc2a0fe70215697f108054fe50419efb6b8fc2295e28b6d9b4d9fe78b0\": rpc error: code = NotFound desc = could not find container \"a84f0bdc2a0fe70215697f108054fe50419efb6b8fc2295e28b6d9b4d9fe78b0\": container with ID starting with a84f0bdc2a0fe70215697f108054fe50419efb6b8fc2295e28b6d9b4d9fe78b0 not found: ID does not exist" May 29 17:00:04.823513 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.823122 2549 scope.go:117] "RemoveContainer" containerID="0198c250e9fdde86bd791e4add6349098d34bd5257b132452d885c4f970a12db" May 29 17:00:04.823513 ip-10-0-141-197 kubenswrapper[2549]: E0529 17:00:04.823360 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0198c250e9fdde86bd791e4add6349098d34bd5257b132452d885c4f970a12db\": container with ID starting with 0198c250e9fdde86bd791e4add6349098d34bd5257b132452d885c4f970a12db not found: ID does not exist" containerID="0198c250e9fdde86bd791e4add6349098d34bd5257b132452d885c4f970a12db" May 29 17:00:04.823513 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.823380 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0198c250e9fdde86bd791e4add6349098d34bd5257b132452d885c4f970a12db"} err="failed to get container status \"0198c250e9fdde86bd791e4add6349098d34bd5257b132452d885c4f970a12db\": rpc error: code = NotFound desc = could not find container \"0198c250e9fdde86bd791e4add6349098d34bd5257b132452d885c4f970a12db\": container with ID starting with 0198c250e9fdde86bd791e4add6349098d34bd5257b132452d885c4f970a12db not found: ID does not exist" May 29 17:00:04.823513 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.823395 2549 scope.go:117] "RemoveContainer" containerID="ddc7d497a54ac3be13465e507ce387e46bef9ea748aebb3eded5bbe472c5ddb8" May 29 17:00:04.823717 ip-10-0-141-197 kubenswrapper[2549]: E0529 17:00:04.823622 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ddc7d497a54ac3be13465e507ce387e46bef9ea748aebb3eded5bbe472c5ddb8\": container with ID starting with ddc7d497a54ac3be13465e507ce387e46bef9ea748aebb3eded5bbe472c5ddb8 not found: ID does not exist" containerID="ddc7d497a54ac3be13465e507ce387e46bef9ea748aebb3eded5bbe472c5ddb8" May 29 17:00:04.823717 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.823642 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddc7d497a54ac3be13465e507ce387e46bef9ea748aebb3eded5bbe472c5ddb8"} err="failed to get container status \"ddc7d497a54ac3be13465e507ce387e46bef9ea748aebb3eded5bbe472c5ddb8\": rpc error: code = NotFound desc = could not find container \"ddc7d497a54ac3be13465e507ce387e46bef9ea748aebb3eded5bbe472c5ddb8\": container with ID starting with ddc7d497a54ac3be13465e507ce387e46bef9ea748aebb3eded5bbe472c5ddb8 not found: ID does not exist" May 29 17:00:04.823717 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.823655 2549 scope.go:117] "RemoveContainer" containerID="d4afe07b4abd51548797adc00a40b267f62379b7139fa8569aee6c5a98c4f1f4" May 29 17:00:04.823878 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.823862 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4afe07b4abd51548797adc00a40b267f62379b7139fa8569aee6c5a98c4f1f4"} err="failed to get container status \"d4afe07b4abd51548797adc00a40b267f62379b7139fa8569aee6c5a98c4f1f4\": rpc error: code = NotFound desc = could not find container \"d4afe07b4abd51548797adc00a40b267f62379b7139fa8569aee6c5a98c4f1f4\": container with ID starting with d4afe07b4abd51548797adc00a40b267f62379b7139fa8569aee6c5a98c4f1f4 not found: ID does not exist" May 29 17:00:04.823928 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.823878 2549 scope.go:117] "RemoveContainer" containerID="6d8b6580d1d20aba4363be3f701ad3d3db74f1186745d56e714f5678e4de440f" May 29 17:00:04.824087 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.824061 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d8b6580d1d20aba4363be3f701ad3d3db74f1186745d56e714f5678e4de440f"} err="failed to get container status \"6d8b6580d1d20aba4363be3f701ad3d3db74f1186745d56e714f5678e4de440f\": rpc error: code = NotFound desc = could not find container \"6d8b6580d1d20aba4363be3f701ad3d3db74f1186745d56e714f5678e4de440f\": container with ID starting with 6d8b6580d1d20aba4363be3f701ad3d3db74f1186745d56e714f5678e4de440f not found: ID does not exist" May 29 17:00:04.824165 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.824089 2549 scope.go:117] "RemoveContainer" containerID="d295027d80b4a8cbfc3cbc5a627a6e7f1b172228c8e8e5eef967d194590b7968" May 29 17:00:04.824330 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.824313 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d295027d80b4a8cbfc3cbc5a627a6e7f1b172228c8e8e5eef967d194590b7968"} err="failed to get container status \"d295027d80b4a8cbfc3cbc5a627a6e7f1b172228c8e8e5eef967d194590b7968\": rpc error: code = NotFound desc = could not find container \"d295027d80b4a8cbfc3cbc5a627a6e7f1b172228c8e8e5eef967d194590b7968\": container with ID starting with d295027d80b4a8cbfc3cbc5a627a6e7f1b172228c8e8e5eef967d194590b7968 not found: ID does not exist" May 29 17:00:04.824330 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.824330 2549 scope.go:117] "RemoveContainer" containerID="f1a278f7d38c84c4614b922dc3772442d3e0ebe391f9f05fc05e66d03efecb66" May 29 17:00:04.824551 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.824536 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1a278f7d38c84c4614b922dc3772442d3e0ebe391f9f05fc05e66d03efecb66"} err="failed to get container status \"f1a278f7d38c84c4614b922dc3772442d3e0ebe391f9f05fc05e66d03efecb66\": rpc error: code = NotFound desc = could not find container \"f1a278f7d38c84c4614b922dc3772442d3e0ebe391f9f05fc05e66d03efecb66\": container with ID starting with f1a278f7d38c84c4614b922dc3772442d3e0ebe391f9f05fc05e66d03efecb66 not found: ID does not exist" May 29 17:00:04.824590 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.824551 2549 scope.go:117] "RemoveContainer" containerID="a84f0bdc2a0fe70215697f108054fe50419efb6b8fc2295e28b6d9b4d9fe78b0" May 29 17:00:04.824779 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.824759 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a84f0bdc2a0fe70215697f108054fe50419efb6b8fc2295e28b6d9b4d9fe78b0"} err="failed to get container status \"a84f0bdc2a0fe70215697f108054fe50419efb6b8fc2295e28b6d9b4d9fe78b0\": rpc error: code = NotFound desc = could not find container \"a84f0bdc2a0fe70215697f108054fe50419efb6b8fc2295e28b6d9b4d9fe78b0\": container with ID starting with a84f0bdc2a0fe70215697f108054fe50419efb6b8fc2295e28b6d9b4d9fe78b0 not found: ID does not exist" May 29 17:00:04.824871 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.824781 2549 scope.go:117] "RemoveContainer" containerID="0198c250e9fdde86bd791e4add6349098d34bd5257b132452d885c4f970a12db" May 29 17:00:04.825048 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.825030 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0198c250e9fdde86bd791e4add6349098d34bd5257b132452d885c4f970a12db"} err="failed to get container status \"0198c250e9fdde86bd791e4add6349098d34bd5257b132452d885c4f970a12db\": rpc error: code = NotFound desc = could not find container \"0198c250e9fdde86bd791e4add6349098d34bd5257b132452d885c4f970a12db\": container with ID starting with 0198c250e9fdde86bd791e4add6349098d34bd5257b132452d885c4f970a12db not found: ID does not exist" May 29 17:00:04.825093 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.825050 2549 scope.go:117] "RemoveContainer" containerID="ddc7d497a54ac3be13465e507ce387e46bef9ea748aebb3eded5bbe472c5ddb8" May 29 17:00:04.825269 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.825254 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddc7d497a54ac3be13465e507ce387e46bef9ea748aebb3eded5bbe472c5ddb8"} err="failed to get container status \"ddc7d497a54ac3be13465e507ce387e46bef9ea748aebb3eded5bbe472c5ddb8\": rpc error: code = NotFound desc = could not find container \"ddc7d497a54ac3be13465e507ce387e46bef9ea748aebb3eded5bbe472c5ddb8\": container with ID starting with ddc7d497a54ac3be13465e507ce387e46bef9ea748aebb3eded5bbe472c5ddb8 not found: ID does not exist" May 29 17:00:04.825269 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.825269 2549 scope.go:117] "RemoveContainer" containerID="d4afe07b4abd51548797adc00a40b267f62379b7139fa8569aee6c5a98c4f1f4" May 29 17:00:04.825429 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.825411 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4afe07b4abd51548797adc00a40b267f62379b7139fa8569aee6c5a98c4f1f4"} err="failed to get container status \"d4afe07b4abd51548797adc00a40b267f62379b7139fa8569aee6c5a98c4f1f4\": rpc error: code = NotFound desc = could not find container \"d4afe07b4abd51548797adc00a40b267f62379b7139fa8569aee6c5a98c4f1f4\": container with ID starting with d4afe07b4abd51548797adc00a40b267f62379b7139fa8569aee6c5a98c4f1f4 not found: ID does not exist" May 29 17:00:04.825490 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.825431 2549 scope.go:117] "RemoveContainer" containerID="6d8b6580d1d20aba4363be3f701ad3d3db74f1186745d56e714f5678e4de440f" May 29 17:00:04.825625 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.825610 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d8b6580d1d20aba4363be3f701ad3d3db74f1186745d56e714f5678e4de440f"} err="failed to get container status \"6d8b6580d1d20aba4363be3f701ad3d3db74f1186745d56e714f5678e4de440f\": rpc error: code = NotFound desc = could not find container \"6d8b6580d1d20aba4363be3f701ad3d3db74f1186745d56e714f5678e4de440f\": container with ID starting with 6d8b6580d1d20aba4363be3f701ad3d3db74f1186745d56e714f5678e4de440f not found: ID does not exist" May 29 17:00:04.825671 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.825626 2549 scope.go:117] "RemoveContainer" containerID="d295027d80b4a8cbfc3cbc5a627a6e7f1b172228c8e8e5eef967d194590b7968" May 29 17:00:04.825862 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.825826 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d295027d80b4a8cbfc3cbc5a627a6e7f1b172228c8e8e5eef967d194590b7968"} err="failed to get container status \"d295027d80b4a8cbfc3cbc5a627a6e7f1b172228c8e8e5eef967d194590b7968\": rpc error: code = NotFound desc = could not find container \"d295027d80b4a8cbfc3cbc5a627a6e7f1b172228c8e8e5eef967d194590b7968\": container with ID starting with d295027d80b4a8cbfc3cbc5a627a6e7f1b172228c8e8e5eef967d194590b7968 not found: ID does not exist" May 29 17:00:04.825862 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.825857 2549 scope.go:117] "RemoveContainer" containerID="f1a278f7d38c84c4614b922dc3772442d3e0ebe391f9f05fc05e66d03efecb66" May 29 17:00:04.826044 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.826025 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1a278f7d38c84c4614b922dc3772442d3e0ebe391f9f05fc05e66d03efecb66"} err="failed to get container status \"f1a278f7d38c84c4614b922dc3772442d3e0ebe391f9f05fc05e66d03efecb66\": rpc error: code = NotFound desc = could not find container \"f1a278f7d38c84c4614b922dc3772442d3e0ebe391f9f05fc05e66d03efecb66\": container with ID starting with f1a278f7d38c84c4614b922dc3772442d3e0ebe391f9f05fc05e66d03efecb66 not found: ID does not exist" May 29 17:00:04.826107 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.826046 2549 scope.go:117] "RemoveContainer" containerID="a84f0bdc2a0fe70215697f108054fe50419efb6b8fc2295e28b6d9b4d9fe78b0" May 29 17:00:04.826263 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.826246 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a84f0bdc2a0fe70215697f108054fe50419efb6b8fc2295e28b6d9b4d9fe78b0"} err="failed to get container status \"a84f0bdc2a0fe70215697f108054fe50419efb6b8fc2295e28b6d9b4d9fe78b0\": rpc error: code = NotFound desc = could not find container \"a84f0bdc2a0fe70215697f108054fe50419efb6b8fc2295e28b6d9b4d9fe78b0\": container with ID starting with a84f0bdc2a0fe70215697f108054fe50419efb6b8fc2295e28b6d9b4d9fe78b0 not found: ID does not exist" May 29 17:00:04.826310 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.826265 2549 scope.go:117] "RemoveContainer" containerID="0198c250e9fdde86bd791e4add6349098d34bd5257b132452d885c4f970a12db" May 29 17:00:04.826460 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.826443 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0198c250e9fdde86bd791e4add6349098d34bd5257b132452d885c4f970a12db"} err="failed to get container status \"0198c250e9fdde86bd791e4add6349098d34bd5257b132452d885c4f970a12db\": rpc error: code = NotFound desc = could not find container \"0198c250e9fdde86bd791e4add6349098d34bd5257b132452d885c4f970a12db\": container with ID starting with 0198c250e9fdde86bd791e4add6349098d34bd5257b132452d885c4f970a12db not found: ID does not exist" May 29 17:00:04.826525 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.826462 2549 scope.go:117] "RemoveContainer" containerID="ddc7d497a54ac3be13465e507ce387e46bef9ea748aebb3eded5bbe472c5ddb8" May 29 17:00:04.826656 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.826636 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddc7d497a54ac3be13465e507ce387e46bef9ea748aebb3eded5bbe472c5ddb8"} err="failed to get container status \"ddc7d497a54ac3be13465e507ce387e46bef9ea748aebb3eded5bbe472c5ddb8\": rpc error: code = NotFound desc = could not find container \"ddc7d497a54ac3be13465e507ce387e46bef9ea748aebb3eded5bbe472c5ddb8\": container with ID starting with ddc7d497a54ac3be13465e507ce387e46bef9ea748aebb3eded5bbe472c5ddb8 not found: ID does not exist" May 29 17:00:04.826701 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.826657 2549 scope.go:117] "RemoveContainer" containerID="d4afe07b4abd51548797adc00a40b267f62379b7139fa8569aee6c5a98c4f1f4" May 29 17:00:04.826871 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.826834 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4afe07b4abd51548797adc00a40b267f62379b7139fa8569aee6c5a98c4f1f4"} err="failed to get container status \"d4afe07b4abd51548797adc00a40b267f62379b7139fa8569aee6c5a98c4f1f4\": rpc error: code = NotFound desc = could not find container \"d4afe07b4abd51548797adc00a40b267f62379b7139fa8569aee6c5a98c4f1f4\": container with ID starting with d4afe07b4abd51548797adc00a40b267f62379b7139fa8569aee6c5a98c4f1f4 not found: ID does not exist" May 29 17:00:04.826942 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.826874 2549 scope.go:117] "RemoveContainer" containerID="6d8b6580d1d20aba4363be3f701ad3d3db74f1186745d56e714f5678e4de440f" May 29 17:00:04.827096 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.827080 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d8b6580d1d20aba4363be3f701ad3d3db74f1186745d56e714f5678e4de440f"} err="failed to get container status \"6d8b6580d1d20aba4363be3f701ad3d3db74f1186745d56e714f5678e4de440f\": rpc error: code = NotFound desc = could not find container \"6d8b6580d1d20aba4363be3f701ad3d3db74f1186745d56e714f5678e4de440f\": container with ID starting with 6d8b6580d1d20aba4363be3f701ad3d3db74f1186745d56e714f5678e4de440f not found: ID does not exist" May 29 17:00:04.827143 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.827098 2549 scope.go:117] "RemoveContainer" containerID="d295027d80b4a8cbfc3cbc5a627a6e7f1b172228c8e8e5eef967d194590b7968" May 29 17:00:04.827286 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.827265 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d295027d80b4a8cbfc3cbc5a627a6e7f1b172228c8e8e5eef967d194590b7968"} err="failed to get container status \"d295027d80b4a8cbfc3cbc5a627a6e7f1b172228c8e8e5eef967d194590b7968\": rpc error: code = NotFound desc = could not find container \"d295027d80b4a8cbfc3cbc5a627a6e7f1b172228c8e8e5eef967d194590b7968\": container with ID starting with d295027d80b4a8cbfc3cbc5a627a6e7f1b172228c8e8e5eef967d194590b7968 not found: ID does not exist" May 29 17:00:04.827353 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.827294 2549 scope.go:117] "RemoveContainer" containerID="f1a278f7d38c84c4614b922dc3772442d3e0ebe391f9f05fc05e66d03efecb66" May 29 17:00:04.827481 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.827465 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1a278f7d38c84c4614b922dc3772442d3e0ebe391f9f05fc05e66d03efecb66"} err="failed to get container status \"f1a278f7d38c84c4614b922dc3772442d3e0ebe391f9f05fc05e66d03efecb66\": rpc error: code = NotFound desc = could not find container \"f1a278f7d38c84c4614b922dc3772442d3e0ebe391f9f05fc05e66d03efecb66\": container with ID starting with f1a278f7d38c84c4614b922dc3772442d3e0ebe391f9f05fc05e66d03efecb66 not found: ID does not exist" May 29 17:00:04.827481 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.827479 2549 scope.go:117] "RemoveContainer" containerID="a84f0bdc2a0fe70215697f108054fe50419efb6b8fc2295e28b6d9b4d9fe78b0" May 29 17:00:04.827628 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.827614 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:04.827686 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.827635 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a84f0bdc2a0fe70215697f108054fe50419efb6b8fc2295e28b6d9b4d9fe78b0"} err="failed to get container status \"a84f0bdc2a0fe70215697f108054fe50419efb6b8fc2295e28b6d9b4d9fe78b0\": rpc error: code = NotFound desc = could not find container \"a84f0bdc2a0fe70215697f108054fe50419efb6b8fc2295e28b6d9b4d9fe78b0\": container with ID starting with a84f0bdc2a0fe70215697f108054fe50419efb6b8fc2295e28b6d9b4d9fe78b0 not found: ID does not exist" May 29 17:00:04.827686 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.827650 2549 scope.go:117] "RemoveContainer" containerID="0198c250e9fdde86bd791e4add6349098d34bd5257b132452d885c4f970a12db" May 29 17:00:04.828003 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.827834 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0198c250e9fdde86bd791e4add6349098d34bd5257b132452d885c4f970a12db"} err="failed to get container status \"0198c250e9fdde86bd791e4add6349098d34bd5257b132452d885c4f970a12db\": rpc error: code = NotFound desc = could not find container \"0198c250e9fdde86bd791e4add6349098d34bd5257b132452d885c4f970a12db\": container with ID starting with 0198c250e9fdde86bd791e4add6349098d34bd5257b132452d885c4f970a12db not found: ID does not exist" May 29 17:00:04.828003 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.827879 2549 scope.go:117] "RemoveContainer" containerID="ddc7d497a54ac3be13465e507ce387e46bef9ea748aebb3eded5bbe472c5ddb8" May 29 17:00:04.828254 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.828231 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddc7d497a54ac3be13465e507ce387e46bef9ea748aebb3eded5bbe472c5ddb8"} err="failed to get container status \"ddc7d497a54ac3be13465e507ce387e46bef9ea748aebb3eded5bbe472c5ddb8\": rpc error: code = NotFound desc = could not find container \"ddc7d497a54ac3be13465e507ce387e46bef9ea748aebb3eded5bbe472c5ddb8\": container with ID starting with ddc7d497a54ac3be13465e507ce387e46bef9ea748aebb3eded5bbe472c5ddb8 not found: ID does not exist" May 29 17:00:04.828301 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.828256 2549 scope.go:117] "RemoveContainer" containerID="d4afe07b4abd51548797adc00a40b267f62379b7139fa8569aee6c5a98c4f1f4" May 29 17:00:04.828505 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.828483 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4afe07b4abd51548797adc00a40b267f62379b7139fa8569aee6c5a98c4f1f4"} err="failed to get container status \"d4afe07b4abd51548797adc00a40b267f62379b7139fa8569aee6c5a98c4f1f4\": rpc error: code = NotFound desc = could not find container \"d4afe07b4abd51548797adc00a40b267f62379b7139fa8569aee6c5a98c4f1f4\": container with ID starting with d4afe07b4abd51548797adc00a40b267f62379b7139fa8569aee6c5a98c4f1f4 not found: ID does not exist" May 29 17:00:04.828562 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.828507 2549 scope.go:117] "RemoveContainer" containerID="6d8b6580d1d20aba4363be3f701ad3d3db74f1186745d56e714f5678e4de440f" May 29 17:00:04.828743 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.828724 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d8b6580d1d20aba4363be3f701ad3d3db74f1186745d56e714f5678e4de440f"} err="failed to get container status \"6d8b6580d1d20aba4363be3f701ad3d3db74f1186745d56e714f5678e4de440f\": rpc error: code = NotFound desc = could not find container \"6d8b6580d1d20aba4363be3f701ad3d3db74f1186745d56e714f5678e4de440f\": container with ID starting with 6d8b6580d1d20aba4363be3f701ad3d3db74f1186745d56e714f5678e4de440f not found: ID does not exist" May 29 17:00:04.828807 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.828744 2549 scope.go:117] "RemoveContainer" containerID="d295027d80b4a8cbfc3cbc5a627a6e7f1b172228c8e8e5eef967d194590b7968" May 29 17:00:04.828980 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.828962 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d295027d80b4a8cbfc3cbc5a627a6e7f1b172228c8e8e5eef967d194590b7968"} err="failed to get container status \"d295027d80b4a8cbfc3cbc5a627a6e7f1b172228c8e8e5eef967d194590b7968\": rpc error: code = NotFound desc = could not find container \"d295027d80b4a8cbfc3cbc5a627a6e7f1b172228c8e8e5eef967d194590b7968\": container with ID starting with d295027d80b4a8cbfc3cbc5a627a6e7f1b172228c8e8e5eef967d194590b7968 not found: ID does not exist" May 29 17:00:04.829033 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.828980 2549 scope.go:117] "RemoveContainer" containerID="f1a278f7d38c84c4614b922dc3772442d3e0ebe391f9f05fc05e66d03efecb66" May 29 17:00:04.829217 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.829192 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1a278f7d38c84c4614b922dc3772442d3e0ebe391f9f05fc05e66d03efecb66"} err="failed to get container status \"f1a278f7d38c84c4614b922dc3772442d3e0ebe391f9f05fc05e66d03efecb66\": rpc error: code = NotFound desc = could not find container \"f1a278f7d38c84c4614b922dc3772442d3e0ebe391f9f05fc05e66d03efecb66\": container with ID starting with f1a278f7d38c84c4614b922dc3772442d3e0ebe391f9f05fc05e66d03efecb66 not found: ID does not exist" May 29 17:00:04.829301 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.829218 2549 scope.go:117] "RemoveContainer" containerID="a84f0bdc2a0fe70215697f108054fe50419efb6b8fc2295e28b6d9b4d9fe78b0" May 29 17:00:04.829441 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.829422 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a84f0bdc2a0fe70215697f108054fe50419efb6b8fc2295e28b6d9b4d9fe78b0"} err="failed to get container status \"a84f0bdc2a0fe70215697f108054fe50419efb6b8fc2295e28b6d9b4d9fe78b0\": rpc error: code = NotFound desc = could not find container \"a84f0bdc2a0fe70215697f108054fe50419efb6b8fc2295e28b6d9b4d9fe78b0\": container with ID starting with a84f0bdc2a0fe70215697f108054fe50419efb6b8fc2295e28b6d9b4d9fe78b0 not found: ID does not exist" May 29 17:00:04.829515 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.829444 2549 scope.go:117] "RemoveContainer" containerID="0198c250e9fdde86bd791e4add6349098d34bd5257b132452d885c4f970a12db" May 29 17:00:04.829561 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.829517 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" May 29 17:00:04.829734 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.829713 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0198c250e9fdde86bd791e4add6349098d34bd5257b132452d885c4f970a12db"} err="failed to get container status \"0198c250e9fdde86bd791e4add6349098d34bd5257b132452d885c4f970a12db\": rpc error: code = NotFound desc = could not find container \"0198c250e9fdde86bd791e4add6349098d34bd5257b132452d885c4f970a12db\": container with ID starting with 0198c250e9fdde86bd791e4add6349098d34bd5257b132452d885c4f970a12db not found: ID does not exist" May 29 17:00:04.829788 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.829735 2549 scope.go:117] "RemoveContainer" containerID="ddc7d497a54ac3be13465e507ce387e46bef9ea748aebb3eded5bbe472c5ddb8" May 29 17:00:04.829882 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.829786 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" May 29 17:00:04.829882 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.829795 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" May 29 17:00:04.829996 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.829938 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" May 29 17:00:04.830045 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.829994 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddc7d497a54ac3be13465e507ce387e46bef9ea748aebb3eded5bbe472c5ddb8"} err="failed to get container status \"ddc7d497a54ac3be13465e507ce387e46bef9ea748aebb3eded5bbe472c5ddb8\": rpc error: code = NotFound desc = could not find container \"ddc7d497a54ac3be13465e507ce387e46bef9ea748aebb3eded5bbe472c5ddb8\": container with ID starting with ddc7d497a54ac3be13465e507ce387e46bef9ea748aebb3eded5bbe472c5ddb8 not found: ID does not exist" May 29 17:00:04.830045 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.830016 2549 scope.go:117] "RemoveContainer" containerID="d4afe07b4abd51548797adc00a40b267f62379b7139fa8569aee6c5a98c4f1f4" May 29 17:00:04.830144 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.830064 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-62ssvscooh466\"" May 29 17:00:04.830265 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.830240 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4afe07b4abd51548797adc00a40b267f62379b7139fa8569aee6c5a98c4f1f4"} err="failed to get container status \"d4afe07b4abd51548797adc00a40b267f62379b7139fa8569aee6c5a98c4f1f4\": rpc error: code = NotFound desc = could not find container \"d4afe07b4abd51548797adc00a40b267f62379b7139fa8569aee6c5a98c4f1f4\": container with ID starting with d4afe07b4abd51548797adc00a40b267f62379b7139fa8569aee6c5a98c4f1f4 not found: ID does not exist" May 29 17:00:04.830265 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.830263 2549 scope.go:117] "RemoveContainer" containerID="6d8b6580d1d20aba4363be3f701ad3d3db74f1186745d56e714f5678e4de440f" May 29 17:00:04.830386 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.830334 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" May 29 17:00:04.830386 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.830343 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" May 29 17:00:04.830386 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.830349 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-v78cc\"" May 29 17:00:04.830386 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.830364 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" May 29 17:00:04.830583 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.830400 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" May 29 17:00:04.830583 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.830448 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" May 29 17:00:04.830583 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.830512 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" May 29 17:00:04.830756 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.830742 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" May 29 17:00:04.830815 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.830793 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d8b6580d1d20aba4363be3f701ad3d3db74f1186745d56e714f5678e4de440f"} err="failed to get container status \"6d8b6580d1d20aba4363be3f701ad3d3db74f1186745d56e714f5678e4de440f\": rpc error: code = NotFound desc = could not find container \"6d8b6580d1d20aba4363be3f701ad3d3db74f1186745d56e714f5678e4de440f\": container with ID starting with 6d8b6580d1d20aba4363be3f701ad3d3db74f1186745d56e714f5678e4de440f not found: ID does not exist" May 29 17:00:04.830899 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.830818 2549 scope.go:117] "RemoveContainer" containerID="d295027d80b4a8cbfc3cbc5a627a6e7f1b172228c8e8e5eef967d194590b7968" May 29 17:00:04.831246 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.831220 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d295027d80b4a8cbfc3cbc5a627a6e7f1b172228c8e8e5eef967d194590b7968"} err="failed to get container status \"d295027d80b4a8cbfc3cbc5a627a6e7f1b172228c8e8e5eef967d194590b7968\": rpc error: code = NotFound desc = could not find container \"d295027d80b4a8cbfc3cbc5a627a6e7f1b172228c8e8e5eef967d194590b7968\": container with ID starting with d295027d80b4a8cbfc3cbc5a627a6e7f1b172228c8e8e5eef967d194590b7968 not found: ID does not exist" May 29 17:00:04.831246 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.831246 2549 scope.go:117] "RemoveContainer" containerID="f1a278f7d38c84c4614b922dc3772442d3e0ebe391f9f05fc05e66d03efecb66" May 29 17:00:04.831566 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.831537 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1a278f7d38c84c4614b922dc3772442d3e0ebe391f9f05fc05e66d03efecb66"} err="failed to get container status \"f1a278f7d38c84c4614b922dc3772442d3e0ebe391f9f05fc05e66d03efecb66\": rpc error: code = NotFound desc = could not find container \"f1a278f7d38c84c4614b922dc3772442d3e0ebe391f9f05fc05e66d03efecb66\": container with ID starting with f1a278f7d38c84c4614b922dc3772442d3e0ebe391f9f05fc05e66d03efecb66 not found: ID does not exist" May 29 17:00:04.831566 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.831567 2549 scope.go:117] "RemoveContainer" containerID="a84f0bdc2a0fe70215697f108054fe50419efb6b8fc2295e28b6d9b4d9fe78b0" May 29 17:00:04.831805 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.831780 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a84f0bdc2a0fe70215697f108054fe50419efb6b8fc2295e28b6d9b4d9fe78b0"} err="failed to get container status \"a84f0bdc2a0fe70215697f108054fe50419efb6b8fc2295e28b6d9b4d9fe78b0\": rpc error: code = NotFound desc = could not find container \"a84f0bdc2a0fe70215697f108054fe50419efb6b8fc2295e28b6d9b4d9fe78b0\": container with ID starting with a84f0bdc2a0fe70215697f108054fe50419efb6b8fc2295e28b6d9b4d9fe78b0 not found: ID does not exist" May 29 17:00:04.831925 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.831808 2549 scope.go:117] "RemoveContainer" containerID="0198c250e9fdde86bd791e4add6349098d34bd5257b132452d885c4f970a12db" May 29 17:00:04.832118 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.832095 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0198c250e9fdde86bd791e4add6349098d34bd5257b132452d885c4f970a12db"} err="failed to get container status \"0198c250e9fdde86bd791e4add6349098d34bd5257b132452d885c4f970a12db\": rpc error: code = NotFound desc = could not find container \"0198c250e9fdde86bd791e4add6349098d34bd5257b132452d885c4f970a12db\": container with ID starting with 0198c250e9fdde86bd791e4add6349098d34bd5257b132452d885c4f970a12db not found: ID does not exist" May 29 17:00:04.832179 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.832119 2549 scope.go:117] "RemoveContainer" containerID="ddc7d497a54ac3be13465e507ce387e46bef9ea748aebb3eded5bbe472c5ddb8" May 29 17:00:04.832400 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.832376 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddc7d497a54ac3be13465e507ce387e46bef9ea748aebb3eded5bbe472c5ddb8"} err="failed to get container status \"ddc7d497a54ac3be13465e507ce387e46bef9ea748aebb3eded5bbe472c5ddb8\": rpc error: code = NotFound desc = could not find container \"ddc7d497a54ac3be13465e507ce387e46bef9ea748aebb3eded5bbe472c5ddb8\": container with ID starting with ddc7d497a54ac3be13465e507ce387e46bef9ea748aebb3eded5bbe472c5ddb8 not found: ID does not exist" May 29 17:00:04.833131 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.833116 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" May 29 17:00:04.835557 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.835536 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" May 29 17:00:04.839189 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.839170 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] May 29 17:00:04.908087 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.908062 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/515b776d-6740-4741-80a7-8759378a7e42-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:04.908204 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.908097 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/515b776d-6740-4741-80a7-8759378a7e42-config\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:04.908204 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.908114 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/515b776d-6740-4741-80a7-8759378a7e42-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:04.908204 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.908144 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/515b776d-6740-4741-80a7-8759378a7e42-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:04.908204 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.908181 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/515b776d-6740-4741-80a7-8759378a7e42-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:04.908204 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.908202 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/515b776d-6740-4741-80a7-8759378a7e42-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:04.908373 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.908226 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/515b776d-6740-4741-80a7-8759378a7e42-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:04.908373 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.908242 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rzgf\" (UniqueName: \"kubernetes.io/projected/515b776d-6740-4741-80a7-8759378a7e42-kube-api-access-6rzgf\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:04.908373 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.908259 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/515b776d-6740-4741-80a7-8759378a7e42-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:04.908373 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.908275 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/515b776d-6740-4741-80a7-8759378a7e42-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:04.908373 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.908295 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/515b776d-6740-4741-80a7-8759378a7e42-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:04.908373 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.908313 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/515b776d-6740-4741-80a7-8759378a7e42-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:04.908373 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.908330 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/515b776d-6740-4741-80a7-8759378a7e42-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:04.908373 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.908348 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/515b776d-6740-4741-80a7-8759378a7e42-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:04.908373 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.908365 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/515b776d-6740-4741-80a7-8759378a7e42-web-config\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:04.908654 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.908409 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/515b776d-6740-4741-80a7-8759378a7e42-config-out\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:04.908654 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.908427 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/515b776d-6740-4741-80a7-8759378a7e42-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:04.908654 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:04.908441 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/515b776d-6740-4741-80a7-8759378a7e42-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:05.009561 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:05.009459 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/515b776d-6740-4741-80a7-8759378a7e42-config-out\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:05.009561 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:05.009513 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/515b776d-6740-4741-80a7-8759378a7e42-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:05.009561 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:05.009531 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/515b776d-6740-4741-80a7-8759378a7e42-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:05.009561 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:05.009558 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/515b776d-6740-4741-80a7-8759378a7e42-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:05.009887 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:05.009579 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/515b776d-6740-4741-80a7-8759378a7e42-config\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:05.009887 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:05.009677 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/515b776d-6740-4741-80a7-8759378a7e42-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:05.009887 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:05.009729 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/515b776d-6740-4741-80a7-8759378a7e42-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:05.009887 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:05.009765 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/515b776d-6740-4741-80a7-8759378a7e42-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:05.009887 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:05.009800 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/515b776d-6740-4741-80a7-8759378a7e42-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:05.009887 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:05.009828 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/515b776d-6740-4741-80a7-8759378a7e42-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:05.009887 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:05.009866 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6rzgf\" (UniqueName: \"kubernetes.io/projected/515b776d-6740-4741-80a7-8759378a7e42-kube-api-access-6rzgf\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:05.010233 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:05.009893 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/515b776d-6740-4741-80a7-8759378a7e42-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:05.010233 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:05.009921 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/515b776d-6740-4741-80a7-8759378a7e42-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:05.010233 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:05.009948 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/515b776d-6740-4741-80a7-8759378a7e42-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:05.010233 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:05.009974 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/515b776d-6740-4741-80a7-8759378a7e42-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:05.010233 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:05.010005 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/515b776d-6740-4741-80a7-8759378a7e42-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:05.010233 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:05.010046 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/515b776d-6740-4741-80a7-8759378a7e42-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:05.010233 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:05.010068 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/515b776d-6740-4741-80a7-8759378a7e42-web-config\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:05.012720 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:05.012524 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/515b776d-6740-4741-80a7-8759378a7e42-config-out\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:05.012720 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:05.012617 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/515b776d-6740-4741-80a7-8759378a7e42-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:05.012720 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:05.012649 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/515b776d-6740-4741-80a7-8759378a7e42-config\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:05.012968 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:05.012815 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/515b776d-6740-4741-80a7-8759378a7e42-web-config\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:05.012968 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:05.012898 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/515b776d-6740-4741-80a7-8759378a7e42-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:05.014016 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:05.013225 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/515b776d-6740-4741-80a7-8759378a7e42-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:05.014016 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:05.013372 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/515b776d-6740-4741-80a7-8759378a7e42-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:05.014016 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:05.013392 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/515b776d-6740-4741-80a7-8759378a7e42-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:05.014016 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:05.013672 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/515b776d-6740-4741-80a7-8759378a7e42-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:05.014016 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:05.013676 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/515b776d-6740-4741-80a7-8759378a7e42-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:05.014016 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:05.013965 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/515b776d-6740-4741-80a7-8759378a7e42-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:05.014359 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:05.014062 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/515b776d-6740-4741-80a7-8759378a7e42-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:05.014534 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:05.014513 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/515b776d-6740-4741-80a7-8759378a7e42-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:05.015111 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:05.015083 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/515b776d-6740-4741-80a7-8759378a7e42-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:05.015568 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:05.015544 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/515b776d-6740-4741-80a7-8759378a7e42-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:05.015971 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:05.015955 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/515b776d-6740-4741-80a7-8759378a7e42-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:05.016473 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:05.016454 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/515b776d-6740-4741-80a7-8759378a7e42-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:05.020811 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:05.020792 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rzgf\" (UniqueName: \"kubernetes.io/projected/515b776d-6740-4741-80a7-8759378a7e42-kube-api-access-6rzgf\") pod \"prometheus-k8s-0\" (UID: \"515b776d-6740-4741-80a7-8759378a7e42\") " pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:05.137887 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:05.137826 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" May 29 17:00:05.266787 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:05.266668 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] May 29 17:00:05.268948 ip-10-0-141-197 kubenswrapper[2549]: W0529 17:00:05.268925 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod515b776d_6740_4741_80a7_8759378a7e42.slice/crio-b69856612c0e633740b865aed3811a69a108d601d18576061d6755899ea6bf69 WatchSource:0}: Error finding container b69856612c0e633740b865aed3811a69a108d601d18576061d6755899ea6bf69: Status 404 returned error can't find the container with id b69856612c0e633740b865aed3811a69a108d601d18576061d6755899ea6bf69 May 29 17:00:05.781391 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:05.781359 2549 generic.go:358] "Generic (PLEG): container finished" podID="515b776d-6740-4741-80a7-8759378a7e42" containerID="1e07e209f2984d1b44cc3846d8cdc8ba02d150900fe5f9cd2680cadff99bd71f" exitCode=0 May 29 17:00:05.781744 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:05.781446 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"515b776d-6740-4741-80a7-8759378a7e42","Type":"ContainerDied","Data":"1e07e209f2984d1b44cc3846d8cdc8ba02d150900fe5f9cd2680cadff99bd71f"} May 29 17:00:05.781744 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:05.781483 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"515b776d-6740-4741-80a7-8759378a7e42","Type":"ContainerStarted","Data":"b69856612c0e633740b865aed3811a69a108d601d18576061d6755899ea6bf69"} May 29 17:00:06.085379 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:06.085348 2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b595abfc-f4c3-43b7-94b9-b6eb00520ca8" path="/var/lib/kubelet/pods/b595abfc-f4c3-43b7-94b9-b6eb00520ca8/volumes" May 29 17:00:06.787615 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:06.787579 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"515b776d-6740-4741-80a7-8759378a7e42","Type":"ContainerStarted","Data":"51629606560dbbdc5b6dbe3f51ea21bd6e4253491a594fb7bb440de797d7e55c"} May 29 17:00:06.787615 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:06.787615 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"515b776d-6740-4741-80a7-8759378a7e42","Type":"ContainerStarted","Data":"03bd88a20d48acf42465542b1938b1843bf8e55304cd54a4f50fbaa67068910c"} May 29 17:00:06.788039 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:06.787625 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"515b776d-6740-4741-80a7-8759378a7e42","Type":"ContainerStarted","Data":"ac1640af08cdf379c1be893c35dced278b9a36dfdd6fe046653b3b2dcb23b021"} May 29 17:00:06.788039 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:06.787634 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"515b776d-6740-4741-80a7-8759378a7e42","Type":"ContainerStarted","Data":"d9f7ed5b861a1bed28f9d3bd3a976faa483b112938f01156d9124e6a24e8aa2e"} May 29 17:00:06.788039 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:06.787643 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"515b776d-6740-4741-80a7-8759378a7e42","Type":"ContainerStarted","Data":"cee4c3ede17c2f9a8c03b220d5719147ed5979ffcb4c7369559dd177d105dc51"} May 29 17:00:06.788039 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:06.787652 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"515b776d-6740-4741-80a7-8759378a7e42","Type":"ContainerStarted","Data":"6ce413da307d6d02052db4d480ac20767eeb321d7a09bcbc5bb7d7a8b61442a0"} May 29 17:00:06.813138 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:06.813089 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.813072471 podStartE2EDuration="2.813072471s" podCreationTimestamp="2026-05-29 17:00:04 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 17:00:06.810373645 +0000 UTC m=+197.296802858" watchObservedRunningTime="2026-05-29 17:00:06.813072471 +0000 UTC m=+197.299501674" May 29 17:00:10.138778 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:00:10.138748 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" May 29 17:01:05.137955 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:05.137915 2549 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" May 29 17:01:05.153065 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:05.153032 2549 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" May 29 17:01:05.967067 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:05.967041 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" May 29 17:01:30.657294 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:30.657255 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-79c8d999ff-nnd7l"] May 29 17:01:30.660521 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:30.660503 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-79c8d999ff-nnd7l" May 29 17:01:30.662461 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:30.662438 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" May 29 17:01:30.662606 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:30.662468 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" May 29 17:01:30.662879 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:30.662865 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-6bck4\"" May 29 17:01:30.669922 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:30.669896 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-79c8d999ff-nnd7l"] May 29 17:01:30.780092 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:30.780055 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpkz6\" (UniqueName: \"kubernetes.io/projected/d3cb6b30-edd3-4fc2-a287-14a49e3e109c-kube-api-access-vpkz6\") pod \"cert-manager-79c8d999ff-nnd7l\" (UID: \"d3cb6b30-edd3-4fc2-a287-14a49e3e109c\") " pod="cert-manager/cert-manager-79c8d999ff-nnd7l" May 29 17:01:30.780092 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:30.780097 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d3cb6b30-edd3-4fc2-a287-14a49e3e109c-bound-sa-token\") pod \"cert-manager-79c8d999ff-nnd7l\" (UID: \"d3cb6b30-edd3-4fc2-a287-14a49e3e109c\") " pod="cert-manager/cert-manager-79c8d999ff-nnd7l" May 29 17:01:30.881381 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:30.881351 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vpkz6\" (UniqueName: \"kubernetes.io/projected/d3cb6b30-edd3-4fc2-a287-14a49e3e109c-kube-api-access-vpkz6\") pod \"cert-manager-79c8d999ff-nnd7l\" (UID: \"d3cb6b30-edd3-4fc2-a287-14a49e3e109c\") " pod="cert-manager/cert-manager-79c8d999ff-nnd7l" May 29 17:01:30.881519 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:30.881390 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d3cb6b30-edd3-4fc2-a287-14a49e3e109c-bound-sa-token\") pod \"cert-manager-79c8d999ff-nnd7l\" (UID: \"d3cb6b30-edd3-4fc2-a287-14a49e3e109c\") " pod="cert-manager/cert-manager-79c8d999ff-nnd7l" May 29 17:01:30.889487 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:30.889465 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpkz6\" (UniqueName: \"kubernetes.io/projected/d3cb6b30-edd3-4fc2-a287-14a49e3e109c-kube-api-access-vpkz6\") pod \"cert-manager-79c8d999ff-nnd7l\" (UID: \"d3cb6b30-edd3-4fc2-a287-14a49e3e109c\") " pod="cert-manager/cert-manager-79c8d999ff-nnd7l" May 29 17:01:30.889674 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:30.889655 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d3cb6b30-edd3-4fc2-a287-14a49e3e109c-bound-sa-token\") pod \"cert-manager-79c8d999ff-nnd7l\" (UID: \"d3cb6b30-edd3-4fc2-a287-14a49e3e109c\") " pod="cert-manager/cert-manager-79c8d999ff-nnd7l" May 29 17:01:30.980781 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:30.980708 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-79c8d999ff-nnd7l" May 29 17:01:31.099181 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:31.099149 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-79c8d999ff-nnd7l"] May 29 17:01:31.102807 ip-10-0-141-197 kubenswrapper[2549]: W0529 17:01:31.102781 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd3cb6b30_edd3_4fc2_a287_14a49e3e109c.slice/crio-d23c083686ece47be34e5f23b37ab8ced281cfa38554a2061e07f9af9b1f5fda WatchSource:0}: Error finding container d23c083686ece47be34e5f23b37ab8ced281cfa38554a2061e07f9af9b1f5fda: Status 404 returned error can't find the container with id d23c083686ece47be34e5f23b37ab8ced281cfa38554a2061e07f9af9b1f5fda May 29 17:01:32.023941 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:32.023890 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-79c8d999ff-nnd7l" event={"ID":"d3cb6b30-edd3-4fc2-a287-14a49e3e109c","Type":"ContainerStarted","Data":"d23c083686ece47be34e5f23b37ab8ced281cfa38554a2061e07f9af9b1f5fda"} May 29 17:01:34.030243 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:34.030204 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-79c8d999ff-nnd7l" event={"ID":"d3cb6b30-edd3-4fc2-a287-14a49e3e109c","Type":"ContainerStarted","Data":"60304f7779f34e98b444c61f0c3e83a10b24dfc5a8dbc75f99140f34abb1c401"} May 29 17:01:34.045439 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:34.045345 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-79c8d999ff-nnd7l" podStartSLOduration=1.40042664 podStartE2EDuration="4.045329605s" podCreationTimestamp="2026-05-29 17:01:30 +0000 UTC" firstStartedPulling="2026-05-29 17:01:31.104541217 +0000 UTC m=+281.590970397" lastFinishedPulling="2026-05-29 17:01:33.749444171 +0000 UTC m=+284.235873362" observedRunningTime="2026-05-29 17:01:34.044750876 +0000 UTC m=+284.531180077" watchObservedRunningTime="2026-05-29 17:01:34.045329605 +0000 UTC m=+284.531758809" May 29 17:01:45.788244 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:45.788210 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-598f575fbf-xfn72"] May 29 17:01:45.792584 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:45.792563 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-598f575fbf-xfn72" May 29 17:01:45.794641 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:45.794617 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" May 29 17:01:45.794761 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:45.794632 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" May 29 17:01:45.794761 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:45.794620 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-57nb9\"" May 29 17:01:45.794761 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:45.794634 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" May 29 17:01:45.794911 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:45.794617 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" May 29 17:01:45.802352 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:45.802331 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c2f55f24-6370-45fd-8cba-77688e4778ba-manifests\") pod \"opendatahub-operator-controller-manager-598f575fbf-xfn72\" (UID: \"c2f55f24-6370-45fd-8cba-77688e4778ba\") " pod="opendatahub/opendatahub-operator-controller-manager-598f575fbf-xfn72" May 29 17:01:45.802429 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:45.802388 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c2f55f24-6370-45fd-8cba-77688e4778ba-apiservice-cert\") pod \"opendatahub-operator-controller-manager-598f575fbf-xfn72\" (UID: \"c2f55f24-6370-45fd-8cba-77688e4778ba\") " pod="opendatahub/opendatahub-operator-controller-manager-598f575fbf-xfn72" May 29 17:01:45.802429 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:45.802408 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dz9p5\" (UniqueName: \"kubernetes.io/projected/c2f55f24-6370-45fd-8cba-77688e4778ba-kube-api-access-dz9p5\") pod \"opendatahub-operator-controller-manager-598f575fbf-xfn72\" (UID: \"c2f55f24-6370-45fd-8cba-77688e4778ba\") " pod="opendatahub/opendatahub-operator-controller-manager-598f575fbf-xfn72" May 29 17:01:45.802500 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:45.802431 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2f55f24-6370-45fd-8cba-77688e4778ba-webhook-cert\") pod \"opendatahub-operator-controller-manager-598f575fbf-xfn72\" (UID: \"c2f55f24-6370-45fd-8cba-77688e4778ba\") " pod="opendatahub/opendatahub-operator-controller-manager-598f575fbf-xfn72" May 29 17:01:45.802500 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:45.802447 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2f55f24-6370-45fd-8cba-77688e4778ba-tmp\") pod \"opendatahub-operator-controller-manager-598f575fbf-xfn72\" (UID: \"c2f55f24-6370-45fd-8cba-77688e4778ba\") " pod="opendatahub/opendatahub-operator-controller-manager-598f575fbf-xfn72" May 29 17:01:45.806110 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:45.806090 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-598f575fbf-xfn72"] May 29 17:01:45.903583 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:45.903554 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c2f55f24-6370-45fd-8cba-77688e4778ba-apiservice-cert\") pod \"opendatahub-operator-controller-manager-598f575fbf-xfn72\" (UID: \"c2f55f24-6370-45fd-8cba-77688e4778ba\") " pod="opendatahub/opendatahub-operator-controller-manager-598f575fbf-xfn72" May 29 17:01:45.903583 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:45.903586 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dz9p5\" (UniqueName: \"kubernetes.io/projected/c2f55f24-6370-45fd-8cba-77688e4778ba-kube-api-access-dz9p5\") pod \"opendatahub-operator-controller-manager-598f575fbf-xfn72\" (UID: \"c2f55f24-6370-45fd-8cba-77688e4778ba\") " pod="opendatahub/opendatahub-operator-controller-manager-598f575fbf-xfn72" May 29 17:01:45.903760 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:45.903618 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2f55f24-6370-45fd-8cba-77688e4778ba-webhook-cert\") pod \"opendatahub-operator-controller-manager-598f575fbf-xfn72\" (UID: \"c2f55f24-6370-45fd-8cba-77688e4778ba\") " pod="opendatahub/opendatahub-operator-controller-manager-598f575fbf-xfn72" May 29 17:01:45.903760 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:45.903641 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2f55f24-6370-45fd-8cba-77688e4778ba-tmp\") pod \"opendatahub-operator-controller-manager-598f575fbf-xfn72\" (UID: \"c2f55f24-6370-45fd-8cba-77688e4778ba\") " pod="opendatahub/opendatahub-operator-controller-manager-598f575fbf-xfn72" May 29 17:01:45.903760 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:45.903686 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c2f55f24-6370-45fd-8cba-77688e4778ba-manifests\") pod \"opendatahub-operator-controller-manager-598f575fbf-xfn72\" (UID: \"c2f55f24-6370-45fd-8cba-77688e4778ba\") " pod="opendatahub/opendatahub-operator-controller-manager-598f575fbf-xfn72" May 29 17:01:45.904201 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:45.904181 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c2f55f24-6370-45fd-8cba-77688e4778ba-manifests\") pod \"opendatahub-operator-controller-manager-598f575fbf-xfn72\" (UID: \"c2f55f24-6370-45fd-8cba-77688e4778ba\") " pod="opendatahub/opendatahub-operator-controller-manager-598f575fbf-xfn72" May 29 17:01:45.906092 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:45.906063 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2f55f24-6370-45fd-8cba-77688e4778ba-tmp\") pod \"opendatahub-operator-controller-manager-598f575fbf-xfn72\" (UID: \"c2f55f24-6370-45fd-8cba-77688e4778ba\") " pod="opendatahub/opendatahub-operator-controller-manager-598f575fbf-xfn72" May 29 17:01:45.906185 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:45.906112 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c2f55f24-6370-45fd-8cba-77688e4778ba-apiservice-cert\") pod \"opendatahub-operator-controller-manager-598f575fbf-xfn72\" (UID: \"c2f55f24-6370-45fd-8cba-77688e4778ba\") " pod="opendatahub/opendatahub-operator-controller-manager-598f575fbf-xfn72" May 29 17:01:45.906226 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:45.906196 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2f55f24-6370-45fd-8cba-77688e4778ba-webhook-cert\") pod \"opendatahub-operator-controller-manager-598f575fbf-xfn72\" (UID: \"c2f55f24-6370-45fd-8cba-77688e4778ba\") " pod="opendatahub/opendatahub-operator-controller-manager-598f575fbf-xfn72" May 29 17:01:45.919339 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:45.919316 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dz9p5\" (UniqueName: \"kubernetes.io/projected/c2f55f24-6370-45fd-8cba-77688e4778ba-kube-api-access-dz9p5\") pod \"opendatahub-operator-controller-manager-598f575fbf-xfn72\" (UID: \"c2f55f24-6370-45fd-8cba-77688e4778ba\") " pod="opendatahub/opendatahub-operator-controller-manager-598f575fbf-xfn72" May 29 17:01:46.102401 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:46.102314 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-598f575fbf-xfn72" May 29 17:01:46.233659 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:46.233571 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-598f575fbf-xfn72"] May 29 17:01:46.238807 ip-10-0-141-197 kubenswrapper[2549]: W0529 17:01:46.238777 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2f55f24_6370_45fd_8cba_77688e4778ba.slice/crio-d6bed870c8bbed6ef246ea076d1e42edb66c3c4b34ce27cee09fa23cfaedc2fc WatchSource:0}: Error finding container d6bed870c8bbed6ef246ea076d1e42edb66c3c4b34ce27cee09fa23cfaedc2fc: Status 404 returned error can't find the container with id d6bed870c8bbed6ef246ea076d1e42edb66c3c4b34ce27cee09fa23cfaedc2fc May 29 17:01:47.067373 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:47.067327 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-598f575fbf-xfn72" event={"ID":"c2f55f24-6370-45fd-8cba-77688e4778ba","Type":"ContainerStarted","Data":"d6bed870c8bbed6ef246ea076d1e42edb66c3c4b34ce27cee09fa23cfaedc2fc"} May 29 17:01:49.994301 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:49.994273 2549 kubelet.go:1628] "Image garbage collection succeeded" May 29 17:01:52.083939 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:52.083833 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-598f575fbf-xfn72" event={"ID":"c2f55f24-6370-45fd-8cba-77688e4778ba","Type":"ContainerStarted","Data":"b6c33c1388bafef6a18f2836e8b773d6b17f80ac6f84891d4b0af02707af9440"} May 29 17:01:53.087536 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:53.087445 2549 generic.go:358] "Generic (PLEG): container finished" podID="c2f55f24-6370-45fd-8cba-77688e4778ba" containerID="b6c33c1388bafef6a18f2836e8b773d6b17f80ac6f84891d4b0af02707af9440" exitCode=0 May 29 17:01:53.087884 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:53.087532 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-598f575fbf-xfn72" event={"ID":"c2f55f24-6370-45fd-8cba-77688e4778ba","Type":"ContainerDied","Data":"b6c33c1388bafef6a18f2836e8b773d6b17f80ac6f84891d4b0af02707af9440"} May 29 17:01:53.088005 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:53.087989 2549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 29 17:01:54.091827 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:54.091789 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-598f575fbf-xfn72" event={"ID":"c2f55f24-6370-45fd-8cba-77688e4778ba","Type":"ContainerStarted","Data":"6e5073297972795a0ee78d92b4aed739bcab2fd9c8208698c127380df50706ad"} May 29 17:01:54.092212 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:54.091988 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-598f575fbf-xfn72" May 29 17:01:54.128009 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:01:54.127959 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-598f575fbf-xfn72" podStartSLOduration=1.959908411 podStartE2EDuration="9.127942384s" podCreationTimestamp="2026-05-29 17:01:45 +0000 UTC" firstStartedPulling="2026-05-29 17:01:46.24054152 +0000 UTC m=+296.726970702" lastFinishedPulling="2026-05-29 17:01:53.408575483 +0000 UTC m=+303.895004675" observedRunningTime="2026-05-29 17:01:54.127203878 +0000 UTC m=+304.613633080" watchObservedRunningTime="2026-05-29 17:01:54.127942384 +0000 UTC m=+304.614371587" May 29 17:02:05.096249 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:05.096221 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-598f575fbf-xfn72" May 29 17:02:07.033675 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:07.033640 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6fbdcf5ddf-w5vnb"] May 29 17:02:07.040098 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:07.040076 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6fbdcf5ddf-w5vnb" May 29 17:02:07.042465 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:07.042444 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" May 29 17:02:07.042811 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:07.042788 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-g4q57\"" May 29 17:02:07.042932 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:07.042788 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" May 29 17:02:07.042932 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:07.042828 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" May 29 17:02:07.043050 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:07.042980 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" May 29 17:02:07.049111 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:07.049090 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6fbdcf5ddf-w5vnb"] May 29 17:02:07.073486 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:07.073459 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcaa6078-0d3a-4ae6-b47f-eaf5b897c091-tls-certs\") pod \"kube-auth-proxy-6fbdcf5ddf-w5vnb\" (UID: \"fcaa6078-0d3a-4ae6-b47f-eaf5b897c091\") " pod="openshift-ingress/kube-auth-proxy-6fbdcf5ddf-w5vnb" May 29 17:02:07.073615 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:07.073541 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xd858\" (UniqueName: \"kubernetes.io/projected/fcaa6078-0d3a-4ae6-b47f-eaf5b897c091-kube-api-access-xd858\") pod \"kube-auth-proxy-6fbdcf5ddf-w5vnb\" (UID: \"fcaa6078-0d3a-4ae6-b47f-eaf5b897c091\") " pod="openshift-ingress/kube-auth-proxy-6fbdcf5ddf-w5vnb" May 29 17:02:07.073615 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:07.073592 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fcaa6078-0d3a-4ae6-b47f-eaf5b897c091-tmp\") pod \"kube-auth-proxy-6fbdcf5ddf-w5vnb\" (UID: \"fcaa6078-0d3a-4ae6-b47f-eaf5b897c091\") " pod="openshift-ingress/kube-auth-proxy-6fbdcf5ddf-w5vnb" May 29 17:02:07.174578 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:07.174546 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcaa6078-0d3a-4ae6-b47f-eaf5b897c091-tls-certs\") pod \"kube-auth-proxy-6fbdcf5ddf-w5vnb\" (UID: \"fcaa6078-0d3a-4ae6-b47f-eaf5b897c091\") " pod="openshift-ingress/kube-auth-proxy-6fbdcf5ddf-w5vnb" May 29 17:02:07.174768 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:07.174590 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xd858\" (UniqueName: \"kubernetes.io/projected/fcaa6078-0d3a-4ae6-b47f-eaf5b897c091-kube-api-access-xd858\") pod \"kube-auth-proxy-6fbdcf5ddf-w5vnb\" (UID: \"fcaa6078-0d3a-4ae6-b47f-eaf5b897c091\") " pod="openshift-ingress/kube-auth-proxy-6fbdcf5ddf-w5vnb" May 29 17:02:07.174768 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:07.174627 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fcaa6078-0d3a-4ae6-b47f-eaf5b897c091-tmp\") pod \"kube-auth-proxy-6fbdcf5ddf-w5vnb\" (UID: \"fcaa6078-0d3a-4ae6-b47f-eaf5b897c091\") " pod="openshift-ingress/kube-auth-proxy-6fbdcf5ddf-w5vnb" May 29 17:02:07.176893 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:07.176871 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fcaa6078-0d3a-4ae6-b47f-eaf5b897c091-tmp\") pod \"kube-auth-proxy-6fbdcf5ddf-w5vnb\" (UID: \"fcaa6078-0d3a-4ae6-b47f-eaf5b897c091\") " pod="openshift-ingress/kube-auth-proxy-6fbdcf5ddf-w5vnb" May 29 17:02:07.177101 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:07.177081 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcaa6078-0d3a-4ae6-b47f-eaf5b897c091-tls-certs\") pod \"kube-auth-proxy-6fbdcf5ddf-w5vnb\" (UID: \"fcaa6078-0d3a-4ae6-b47f-eaf5b897c091\") " pod="openshift-ingress/kube-auth-proxy-6fbdcf5ddf-w5vnb" May 29 17:02:07.189663 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:07.189642 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xd858\" (UniqueName: \"kubernetes.io/projected/fcaa6078-0d3a-4ae6-b47f-eaf5b897c091-kube-api-access-xd858\") pod \"kube-auth-proxy-6fbdcf5ddf-w5vnb\" (UID: \"fcaa6078-0d3a-4ae6-b47f-eaf5b897c091\") " pod="openshift-ingress/kube-auth-proxy-6fbdcf5ddf-w5vnb" May 29 17:02:07.351303 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:07.351212 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6fbdcf5ddf-w5vnb" May 29 17:02:07.477714 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:07.477689 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6fbdcf5ddf-w5vnb"] May 29 17:02:07.480001 ip-10-0-141-197 kubenswrapper[2549]: W0529 17:02:07.479976 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcaa6078_0d3a_4ae6_b47f_eaf5b897c091.slice/crio-b7e840e989fdf5d8ce1764bfc070b90442096dcb8715c6ff1701d2bcb8af22cb WatchSource:0}: Error finding container b7e840e989fdf5d8ce1764bfc070b90442096dcb8715c6ff1701d2bcb8af22cb: Status 404 returned error can't find the container with id b7e840e989fdf5d8ce1764bfc070b90442096dcb8715c6ff1701d2bcb8af22cb May 29 17:02:08.135159 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:08.134986 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6fbdcf5ddf-w5vnb" event={"ID":"fcaa6078-0d3a-4ae6-b47f-eaf5b897c091","Type":"ContainerStarted","Data":"b7e840e989fdf5d8ce1764bfc070b90442096dcb8715c6ff1701d2bcb8af22cb"} May 29 17:02:11.145617 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:11.145580 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6fbdcf5ddf-w5vnb" event={"ID":"fcaa6078-0d3a-4ae6-b47f-eaf5b897c091","Type":"ContainerStarted","Data":"1c95f4d3ecf3b76b35fa2ec581f4d13c9cc3c105269872f1998cf2e622eb45cc"} May 29 17:02:11.161568 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:11.161524 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6fbdcf5ddf-w5vnb" podStartSLOduration=1.061678967 podStartE2EDuration="4.161509566s" podCreationTimestamp="2026-05-29 17:02:07 +0000 UTC" firstStartedPulling="2026-05-29 17:02:07.481688961 +0000 UTC m=+317.968118141" lastFinishedPulling="2026-05-29 17:02:10.581519555 +0000 UTC m=+321.067948740" observedRunningTime="2026-05-29 17:02:11.159886206 +0000 UTC m=+321.646315408" watchObservedRunningTime="2026-05-29 17:02:11.161509566 +0000 UTC m=+321.647938767" May 29 17:02:15.014469 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:15.014432 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-674dbbfb6b-bqv25"] May 29 17:02:15.017766 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:15.017744 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-674dbbfb6b-bqv25" May 29 17:02:15.020298 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:15.020274 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" May 29 17:02:15.020298 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:15.020289 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" May 29 17:02:15.020469 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:15.020307 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" May 29 17:02:15.020469 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:15.020274 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" May 29 17:02:15.020469 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:15.020289 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-hlfb6\"" May 29 17:02:15.020469 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:15.020293 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" May 29 17:02:15.025370 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:15.025346 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-674dbbfb6b-bqv25"] May 29 17:02:15.138467 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:15.138442 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/65c6a255-c62f-47c2-8a95-af480b16ed10-metrics-cert\") pod \"lws-controller-manager-674dbbfb6b-bqv25\" (UID: \"65c6a255-c62f-47c2-8a95-af480b16ed10\") " pod="openshift-lws-operator/lws-controller-manager-674dbbfb6b-bqv25" May 29 17:02:15.138613 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:15.138496 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/65c6a255-c62f-47c2-8a95-af480b16ed10-manager-config\") pod \"lws-controller-manager-674dbbfb6b-bqv25\" (UID: \"65c6a255-c62f-47c2-8a95-af480b16ed10\") " pod="openshift-lws-operator/lws-controller-manager-674dbbfb6b-bqv25" May 29 17:02:15.138613 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:15.138526 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8p7s\" (UniqueName: \"kubernetes.io/projected/65c6a255-c62f-47c2-8a95-af480b16ed10-kube-api-access-r8p7s\") pod \"lws-controller-manager-674dbbfb6b-bqv25\" (UID: \"65c6a255-c62f-47c2-8a95-af480b16ed10\") " pod="openshift-lws-operator/lws-controller-manager-674dbbfb6b-bqv25" May 29 17:02:15.138613 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:15.138563 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/65c6a255-c62f-47c2-8a95-af480b16ed10-cert\") pod \"lws-controller-manager-674dbbfb6b-bqv25\" (UID: \"65c6a255-c62f-47c2-8a95-af480b16ed10\") " pod="openshift-lws-operator/lws-controller-manager-674dbbfb6b-bqv25" May 29 17:02:15.238916 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:15.238884 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/65c6a255-c62f-47c2-8a95-af480b16ed10-metrics-cert\") pod \"lws-controller-manager-674dbbfb6b-bqv25\" (UID: \"65c6a255-c62f-47c2-8a95-af480b16ed10\") " pod="openshift-lws-operator/lws-controller-manager-674dbbfb6b-bqv25" May 29 17:02:15.239058 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:15.238936 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/65c6a255-c62f-47c2-8a95-af480b16ed10-manager-config\") pod \"lws-controller-manager-674dbbfb6b-bqv25\" (UID: \"65c6a255-c62f-47c2-8a95-af480b16ed10\") " pod="openshift-lws-operator/lws-controller-manager-674dbbfb6b-bqv25" May 29 17:02:15.239058 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:15.238963 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r8p7s\" (UniqueName: \"kubernetes.io/projected/65c6a255-c62f-47c2-8a95-af480b16ed10-kube-api-access-r8p7s\") pod \"lws-controller-manager-674dbbfb6b-bqv25\" (UID: \"65c6a255-c62f-47c2-8a95-af480b16ed10\") " pod="openshift-lws-operator/lws-controller-manager-674dbbfb6b-bqv25" May 29 17:02:15.239058 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:15.238997 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/65c6a255-c62f-47c2-8a95-af480b16ed10-cert\") pod \"lws-controller-manager-674dbbfb6b-bqv25\" (UID: \"65c6a255-c62f-47c2-8a95-af480b16ed10\") " pod="openshift-lws-operator/lws-controller-manager-674dbbfb6b-bqv25" May 29 17:02:15.239633 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:15.239612 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/65c6a255-c62f-47c2-8a95-af480b16ed10-manager-config\") pod \"lws-controller-manager-674dbbfb6b-bqv25\" (UID: \"65c6a255-c62f-47c2-8a95-af480b16ed10\") " pod="openshift-lws-operator/lws-controller-manager-674dbbfb6b-bqv25" May 29 17:02:15.241323 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:15.241298 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/65c6a255-c62f-47c2-8a95-af480b16ed10-metrics-cert\") pod \"lws-controller-manager-674dbbfb6b-bqv25\" (UID: \"65c6a255-c62f-47c2-8a95-af480b16ed10\") " pod="openshift-lws-operator/lws-controller-manager-674dbbfb6b-bqv25" May 29 17:02:15.241413 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:15.241346 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/65c6a255-c62f-47c2-8a95-af480b16ed10-cert\") pod \"lws-controller-manager-674dbbfb6b-bqv25\" (UID: \"65c6a255-c62f-47c2-8a95-af480b16ed10\") " pod="openshift-lws-operator/lws-controller-manager-674dbbfb6b-bqv25" May 29 17:02:15.246564 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:15.246537 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8p7s\" (UniqueName: \"kubernetes.io/projected/65c6a255-c62f-47c2-8a95-af480b16ed10-kube-api-access-r8p7s\") pod \"lws-controller-manager-674dbbfb6b-bqv25\" (UID: \"65c6a255-c62f-47c2-8a95-af480b16ed10\") " pod="openshift-lws-operator/lws-controller-manager-674dbbfb6b-bqv25" May 29 17:02:15.327113 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:15.327086 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-674dbbfb6b-bqv25" May 29 17:02:15.444855 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:15.444812 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-674dbbfb6b-bqv25"] May 29 17:02:15.447530 ip-10-0-141-197 kubenswrapper[2549]: W0529 17:02:15.447500 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65c6a255_c62f_47c2_8a95_af480b16ed10.slice/crio-09c1dbe5a37ad20187aa6d2efde0e65a83a70c7f6ffc0ea4cf8d563ee0d3bd84 WatchSource:0}: Error finding container 09c1dbe5a37ad20187aa6d2efde0e65a83a70c7f6ffc0ea4cf8d563ee0d3bd84: Status 404 returned error can't find the container with id 09c1dbe5a37ad20187aa6d2efde0e65a83a70c7f6ffc0ea4cf8d563ee0d3bd84 May 29 17:02:16.159578 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:16.159542 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-674dbbfb6b-bqv25" event={"ID":"65c6a255-c62f-47c2-8a95-af480b16ed10","Type":"ContainerStarted","Data":"09c1dbe5a37ad20187aa6d2efde0e65a83a70c7f6ffc0ea4cf8d563ee0d3bd84"} May 29 17:02:19.172368 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:19.172334 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-674dbbfb6b-bqv25" event={"ID":"65c6a255-c62f-47c2-8a95-af480b16ed10","Type":"ContainerStarted","Data":"e161e348cac51b2500700ca3b83ca387b828d02da37d4ce5bb7dbbb476d13fac"} May 29 17:02:19.172763 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:19.172447 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-674dbbfb6b-bqv25" May 29 17:02:19.195228 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:19.195181 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-674dbbfb6b-bqv25" podStartSLOduration=2.527391911 podStartE2EDuration="5.195168573s" podCreationTimestamp="2026-05-29 17:02:14 +0000 UTC" firstStartedPulling="2026-05-29 17:02:15.449452225 +0000 UTC m=+325.935881405" lastFinishedPulling="2026-05-29 17:02:18.117228872 +0000 UTC m=+328.603658067" observedRunningTime="2026-05-29 17:02:19.193343539 +0000 UTC m=+329.679772741" watchObservedRunningTime="2026-05-29 17:02:19.195168573 +0000 UTC m=+329.681597775" May 29 17:02:30.177208 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:02:30.177180 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-674dbbfb6b-bqv25" May 29 17:03:51.826898 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:03:51.826855 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-d67978758-jpn66"] May 29 17:03:51.829962 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:03:51.829945 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d67978758-jpn66" May 29 17:03:51.832365 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:03:51.832342 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" May 29 17:03:51.832471 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:03:51.832376 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" May 29 17:03:51.832933 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:03:51.832901 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" May 29 17:03:51.832933 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:03:51.832920 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5lnvn\"" May 29 17:03:51.833096 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:03:51.832938 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" May 29 17:03:51.833096 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:03:51.832921 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" May 29 17:03:51.833096 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:03:51.833013 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" May 29 17:03:51.833096 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:03:51.833009 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" May 29 17:03:51.838263 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:03:51.838242 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" May 29 17:03:51.841319 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:03:51.841301 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d67978758-jpn66"] May 29 17:03:51.968985 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:03:51.968943 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f259edd2-e076-4195-a908-27bec492bd8e-console-serving-cert\") pod \"console-d67978758-jpn66\" (UID: \"f259edd2-e076-4195-a908-27bec492bd8e\") " pod="openshift-console/console-d67978758-jpn66" May 29 17:03:51.968985 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:03:51.968991 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f259edd2-e076-4195-a908-27bec492bd8e-console-oauth-config\") pod \"console-d67978758-jpn66\" (UID: \"f259edd2-e076-4195-a908-27bec492bd8e\") " pod="openshift-console/console-d67978758-jpn66" May 29 17:03:51.969184 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:03:51.969018 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f259edd2-e076-4195-a908-27bec492bd8e-trusted-ca-bundle\") pod \"console-d67978758-jpn66\" (UID: \"f259edd2-e076-4195-a908-27bec492bd8e\") " pod="openshift-console/console-d67978758-jpn66" May 29 17:03:51.969184 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:03:51.969042 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phw4f\" (UniqueName: \"kubernetes.io/projected/f259edd2-e076-4195-a908-27bec492bd8e-kube-api-access-phw4f\") pod \"console-d67978758-jpn66\" (UID: \"f259edd2-e076-4195-a908-27bec492bd8e\") " pod="openshift-console/console-d67978758-jpn66" May 29 17:03:51.969184 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:03:51.969095 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f259edd2-e076-4195-a908-27bec492bd8e-oauth-serving-cert\") pod \"console-d67978758-jpn66\" (UID: \"f259edd2-e076-4195-a908-27bec492bd8e\") " pod="openshift-console/console-d67978758-jpn66" May 29 17:03:51.969184 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:03:51.969136 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f259edd2-e076-4195-a908-27bec492bd8e-console-config\") pod \"console-d67978758-jpn66\" (UID: \"f259edd2-e076-4195-a908-27bec492bd8e\") " pod="openshift-console/console-d67978758-jpn66" May 29 17:03:51.969184 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:03:51.969166 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f259edd2-e076-4195-a908-27bec492bd8e-service-ca\") pod \"console-d67978758-jpn66\" (UID: \"f259edd2-e076-4195-a908-27bec492bd8e\") " pod="openshift-console/console-d67978758-jpn66" May 29 17:03:52.070270 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:03:52.070219 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f259edd2-e076-4195-a908-27bec492bd8e-console-serving-cert\") pod \"console-d67978758-jpn66\" (UID: \"f259edd2-e076-4195-a908-27bec492bd8e\") " pod="openshift-console/console-d67978758-jpn66" May 29 17:03:52.070453 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:03:52.070279 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f259edd2-e076-4195-a908-27bec492bd8e-console-oauth-config\") pod \"console-d67978758-jpn66\" (UID: \"f259edd2-e076-4195-a908-27bec492bd8e\") " pod="openshift-console/console-d67978758-jpn66" May 29 17:03:52.070453 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:03:52.070321 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f259edd2-e076-4195-a908-27bec492bd8e-trusted-ca-bundle\") pod \"console-d67978758-jpn66\" (UID: \"f259edd2-e076-4195-a908-27bec492bd8e\") " pod="openshift-console/console-d67978758-jpn66" May 29 17:03:52.070453 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:03:52.070344 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-phw4f\" (UniqueName: \"kubernetes.io/projected/f259edd2-e076-4195-a908-27bec492bd8e-kube-api-access-phw4f\") pod \"console-d67978758-jpn66\" (UID: \"f259edd2-e076-4195-a908-27bec492bd8e\") " pod="openshift-console/console-d67978758-jpn66" May 29 17:03:52.070453 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:03:52.070404 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f259edd2-e076-4195-a908-27bec492bd8e-oauth-serving-cert\") pod \"console-d67978758-jpn66\" (UID: \"f259edd2-e076-4195-a908-27bec492bd8e\") " pod="openshift-console/console-d67978758-jpn66" May 29 17:03:52.070453 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:03:52.070447 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f259edd2-e076-4195-a908-27bec492bd8e-console-config\") pod \"console-d67978758-jpn66\" (UID: \"f259edd2-e076-4195-a908-27bec492bd8e\") " pod="openshift-console/console-d67978758-jpn66" May 29 17:03:52.070723 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:03:52.070479 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f259edd2-e076-4195-a908-27bec492bd8e-service-ca\") pod \"console-d67978758-jpn66\" (UID: \"f259edd2-e076-4195-a908-27bec492bd8e\") " pod="openshift-console/console-d67978758-jpn66" May 29 17:03:52.071214 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:03:52.071185 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f259edd2-e076-4195-a908-27bec492bd8e-oauth-serving-cert\") pod \"console-d67978758-jpn66\" (UID: \"f259edd2-e076-4195-a908-27bec492bd8e\") " pod="openshift-console/console-d67978758-jpn66" May 29 17:03:52.071214 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:03:52.071210 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f259edd2-e076-4195-a908-27bec492bd8e-trusted-ca-bundle\") pod \"console-d67978758-jpn66\" (UID: \"f259edd2-e076-4195-a908-27bec492bd8e\") " pod="openshift-console/console-d67978758-jpn66" May 29 17:03:52.071363 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:03:52.071229 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f259edd2-e076-4195-a908-27bec492bd8e-service-ca\") pod \"console-d67978758-jpn66\" (UID: \"f259edd2-e076-4195-a908-27bec492bd8e\") " pod="openshift-console/console-d67978758-jpn66" May 29 17:03:52.071363 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:03:52.071300 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f259edd2-e076-4195-a908-27bec492bd8e-console-config\") pod \"console-d67978758-jpn66\" (UID: \"f259edd2-e076-4195-a908-27bec492bd8e\") " pod="openshift-console/console-d67978758-jpn66" May 29 17:03:52.072828 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:03:52.072808 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f259edd2-e076-4195-a908-27bec492bd8e-console-serving-cert\") pod \"console-d67978758-jpn66\" (UID: \"f259edd2-e076-4195-a908-27bec492bd8e\") " pod="openshift-console/console-d67978758-jpn66" May 29 17:03:52.072828 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:03:52.072823 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f259edd2-e076-4195-a908-27bec492bd8e-console-oauth-config\") pod \"console-d67978758-jpn66\" (UID: \"f259edd2-e076-4195-a908-27bec492bd8e\") " pod="openshift-console/console-d67978758-jpn66" May 29 17:03:52.078476 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:03:52.078426 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phw4f\" (UniqueName: \"kubernetes.io/projected/f259edd2-e076-4195-a908-27bec492bd8e-kube-api-access-phw4f\") pod \"console-d67978758-jpn66\" (UID: \"f259edd2-e076-4195-a908-27bec492bd8e\") " pod="openshift-console/console-d67978758-jpn66" May 29 17:03:52.140621 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:03:52.140586 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d67978758-jpn66" May 29 17:03:52.265079 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:03:52.265053 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d67978758-jpn66"] May 29 17:03:52.267885 ip-10-0-141-197 kubenswrapper[2549]: W0529 17:03:52.267833 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf259edd2_e076_4195_a908_27bec492bd8e.slice/crio-00822c67633466029b86efbde5db08239f988b792e91b6105dea53429ce78819 WatchSource:0}: Error finding container 00822c67633466029b86efbde5db08239f988b792e91b6105dea53429ce78819: Status 404 returned error can't find the container with id 00822c67633466029b86efbde5db08239f988b792e91b6105dea53429ce78819 May 29 17:03:52.432191 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:03:52.432097 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d67978758-jpn66" event={"ID":"f259edd2-e076-4195-a908-27bec492bd8e","Type":"ContainerStarted","Data":"61ff1396b34aed8ace0789497b44950d499654387e96e36c7bd27abfbb383851"} May 29 17:03:52.432191 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:03:52.432136 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d67978758-jpn66" event={"ID":"f259edd2-e076-4195-a908-27bec492bd8e","Type":"ContainerStarted","Data":"00822c67633466029b86efbde5db08239f988b792e91b6105dea53429ce78819"} May 29 17:03:52.450812 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:03:52.450762 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-d67978758-jpn66" podStartSLOduration=1.450745987 podStartE2EDuration="1.450745987s" podCreationTimestamp="2026-05-29 17:03:51 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 17:03:52.449779036 +0000 UTC m=+422.936208240" watchObservedRunningTime="2026-05-29 17:03:52.450745987 +0000 UTC m=+422.937175189" May 29 17:04:01.329691 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:01.329652 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-4xnfw"] May 29 17:04:01.333086 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:01.333070 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-4xnfw" May 29 17:04:01.335081 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:01.335057 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" May 29 17:04:01.335179 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:01.335106 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-jhxp8\"" May 29 17:04:01.335491 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:01.335475 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" May 29 17:04:01.335661 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:01.335648 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" May 29 17:04:01.336314 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:01.336297 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" May 29 17:04:01.342002 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:01.341983 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-4xnfw"] May 29 17:04:01.448018 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:01.447986 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/61542e3a-fe67-47f8-8bd5-adb7b9b93def-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-4xnfw\" (UID: \"61542e3a-fe67-47f8-8bd5-adb7b9b93def\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-4xnfw" May 29 17:04:01.448176 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:01.448105 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/61542e3a-fe67-47f8-8bd5-adb7b9b93def-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-4xnfw\" (UID: \"61542e3a-fe67-47f8-8bd5-adb7b9b93def\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-4xnfw" May 29 17:04:01.448176 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:01.448130 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qkht\" (UniqueName: \"kubernetes.io/projected/61542e3a-fe67-47f8-8bd5-adb7b9b93def-kube-api-access-6qkht\") pod \"kuadrant-console-plugin-6cb54b5c86-4xnfw\" (UID: \"61542e3a-fe67-47f8-8bd5-adb7b9b93def\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-4xnfw" May 29 17:04:01.548760 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:01.548730 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/61542e3a-fe67-47f8-8bd5-adb7b9b93def-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-4xnfw\" (UID: \"61542e3a-fe67-47f8-8bd5-adb7b9b93def\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-4xnfw" May 29 17:04:01.548760 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:01.548761 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6qkht\" (UniqueName: \"kubernetes.io/projected/61542e3a-fe67-47f8-8bd5-adb7b9b93def-kube-api-access-6qkht\") pod \"kuadrant-console-plugin-6cb54b5c86-4xnfw\" (UID: \"61542e3a-fe67-47f8-8bd5-adb7b9b93def\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-4xnfw" May 29 17:04:01.548982 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:01.548801 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/61542e3a-fe67-47f8-8bd5-adb7b9b93def-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-4xnfw\" (UID: \"61542e3a-fe67-47f8-8bd5-adb7b9b93def\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-4xnfw" May 29 17:04:01.548982 ip-10-0-141-197 kubenswrapper[2549]: E0529 17:04:01.548871 2549 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found May 29 17:04:01.548982 ip-10-0-141-197 kubenswrapper[2549]: E0529 17:04:01.548931 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61542e3a-fe67-47f8-8bd5-adb7b9b93def-plugin-serving-cert podName:61542e3a-fe67-47f8-8bd5-adb7b9b93def nodeName:}" failed. No retries permitted until 2026-05-29 17:04:02.048913875 +0000 UTC m=+432.535343056 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/61542e3a-fe67-47f8-8bd5-adb7b9b93def-plugin-serving-cert") pod "kuadrant-console-plugin-6cb54b5c86-4xnfw" (UID: "61542e3a-fe67-47f8-8bd5-adb7b9b93def") : secret "plugin-serving-cert" not found May 29 17:04:01.549379 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:01.549363 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/61542e3a-fe67-47f8-8bd5-adb7b9b93def-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-4xnfw\" (UID: \"61542e3a-fe67-47f8-8bd5-adb7b9b93def\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-4xnfw" May 29 17:04:01.558869 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:01.558833 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qkht\" (UniqueName: \"kubernetes.io/projected/61542e3a-fe67-47f8-8bd5-adb7b9b93def-kube-api-access-6qkht\") pod \"kuadrant-console-plugin-6cb54b5c86-4xnfw\" (UID: \"61542e3a-fe67-47f8-8bd5-adb7b9b93def\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-4xnfw" May 29 17:04:02.053468 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:02.053427 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/61542e3a-fe67-47f8-8bd5-adb7b9b93def-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-4xnfw\" (UID: \"61542e3a-fe67-47f8-8bd5-adb7b9b93def\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-4xnfw" May 29 17:04:02.055936 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:02.055913 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/61542e3a-fe67-47f8-8bd5-adb7b9b93def-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-4xnfw\" (UID: \"61542e3a-fe67-47f8-8bd5-adb7b9b93def\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-4xnfw" May 29 17:04:02.140955 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:02.140923 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-d67978758-jpn66" May 29 17:04:02.141087 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:02.140966 2549 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-d67978758-jpn66" May 29 17:04:02.146477 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:02.146454 2549 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-d67978758-jpn66" May 29 17:04:02.243184 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:02.243158 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-4xnfw" May 29 17:04:02.370162 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:02.370130 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-4xnfw"] May 29 17:04:02.373292 ip-10-0-141-197 kubenswrapper[2549]: W0529 17:04:02.373269 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod61542e3a_fe67_47f8_8bd5_adb7b9b93def.slice/crio-284d54d45deb7b2612e2a7693c1a88ac9932519c7fe843bb6e5b1e44ae5abda0 WatchSource:0}: Error finding container 284d54d45deb7b2612e2a7693c1a88ac9932519c7fe843bb6e5b1e44ae5abda0: Status 404 returned error can't find the container with id 284d54d45deb7b2612e2a7693c1a88ac9932519c7fe843bb6e5b1e44ae5abda0 May 29 17:04:02.465343 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:02.465309 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-4xnfw" event={"ID":"61542e3a-fe67-47f8-8bd5-adb7b9b93def","Type":"ContainerStarted","Data":"284d54d45deb7b2612e2a7693c1a88ac9932519c7fe843bb6e5b1e44ae5abda0"} May 29 17:04:02.469263 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:02.469241 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-d67978758-jpn66" May 29 17:04:10.917931 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:10.917835 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-dqsdk"] May 29 17:04:10.920008 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:10.919990 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-dqsdk" May 29 17:04:10.922538 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:10.922518 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-djg7n\"" May 29 17:04:10.939040 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:10.939016 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-dqsdk"] May 29 17:04:11.034613 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:11.034578 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2q7x\" (UniqueName: \"kubernetes.io/projected/5609b247-7467-4b4b-81e1-9682e49c70a7-kube-api-access-l2q7x\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-dqsdk\" (UID: \"5609b247-7467-4b4b-81e1-9682e49c70a7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-dqsdk" May 29 17:04:11.034821 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:11.034631 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5609b247-7467-4b4b-81e1-9682e49c70a7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-dqsdk\" (UID: \"5609b247-7467-4b4b-81e1-9682e49c70a7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-dqsdk" May 29 17:04:11.135744 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:11.135709 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-l2q7x\" (UniqueName: \"kubernetes.io/projected/5609b247-7467-4b4b-81e1-9682e49c70a7-kube-api-access-l2q7x\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-dqsdk\" (UID: \"5609b247-7467-4b4b-81e1-9682e49c70a7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-dqsdk" May 29 17:04:11.135909 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:11.135752 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5609b247-7467-4b4b-81e1-9682e49c70a7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-dqsdk\" (UID: \"5609b247-7467-4b4b-81e1-9682e49c70a7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-dqsdk" May 29 17:04:11.136159 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:11.136140 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5609b247-7467-4b4b-81e1-9682e49c70a7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-dqsdk\" (UID: \"5609b247-7467-4b4b-81e1-9682e49c70a7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-dqsdk" May 29 17:04:11.146503 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:11.146477 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2q7x\" (UniqueName: \"kubernetes.io/projected/5609b247-7467-4b4b-81e1-9682e49c70a7-kube-api-access-l2q7x\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-dqsdk\" (UID: \"5609b247-7467-4b4b-81e1-9682e49c70a7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-dqsdk" May 29 17:04:11.230766 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:11.230696 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-dqsdk" May 29 17:04:11.361463 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:11.361423 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-dqsdk"] May 29 17:04:11.365325 ip-10-0-141-197 kubenswrapper[2549]: W0529 17:04:11.365291 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5609b247_7467_4b4b_81e1_9682e49c70a7.slice/crio-db9aa1d7c047b4728c64476d2b6f1dbcd9dd8bf2d2ad72594f02960d008cc358 WatchSource:0}: Error finding container db9aa1d7c047b4728c64476d2b6f1dbcd9dd8bf2d2ad72594f02960d008cc358: Status 404 returned error can't find the container with id db9aa1d7c047b4728c64476d2b6f1dbcd9dd8bf2d2ad72594f02960d008cc358 May 29 17:04:11.497221 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:11.497146 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-dqsdk" event={"ID":"5609b247-7467-4b4b-81e1-9682e49c70a7","Type":"ContainerStarted","Data":"db9aa1d7c047b4728c64476d2b6f1dbcd9dd8bf2d2ad72594f02960d008cc358"} May 29 17:04:11.588254 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:11.588186 2549 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-dqsdk"] May 29 17:04:11.604155 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:11.604127 2549 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-dqsdk"] May 29 17:04:26.549941 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:26.549900 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-4xnfw" event={"ID":"61542e3a-fe67-47f8-8bd5-adb7b9b93def","Type":"ContainerStarted","Data":"2c64f032f2f234b7bca5cdeaf11cbbff002fee3d5c60c8cd81743c7b183cb63e"} May 29 17:04:26.568498 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:26.567856 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-4xnfw" podStartSLOduration=2.023498294 podStartE2EDuration="25.567825853s" podCreationTimestamp="2026-05-29 17:04:01 +0000 UTC" firstStartedPulling="2026-05-29 17:04:02.374701574 +0000 UTC m=+432.861130757" lastFinishedPulling="2026-05-29 17:04:25.919029136 +0000 UTC m=+456.405458316" observedRunningTime="2026-05-29 17:04:26.567493711 +0000 UTC m=+457.053922914" watchObservedRunningTime="2026-05-29 17:04:26.567825853 +0000 UTC m=+457.054255061" May 29 17:04:29.562023 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:29.561981 2549 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-dqsdk" podUID="5609b247-7467-4b4b-81e1-9682e49c70a7" containerName="manager" containerID="cri-o://c1a58e44d73a17b772664857046feaa96c168b96cfade13af583b7db21bc75a0" gracePeriod=2 May 29 17:04:29.563801 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:29.563765 2549 status_manager.go:895] "Failed to get status for pod" podUID="5609b247-7467-4b4b-81e1-9682e49c70a7" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-dqsdk" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-dqsdk\" is forbidden: User \"system:node:ip-10-0-141-197.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-197.ec2.internal' and this object" May 29 17:04:29.787268 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:29.787244 2549 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-dqsdk" May 29 17:04:29.788982 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:29.788951 2549 status_manager.go:895] "Failed to get status for pod" podUID="5609b247-7467-4b4b-81e1-9682e49c70a7" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-dqsdk" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-dqsdk\" is forbidden: User \"system:node:ip-10-0-141-197.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-197.ec2.internal' and this object" May 29 17:04:29.905688 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:29.905605 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5609b247-7467-4b4b-81e1-9682e49c70a7-extensions-socket-volume\") pod \"5609b247-7467-4b4b-81e1-9682e49c70a7\" (UID: \"5609b247-7467-4b4b-81e1-9682e49c70a7\") " May 29 17:04:29.905688 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:29.905672 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l2q7x\" (UniqueName: \"kubernetes.io/projected/5609b247-7467-4b4b-81e1-9682e49c70a7-kube-api-access-l2q7x\") pod \"5609b247-7467-4b4b-81e1-9682e49c70a7\" (UID: \"5609b247-7467-4b4b-81e1-9682e49c70a7\") " May 29 17:04:29.905890 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:29.905867 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5609b247-7467-4b4b-81e1-9682e49c70a7-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5609b247-7467-4b4b-81e1-9682e49c70a7" (UID: "5609b247-7467-4b4b-81e1-9682e49c70a7"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 29 17:04:29.907805 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:29.907779 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5609b247-7467-4b4b-81e1-9682e49c70a7-kube-api-access-l2q7x" (OuterVolumeSpecName: "kube-api-access-l2q7x") pod "5609b247-7467-4b4b-81e1-9682e49c70a7" (UID: "5609b247-7467-4b4b-81e1-9682e49c70a7"). InnerVolumeSpecName "kube-api-access-l2q7x". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 29 17:04:30.006161 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:30.006126 2549 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5609b247-7467-4b4b-81e1-9682e49c70a7-extensions-socket-volume\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 17:04:30.006161 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:30.006154 2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2q7x\" (UniqueName: \"kubernetes.io/projected/5609b247-7467-4b4b-81e1-9682e49c70a7-kube-api-access-l2q7x\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 17:04:30.083791 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:30.083747 2549 status_manager.go:895] "Failed to get status for pod" podUID="5609b247-7467-4b4b-81e1-9682e49c70a7" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-dqsdk" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-dqsdk\" is forbidden: User \"system:node:ip-10-0-141-197.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-197.ec2.internal' and this object" May 29 17:04:30.084487 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:30.084467 2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5609b247-7467-4b4b-81e1-9682e49c70a7" path="/var/lib/kubelet/pods/5609b247-7467-4b4b-81e1-9682e49c70a7/volumes" May 29 17:04:30.566040 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:30.566004 2549 generic.go:358] "Generic (PLEG): container finished" podID="5609b247-7467-4b4b-81e1-9682e49c70a7" containerID="c1a58e44d73a17b772664857046feaa96c168b96cfade13af583b7db21bc75a0" exitCode=2 May 29 17:04:30.566469 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:30.566053 2549 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-dqsdk" May 29 17:04:30.566469 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:30.566069 2549 scope.go:117] "RemoveContainer" containerID="c1a58e44d73a17b772664857046feaa96c168b96cfade13af583b7db21bc75a0" May 29 17:04:30.567952 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:30.567913 2549 status_manager.go:895] "Failed to get status for pod" podUID="5609b247-7467-4b4b-81e1-9682e49c70a7" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-dqsdk" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-dqsdk\" is forbidden: User \"system:node:ip-10-0-141-197.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-197.ec2.internal' and this object" May 29 17:04:30.569871 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:30.569831 2549 status_manager.go:895] "Failed to get status for pod" podUID="5609b247-7467-4b4b-81e1-9682e49c70a7" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-dqsdk" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-dqsdk\" is forbidden: User \"system:node:ip-10-0-141-197.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-197.ec2.internal' and this object" May 29 17:04:30.573795 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:30.573773 2549 scope.go:117] "RemoveContainer" containerID="c1a58e44d73a17b772664857046feaa96c168b96cfade13af583b7db21bc75a0" May 29 17:04:30.574104 ip-10-0-141-197 kubenswrapper[2549]: E0529 17:04:30.574087 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1a58e44d73a17b772664857046feaa96c168b96cfade13af583b7db21bc75a0\": container with ID starting with c1a58e44d73a17b772664857046feaa96c168b96cfade13af583b7db21bc75a0 not found: ID does not exist" containerID="c1a58e44d73a17b772664857046feaa96c168b96cfade13af583b7db21bc75a0" May 29 17:04:30.574173 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:30.574111 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1a58e44d73a17b772664857046feaa96c168b96cfade13af583b7db21bc75a0"} err="failed to get container status \"c1a58e44d73a17b772664857046feaa96c168b96cfade13af583b7db21bc75a0\": rpc error: code = NotFound desc = could not find container \"c1a58e44d73a17b772664857046feaa96c168b96cfade13af583b7db21bc75a0\": container with ID starting with c1a58e44d73a17b772664857046feaa96c168b96cfade13af583b7db21bc75a0 not found: ID does not exist" May 29 17:04:56.279582 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:56.279545 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-n8q8w"] May 29 17:04:56.280142 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:56.280040 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5609b247-7467-4b4b-81e1-9682e49c70a7" containerName="manager" May 29 17:04:56.280142 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:56.280059 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="5609b247-7467-4b4b-81e1-9682e49c70a7" containerName="manager" May 29 17:04:56.280142 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:56.280128 2549 memory_manager.go:356] "RemoveStaleState removing state" podUID="5609b247-7467-4b4b-81e1-9682e49c70a7" containerName="manager" May 29 17:04:56.387234 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:56.387195 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-n8q8w"] May 29 17:04:56.387234 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:56.387239 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-n8q8w"] May 29 17:04:56.387445 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:56.387353 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-n8q8w" May 29 17:04:56.389330 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:56.389306 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" May 29 17:04:56.538224 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:56.538140 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5f182a53-4087-4b8e-9c75-26bfc8863fef-config-file\") pod \"limitador-limitador-7d549b5b-n8q8w\" (UID: \"5f182a53-4087-4b8e-9c75-26bfc8863fef\") " pod="kuadrant-system/limitador-limitador-7d549b5b-n8q8w" May 29 17:04:56.538224 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:56.538192 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwrmx\" (UniqueName: \"kubernetes.io/projected/5f182a53-4087-4b8e-9c75-26bfc8863fef-kube-api-access-wwrmx\") pod \"limitador-limitador-7d549b5b-n8q8w\" (UID: \"5f182a53-4087-4b8e-9c75-26bfc8863fef\") " pod="kuadrant-system/limitador-limitador-7d549b5b-n8q8w" May 29 17:04:56.639443 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:56.639410 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5f182a53-4087-4b8e-9c75-26bfc8863fef-config-file\") pod \"limitador-limitador-7d549b5b-n8q8w\" (UID: \"5f182a53-4087-4b8e-9c75-26bfc8863fef\") " pod="kuadrant-system/limitador-limitador-7d549b5b-n8q8w" May 29 17:04:56.639627 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:56.639459 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wwrmx\" (UniqueName: \"kubernetes.io/projected/5f182a53-4087-4b8e-9c75-26bfc8863fef-kube-api-access-wwrmx\") pod \"limitador-limitador-7d549b5b-n8q8w\" (UID: \"5f182a53-4087-4b8e-9c75-26bfc8863fef\") " pod="kuadrant-system/limitador-limitador-7d549b5b-n8q8w" May 29 17:04:56.640083 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:56.640057 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5f182a53-4087-4b8e-9c75-26bfc8863fef-config-file\") pod \"limitador-limitador-7d549b5b-n8q8w\" (UID: \"5f182a53-4087-4b8e-9c75-26bfc8863fef\") " pod="kuadrant-system/limitador-limitador-7d549b5b-n8q8w" May 29 17:04:56.646812 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:56.646794 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwrmx\" (UniqueName: \"kubernetes.io/projected/5f182a53-4087-4b8e-9c75-26bfc8863fef-kube-api-access-wwrmx\") pod \"limitador-limitador-7d549b5b-n8q8w\" (UID: \"5f182a53-4087-4b8e-9c75-26bfc8863fef\") " pod="kuadrant-system/limitador-limitador-7d549b5b-n8q8w" May 29 17:04:56.697887 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:56.697861 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-n8q8w" May 29 17:04:56.823074 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:56.822991 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-n8q8w"] May 29 17:04:56.826639 ip-10-0-141-197 kubenswrapper[2549]: W0529 17:04:56.826614 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f182a53_4087_4b8e_9c75_26bfc8863fef.slice/crio-bb25ffabf1da1a75ec9fe725d1820ecb58e1d09a3bdcf82174a244d9cbbe5e1f WatchSource:0}: Error finding container bb25ffabf1da1a75ec9fe725d1820ecb58e1d09a3bdcf82174a244d9cbbe5e1f: Status 404 returned error can't find the container with id bb25ffabf1da1a75ec9fe725d1820ecb58e1d09a3bdcf82174a244d9cbbe5e1f May 29 17:04:57.085406 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:57.085326 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-q6bfm"] May 29 17:04:57.097552 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:57.097525 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-q6bfm"] May 29 17:04:57.097693 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:57.097628 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-q6bfm" May 29 17:04:57.099932 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:57.099908 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-xgshf\"" May 29 17:04:57.246820 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:57.246790 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhqbf\" (UniqueName: \"kubernetes.io/projected/07a7bc8a-42e9-4d40-b302-e0a008944c14-kube-api-access-mhqbf\") pod \"authorino-f99f4b5cd-q6bfm\" (UID: \"07a7bc8a-42e9-4d40-b302-e0a008944c14\") " pod="kuadrant-system/authorino-f99f4b5cd-q6bfm" May 29 17:04:57.347869 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:57.347779 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mhqbf\" (UniqueName: \"kubernetes.io/projected/07a7bc8a-42e9-4d40-b302-e0a008944c14-kube-api-access-mhqbf\") pod \"authorino-f99f4b5cd-q6bfm\" (UID: \"07a7bc8a-42e9-4d40-b302-e0a008944c14\") " pod="kuadrant-system/authorino-f99f4b5cd-q6bfm" May 29 17:04:57.354774 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:57.354752 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhqbf\" (UniqueName: \"kubernetes.io/projected/07a7bc8a-42e9-4d40-b302-e0a008944c14-kube-api-access-mhqbf\") pod \"authorino-f99f4b5cd-q6bfm\" (UID: \"07a7bc8a-42e9-4d40-b302-e0a008944c14\") " pod="kuadrant-system/authorino-f99f4b5cd-q6bfm" May 29 17:04:57.407862 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:57.407818 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-q6bfm" May 29 17:04:57.529336 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:57.529303 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-q6bfm"] May 29 17:04:57.532723 ip-10-0-141-197 kubenswrapper[2549]: W0529 17:04:57.532692 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07a7bc8a_42e9_4d40_b302_e0a008944c14.slice/crio-2bd5de7ac2e914156c6e1d04f8119e985d1f577c61d9a41e5d6729ed71de2cb9 WatchSource:0}: Error finding container 2bd5de7ac2e914156c6e1d04f8119e985d1f577c61d9a41e5d6729ed71de2cb9: Status 404 returned error can't find the container with id 2bd5de7ac2e914156c6e1d04f8119e985d1f577c61d9a41e5d6729ed71de2cb9 May 29 17:04:57.662340 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:57.662267 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-q6bfm" event={"ID":"07a7bc8a-42e9-4d40-b302-e0a008944c14","Type":"ContainerStarted","Data":"2bd5de7ac2e914156c6e1d04f8119e985d1f577c61d9a41e5d6729ed71de2cb9"} May 29 17:04:57.664021 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:04:57.663971 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-n8q8w" event={"ID":"5f182a53-4087-4b8e-9c75-26bfc8863fef","Type":"ContainerStarted","Data":"bb25ffabf1da1a75ec9fe725d1820ecb58e1d09a3bdcf82174a244d9cbbe5e1f"} May 29 17:05:01.125039 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:01.125000 2549 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-q6bfm"] May 29 17:05:01.682642 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:01.682554 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-q6bfm" event={"ID":"07a7bc8a-42e9-4d40-b302-e0a008944c14","Type":"ContainerStarted","Data":"4c556d4c1151cbe53b8533f3cdd11a0c6d72b13d5c750d552862685534907b92"} May 29 17:05:01.682642 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:01.682617 2549 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-f99f4b5cd-q6bfm" podUID="07a7bc8a-42e9-4d40-b302-e0a008944c14" containerName="authorino" containerID="cri-o://4c556d4c1151cbe53b8533f3cdd11a0c6d72b13d5c750d552862685534907b92" gracePeriod=30 May 29 17:05:01.684024 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:01.683997 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-n8q8w" event={"ID":"5f182a53-4087-4b8e-9c75-26bfc8863fef","Type":"ContainerStarted","Data":"3d2a28071f471c7ae74eb2c84dbccea06631bbfbf4404a141f1041ade9363cea"} May 29 17:05:01.684137 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:01.684055 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7d549b5b-n8q8w" May 29 17:05:01.698131 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:01.698089 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-f99f4b5cd-q6bfm" podStartSLOduration=0.847113423 podStartE2EDuration="4.698078166s" podCreationTimestamp="2026-05-29 17:04:57 +0000 UTC" firstStartedPulling="2026-05-29 17:04:57.53448855 +0000 UTC m=+488.020917743" lastFinishedPulling="2026-05-29 17:05:01.385453299 +0000 UTC m=+491.871882486" observedRunningTime="2026-05-29 17:05:01.696644182 +0000 UTC m=+492.183073384" watchObservedRunningTime="2026-05-29 17:05:01.698078166 +0000 UTC m=+492.184507368" May 29 17:05:01.711460 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:01.711406 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7d549b5b-n8q8w" podStartSLOduration=1.154660828 podStartE2EDuration="5.711392805s" podCreationTimestamp="2026-05-29 17:04:56 +0000 UTC" firstStartedPulling="2026-05-29 17:04:56.828722711 +0000 UTC m=+487.315151891" lastFinishedPulling="2026-05-29 17:05:01.385454683 +0000 UTC m=+491.871883868" observedRunningTime="2026-05-29 17:05:01.710711185 +0000 UTC m=+492.197140386" watchObservedRunningTime="2026-05-29 17:05:01.711392805 +0000 UTC m=+492.197822010" May 29 17:05:01.923363 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:01.923340 2549 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-q6bfm" May 29 17:05:01.993484 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:01.993390 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mhqbf\" (UniqueName: \"kubernetes.io/projected/07a7bc8a-42e9-4d40-b302-e0a008944c14-kube-api-access-mhqbf\") pod \"07a7bc8a-42e9-4d40-b302-e0a008944c14\" (UID: \"07a7bc8a-42e9-4d40-b302-e0a008944c14\") " May 29 17:05:01.995414 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:01.995388 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07a7bc8a-42e9-4d40-b302-e0a008944c14-kube-api-access-mhqbf" (OuterVolumeSpecName: "kube-api-access-mhqbf") pod "07a7bc8a-42e9-4d40-b302-e0a008944c14" (UID: "07a7bc8a-42e9-4d40-b302-e0a008944c14"). InnerVolumeSpecName "kube-api-access-mhqbf". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 29 17:05:02.094104 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:02.094069 2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhqbf\" (UniqueName: \"kubernetes.io/projected/07a7bc8a-42e9-4d40-b302-e0a008944c14-kube-api-access-mhqbf\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 17:05:02.688398 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:02.688313 2549 generic.go:358] "Generic (PLEG): container finished" podID="07a7bc8a-42e9-4d40-b302-e0a008944c14" containerID="4c556d4c1151cbe53b8533f3cdd11a0c6d72b13d5c750d552862685534907b92" exitCode=0 May 29 17:05:02.688398 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:02.688366 2549 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-q6bfm" May 29 17:05:02.688901 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:02.688393 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-q6bfm" event={"ID":"07a7bc8a-42e9-4d40-b302-e0a008944c14","Type":"ContainerDied","Data":"4c556d4c1151cbe53b8533f3cdd11a0c6d72b13d5c750d552862685534907b92"} May 29 17:05:02.688901 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:02.688426 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-q6bfm" event={"ID":"07a7bc8a-42e9-4d40-b302-e0a008944c14","Type":"ContainerDied","Data":"2bd5de7ac2e914156c6e1d04f8119e985d1f577c61d9a41e5d6729ed71de2cb9"} May 29 17:05:02.688901 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:02.688443 2549 scope.go:117] "RemoveContainer" containerID="4c556d4c1151cbe53b8533f3cdd11a0c6d72b13d5c750d552862685534907b92" May 29 17:05:02.695822 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:02.695805 2549 scope.go:117] "RemoveContainer" containerID="4c556d4c1151cbe53b8533f3cdd11a0c6d72b13d5c750d552862685534907b92" May 29 17:05:02.696080 ip-10-0-141-197 kubenswrapper[2549]: E0529 17:05:02.696061 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c556d4c1151cbe53b8533f3cdd11a0c6d72b13d5c750d552862685534907b92\": container with ID starting with 4c556d4c1151cbe53b8533f3cdd11a0c6d72b13d5c750d552862685534907b92 not found: ID does not exist" containerID="4c556d4c1151cbe53b8533f3cdd11a0c6d72b13d5c750d552862685534907b92" May 29 17:05:02.696147 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:02.696092 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c556d4c1151cbe53b8533f3cdd11a0c6d72b13d5c750d552862685534907b92"} err="failed to get container status \"4c556d4c1151cbe53b8533f3cdd11a0c6d72b13d5c750d552862685534907b92\": rpc error: code = NotFound desc = could not find container \"4c556d4c1151cbe53b8533f3cdd11a0c6d72b13d5c750d552862685534907b92\": container with ID starting with 4c556d4c1151cbe53b8533f3cdd11a0c6d72b13d5c750d552862685534907b92 not found: ID does not exist" May 29 17:05:02.702409 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:02.702387 2549 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-q6bfm"] May 29 17:05:02.705726 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:02.705706 2549 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-q6bfm"] May 29 17:05:04.085093 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:04.085062 2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07a7bc8a-42e9-4d40-b302-e0a008944c14" path="/var/lib/kubelet/pods/07a7bc8a-42e9-4d40-b302-e0a008944c14/volumes" May 29 17:05:11.252920 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:11.252883 2549 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-n8q8w"] May 29 17:05:11.253375 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:11.253181 2549 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-7d549b5b-n8q8w" podUID="5f182a53-4087-4b8e-9c75-26bfc8863fef" containerName="limitador" containerID="cri-o://3d2a28071f471c7ae74eb2c84dbccea06631bbfbf4404a141f1041ade9363cea" gracePeriod=30 May 29 17:05:11.253878 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:11.253832 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7d549b5b-n8q8w" May 29 17:05:11.721091 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:11.721060 2549 generic.go:358] "Generic (PLEG): container finished" podID="5f182a53-4087-4b8e-9c75-26bfc8863fef" containerID="3d2a28071f471c7ae74eb2c84dbccea06631bbfbf4404a141f1041ade9363cea" exitCode=0 May 29 17:05:11.721230 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:11.721085 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-n8q8w" event={"ID":"5f182a53-4087-4b8e-9c75-26bfc8863fef","Type":"ContainerDied","Data":"3d2a28071f471c7ae74eb2c84dbccea06631bbfbf4404a141f1041ade9363cea"} May 29 17:05:11.784694 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:11.784671 2549 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-n8q8w" May 29 17:05:11.876079 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:11.875998 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5f182a53-4087-4b8e-9c75-26bfc8863fef-config-file\") pod \"5f182a53-4087-4b8e-9c75-26bfc8863fef\" (UID: \"5f182a53-4087-4b8e-9c75-26bfc8863fef\") " May 29 17:05:11.876079 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:11.876072 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wwrmx\" (UniqueName: \"kubernetes.io/projected/5f182a53-4087-4b8e-9c75-26bfc8863fef-kube-api-access-wwrmx\") pod \"5f182a53-4087-4b8e-9c75-26bfc8863fef\" (UID: \"5f182a53-4087-4b8e-9c75-26bfc8863fef\") " May 29 17:05:11.876347 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:11.876324 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f182a53-4087-4b8e-9c75-26bfc8863fef-config-file" (OuterVolumeSpecName: "config-file") pod "5f182a53-4087-4b8e-9c75-26bfc8863fef" (UID: "5f182a53-4087-4b8e-9c75-26bfc8863fef"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 29 17:05:11.878085 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:11.878053 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f182a53-4087-4b8e-9c75-26bfc8863fef-kube-api-access-wwrmx" (OuterVolumeSpecName: "kube-api-access-wwrmx") pod "5f182a53-4087-4b8e-9c75-26bfc8863fef" (UID: "5f182a53-4087-4b8e-9c75-26bfc8863fef"). InnerVolumeSpecName "kube-api-access-wwrmx". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 29 17:05:11.977430 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:11.977391 2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wwrmx\" (UniqueName: \"kubernetes.io/projected/5f182a53-4087-4b8e-9c75-26bfc8863fef-kube-api-access-wwrmx\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 17:05:11.977430 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:11.977425 2549 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5f182a53-4087-4b8e-9c75-26bfc8863fef-config-file\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 17:05:12.725586 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:12.725552 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-n8q8w" event={"ID":"5f182a53-4087-4b8e-9c75-26bfc8863fef","Type":"ContainerDied","Data":"bb25ffabf1da1a75ec9fe725d1820ecb58e1d09a3bdcf82174a244d9cbbe5e1f"} May 29 17:05:12.725586 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:12.725581 2549 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-n8q8w" May 29 17:05:12.725586 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:12.725592 2549 scope.go:117] "RemoveContainer" containerID="3d2a28071f471c7ae74eb2c84dbccea06631bbfbf4404a141f1041ade9363cea" May 29 17:05:12.738893 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:12.738871 2549 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-n8q8w"] May 29 17:05:12.744473 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:12.744044 2549 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-n8q8w"] May 29 17:05:14.084730 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:14.084690 2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f182a53-4087-4b8e-9c75-26bfc8863fef" path="/var/lib/kubelet/pods/5f182a53-4087-4b8e-9c75-26bfc8863fef/volumes" May 29 17:05:16.012052 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:16.012018 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-68c4fbbd6f-qt7tp"] May 29 17:05:16.012506 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:16.012488 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07a7bc8a-42e9-4d40-b302-e0a008944c14" containerName="authorino" May 29 17:05:16.012549 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:16.012509 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="07a7bc8a-42e9-4d40-b302-e0a008944c14" containerName="authorino" May 29 17:05:16.012549 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:16.012540 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f182a53-4087-4b8e-9c75-26bfc8863fef" containerName="limitador" May 29 17:05:16.012609 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:16.012549 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f182a53-4087-4b8e-9c75-26bfc8863fef" containerName="limitador" May 29 17:05:16.012640 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:16.012627 2549 memory_manager.go:356] "RemoveStaleState removing state" podUID="07a7bc8a-42e9-4d40-b302-e0a008944c14" containerName="authorino" May 29 17:05:16.012673 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:16.012642 2549 memory_manager.go:356] "RemoveStaleState removing state" podUID="5f182a53-4087-4b8e-9c75-26bfc8863fef" containerName="limitador" May 29 17:05:16.015450 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:16.015421 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-68c4fbbd6f-qt7tp" May 29 17:05:16.017766 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:16.017747 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-nmb7p\"" May 29 17:05:16.017892 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:16.017769 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" May 29 17:05:16.026964 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:16.026941 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-68c4fbbd6f-qt7tp"] May 29 17:05:16.111213 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:16.111182 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b2f2adcc-f87e-4153-aacb-f6a57b9a3ea4-data\") pod \"postgres-68c4fbbd6f-qt7tp\" (UID: \"b2f2adcc-f87e-4153-aacb-f6a57b9a3ea4\") " pod="opendatahub/postgres-68c4fbbd6f-qt7tp" May 29 17:05:16.111361 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:16.111230 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmd2v\" (UniqueName: \"kubernetes.io/projected/b2f2adcc-f87e-4153-aacb-f6a57b9a3ea4-kube-api-access-rmd2v\") pod \"postgres-68c4fbbd6f-qt7tp\" (UID: \"b2f2adcc-f87e-4153-aacb-f6a57b9a3ea4\") " pod="opendatahub/postgres-68c4fbbd6f-qt7tp" May 29 17:05:16.212831 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:16.212792 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b2f2adcc-f87e-4153-aacb-f6a57b9a3ea4-data\") pod \"postgres-68c4fbbd6f-qt7tp\" (UID: \"b2f2adcc-f87e-4153-aacb-f6a57b9a3ea4\") " pod="opendatahub/postgres-68c4fbbd6f-qt7tp" May 29 17:05:16.213035 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:16.212926 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rmd2v\" (UniqueName: \"kubernetes.io/projected/b2f2adcc-f87e-4153-aacb-f6a57b9a3ea4-kube-api-access-rmd2v\") pod \"postgres-68c4fbbd6f-qt7tp\" (UID: \"b2f2adcc-f87e-4153-aacb-f6a57b9a3ea4\") " pod="opendatahub/postgres-68c4fbbd6f-qt7tp" May 29 17:05:16.213280 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:16.213248 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b2f2adcc-f87e-4153-aacb-f6a57b9a3ea4-data\") pod \"postgres-68c4fbbd6f-qt7tp\" (UID: \"b2f2adcc-f87e-4153-aacb-f6a57b9a3ea4\") " pod="opendatahub/postgres-68c4fbbd6f-qt7tp" May 29 17:05:16.221421 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:16.221398 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmd2v\" (UniqueName: \"kubernetes.io/projected/b2f2adcc-f87e-4153-aacb-f6a57b9a3ea4-kube-api-access-rmd2v\") pod \"postgres-68c4fbbd6f-qt7tp\" (UID: \"b2f2adcc-f87e-4153-aacb-f6a57b9a3ea4\") " pod="opendatahub/postgres-68c4fbbd6f-qt7tp" May 29 17:05:16.326042 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:16.326007 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-68c4fbbd6f-qt7tp" May 29 17:05:16.451509 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:16.451475 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-68c4fbbd6f-qt7tp"] May 29 17:05:16.455385 ip-10-0-141-197 kubenswrapper[2549]: W0529 17:05:16.455357 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2f2adcc_f87e_4153_aacb_f6a57b9a3ea4.slice/crio-150f319b566a8d678d9a26a24e2ac4f764b5171dcd7f5a3441df896e4cb7f84a WatchSource:0}: Error finding container 150f319b566a8d678d9a26a24e2ac4f764b5171dcd7f5a3441df896e4cb7f84a: Status 404 returned error can't find the container with id 150f319b566a8d678d9a26a24e2ac4f764b5171dcd7f5a3441df896e4cb7f84a May 29 17:05:16.739961 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:16.739871 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-68c4fbbd6f-qt7tp" event={"ID":"b2f2adcc-f87e-4153-aacb-f6a57b9a3ea4","Type":"ContainerStarted","Data":"150f319b566a8d678d9a26a24e2ac4f764b5171dcd7f5a3441df896e4cb7f84a"} May 29 17:05:18.747378 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:18.747347 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-68c4fbbd6f-qt7tp" event={"ID":"b2f2adcc-f87e-4153-aacb-f6a57b9a3ea4","Type":"ContainerStarted","Data":"d5a176a7709a7c63efcdf3cf75ecb0fd546e9849da7b925e0c8a6eb1579bbecb"} May 29 17:05:18.747661 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:18.747397 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-68c4fbbd6f-qt7tp" May 29 17:05:18.761236 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:18.760677 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/postgres-68c4fbbd6f-qt7tp" podStartSLOduration=1.545960048 podStartE2EDuration="3.760662849s" podCreationTimestamp="2026-05-29 17:05:15 +0000 UTC" firstStartedPulling="2026-05-29 17:05:16.456683987 +0000 UTC m=+506.943113184" lastFinishedPulling="2026-05-29 17:05:18.671386792 +0000 UTC m=+509.157815985" observedRunningTime="2026-05-29 17:05:18.760443025 +0000 UTC m=+509.246872228" watchObservedRunningTime="2026-05-29 17:05:18.760662849 +0000 UTC m=+509.247092051" May 29 17:05:24.783724 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:24.783698 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-68c4fbbd6f-qt7tp" May 29 17:05:33.709210 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:33.709171 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["keycloak-system/keycloak-operator-7864d68cbd-4ft65"] May 29 17:05:33.720592 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:33.720565 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/keycloak-operator-7864d68cbd-4ft65" May 29 17:05:33.723922 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:33.723635 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"keycloak-operator-dockercfg-9ljb2\"" May 29 17:05:33.723922 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:33.723653 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" May 29 17:05:33.723922 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:33.723740 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" May 29 17:05:33.726742 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:33.726195 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/keycloak-operator-7864d68cbd-4ft65"] May 29 17:05:33.858702 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:33.858668 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f42tv\" (UniqueName: \"kubernetes.io/projected/a1d02d92-294f-4d23-be6e-85339840f826-kube-api-access-f42tv\") pod \"keycloak-operator-7864d68cbd-4ft65\" (UID: \"a1d02d92-294f-4d23-be6e-85339840f826\") " pod="keycloak-system/keycloak-operator-7864d68cbd-4ft65" May 29 17:05:33.959422 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:33.959339 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-f42tv\" (UniqueName: \"kubernetes.io/projected/a1d02d92-294f-4d23-be6e-85339840f826-kube-api-access-f42tv\") pod \"keycloak-operator-7864d68cbd-4ft65\" (UID: \"a1d02d92-294f-4d23-be6e-85339840f826\") " pod="keycloak-system/keycloak-operator-7864d68cbd-4ft65" May 29 17:05:33.967487 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:33.967460 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f42tv\" (UniqueName: \"kubernetes.io/projected/a1d02d92-294f-4d23-be6e-85339840f826-kube-api-access-f42tv\") pod \"keycloak-operator-7864d68cbd-4ft65\" (UID: \"a1d02d92-294f-4d23-be6e-85339840f826\") " pod="keycloak-system/keycloak-operator-7864d68cbd-4ft65" May 29 17:05:34.036394 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:34.036355 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/keycloak-operator-7864d68cbd-4ft65" May 29 17:05:34.159829 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:34.159806 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/keycloak-operator-7864d68cbd-4ft65"] May 29 17:05:34.162026 ip-10-0-141-197 kubenswrapper[2549]: W0529 17:05:34.161992 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1d02d92_294f_4d23_be6e_85339840f826.slice/crio-37ce635a2ab5ca5438b7e7de2a6353a48d84410c0e198fcc272ddab2d0a7f758 WatchSource:0}: Error finding container 37ce635a2ab5ca5438b7e7de2a6353a48d84410c0e198fcc272ddab2d0a7f758: Status 404 returned error can't find the container with id 37ce635a2ab5ca5438b7e7de2a6353a48d84410c0e198fcc272ddab2d0a7f758 May 29 17:05:34.799401 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:34.799348 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/keycloak-operator-7864d68cbd-4ft65" event={"ID":"a1d02d92-294f-4d23-be6e-85339840f826","Type":"ContainerStarted","Data":"37ce635a2ab5ca5438b7e7de2a6353a48d84410c0e198fcc272ddab2d0a7f758"} May 29 17:05:39.818701 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:39.818663 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/keycloak-operator-7864d68cbd-4ft65" event={"ID":"a1d02d92-294f-4d23-be6e-85339840f826","Type":"ContainerStarted","Data":"8413ec0766772c4c58adfebee2503c9cdb35ecf48c20eca58879c22091c77c07"} May 29 17:05:39.833606 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:05:39.833521 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keycloak-system/keycloak-operator-7864d68cbd-4ft65" podStartSLOduration=1.378219047 podStartE2EDuration="6.833501603s" podCreationTimestamp="2026-05-29 17:05:33 +0000 UTC" firstStartedPulling="2026-05-29 17:05:34.163312343 +0000 UTC m=+524.649741522" lastFinishedPulling="2026-05-29 17:05:39.618594887 +0000 UTC m=+530.105024078" observedRunningTime="2026-05-29 17:05:39.832021939 +0000 UTC m=+530.318451153" watchObservedRunningTime="2026-05-29 17:05:39.833501603 +0000 UTC m=+530.319930806" May 29 17:06:18.563327 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:18.563289 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8b475cf9f-w2wbb"] May 29 17:06:18.569484 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:18.569462 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-w2wbb" May 29 17:06:18.571314 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:18.571293 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-xgshf\"" May 29 17:06:18.573716 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:18.573693 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-w2wbb"] May 29 17:06:18.656024 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:18.655997 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqk2z\" (UniqueName: \"kubernetes.io/projected/f1fd9f41-c6f9-4458-a72f-c851d2426b02-kube-api-access-pqk2z\") pod \"authorino-8b475cf9f-w2wbb\" (UID: \"f1fd9f41-c6f9-4458-a72f-c851d2426b02\") " pod="kuadrant-system/authorino-8b475cf9f-w2wbb" May 29 17:06:18.722901 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:18.722867 2549 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-w2wbb"] May 29 17:06:18.723091 ip-10-0-141-197 kubenswrapper[2549]: E0529 17:06:18.723070 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-pqk2z], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-8b475cf9f-w2wbb" podUID="f1fd9f41-c6f9-4458-a72f-c851d2426b02" May 29 17:06:18.747967 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:18.747936 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56fdd757f5-w8flk"] May 29 17:06:18.750032 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:18.750016 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56fdd757f5-w8flk" May 29 17:06:18.751755 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:18.751729 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" May 29 17:06:18.756548 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:18.756523 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pqk2z\" (UniqueName: \"kubernetes.io/projected/f1fd9f41-c6f9-4458-a72f-c851d2426b02-kube-api-access-pqk2z\") pod \"authorino-8b475cf9f-w2wbb\" (UID: \"f1fd9f41-c6f9-4458-a72f-c851d2426b02\") " pod="kuadrant-system/authorino-8b475cf9f-w2wbb" May 29 17:06:18.756944 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:18.756827 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56fdd757f5-w8flk"] May 29 17:06:18.764236 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:18.764210 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqk2z\" (UniqueName: \"kubernetes.io/projected/f1fd9f41-c6f9-4458-a72f-c851d2426b02-kube-api-access-pqk2z\") pod \"authorino-8b475cf9f-w2wbb\" (UID: \"f1fd9f41-c6f9-4458-a72f-c851d2426b02\") " pod="kuadrant-system/authorino-8b475cf9f-w2wbb" May 29 17:06:18.825548 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:18.825479 2549 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56fdd757f5-w8flk"] May 29 17:06:18.825709 ip-10-0-141-197 kubenswrapper[2549]: E0529 17:06:18.825692 2549 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-pplr4 tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-56fdd757f5-w8flk" podUID="a4b2dd6a-ff07-4ace-bf95-6a82a8e0933c" May 29 17:06:18.852688 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:18.852662 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7ff4d677dc-kpgc7"] May 29 17:06:18.854858 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:18.854825 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7ff4d677dc-kpgc7" May 29 17:06:18.856998 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:18.856978 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a4b2dd6a-ff07-4ace-bf95-6a82a8e0933c-tls-cert\") pod \"authorino-56fdd757f5-w8flk\" (UID: \"a4b2dd6a-ff07-4ace-bf95-6a82a8e0933c\") " pod="kuadrant-system/authorino-56fdd757f5-w8flk" May 29 17:06:18.857090 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:18.857054 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pplr4\" (UniqueName: \"kubernetes.io/projected/a4b2dd6a-ff07-4ace-bf95-6a82a8e0933c-kube-api-access-pplr4\") pod \"authorino-56fdd757f5-w8flk\" (UID: \"a4b2dd6a-ff07-4ace-bf95-6a82a8e0933c\") " pod="kuadrant-system/authorino-56fdd757f5-w8flk" May 29 17:06:18.862806 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:18.862788 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7ff4d677dc-kpgc7"] May 29 17:06:18.949209 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:18.949184 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56fdd757f5-w8flk" May 29 17:06:18.949333 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:18.949187 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-w2wbb" May 29 17:06:18.953343 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:18.953323 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56fdd757f5-w8flk" May 29 17:06:18.956708 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:18.956691 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-w2wbb" May 29 17:06:18.958070 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:18.958052 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cc8d9d95-eaea-499f-8cc5-3db2c3b3ca8d-tls-cert\") pod \"authorino-7ff4d677dc-kpgc7\" (UID: \"cc8d9d95-eaea-499f-8cc5-3db2c3b3ca8d\") " pod="kuadrant-system/authorino-7ff4d677dc-kpgc7" May 29 17:06:18.958136 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:18.958103 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pplr4\" (UniqueName: \"kubernetes.io/projected/a4b2dd6a-ff07-4ace-bf95-6a82a8e0933c-kube-api-access-pplr4\") pod \"authorino-56fdd757f5-w8flk\" (UID: \"a4b2dd6a-ff07-4ace-bf95-6a82a8e0933c\") " pod="kuadrant-system/authorino-56fdd757f5-w8flk" May 29 17:06:18.958182 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:18.958157 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a4b2dd6a-ff07-4ace-bf95-6a82a8e0933c-tls-cert\") pod \"authorino-56fdd757f5-w8flk\" (UID: \"a4b2dd6a-ff07-4ace-bf95-6a82a8e0933c\") " pod="kuadrant-system/authorino-56fdd757f5-w8flk" May 29 17:06:18.958220 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:18.958194 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltn4s\" (UniqueName: \"kubernetes.io/projected/cc8d9d95-eaea-499f-8cc5-3db2c3b3ca8d-kube-api-access-ltn4s\") pod \"authorino-7ff4d677dc-kpgc7\" (UID: \"cc8d9d95-eaea-499f-8cc5-3db2c3b3ca8d\") " pod="kuadrant-system/authorino-7ff4d677dc-kpgc7" May 29 17:06:18.960420 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:18.960395 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a4b2dd6a-ff07-4ace-bf95-6a82a8e0933c-tls-cert\") pod \"authorino-56fdd757f5-w8flk\" (UID: \"a4b2dd6a-ff07-4ace-bf95-6a82a8e0933c\") " pod="kuadrant-system/authorino-56fdd757f5-w8flk" May 29 17:06:18.965171 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:18.965150 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pplr4\" (UniqueName: \"kubernetes.io/projected/a4b2dd6a-ff07-4ace-bf95-6a82a8e0933c-kube-api-access-pplr4\") pod \"authorino-56fdd757f5-w8flk\" (UID: \"a4b2dd6a-ff07-4ace-bf95-6a82a8e0933c\") " pod="kuadrant-system/authorino-56fdd757f5-w8flk" May 29 17:06:19.059436 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:19.059407 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pqk2z\" (UniqueName: \"kubernetes.io/projected/f1fd9f41-c6f9-4458-a72f-c851d2426b02-kube-api-access-pqk2z\") pod \"f1fd9f41-c6f9-4458-a72f-c851d2426b02\" (UID: \"f1fd9f41-c6f9-4458-a72f-c851d2426b02\") " May 29 17:06:19.059569 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:19.059460 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pplr4\" (UniqueName: \"kubernetes.io/projected/a4b2dd6a-ff07-4ace-bf95-6a82a8e0933c-kube-api-access-pplr4\") pod \"a4b2dd6a-ff07-4ace-bf95-6a82a8e0933c\" (UID: \"a4b2dd6a-ff07-4ace-bf95-6a82a8e0933c\") " May 29 17:06:19.059569 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:19.059485 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a4b2dd6a-ff07-4ace-bf95-6a82a8e0933c-tls-cert\") pod \"a4b2dd6a-ff07-4ace-bf95-6a82a8e0933c\" (UID: \"a4b2dd6a-ff07-4ace-bf95-6a82a8e0933c\") " May 29 17:06:19.059683 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:19.059661 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ltn4s\" (UniqueName: \"kubernetes.io/projected/cc8d9d95-eaea-499f-8cc5-3db2c3b3ca8d-kube-api-access-ltn4s\") pod \"authorino-7ff4d677dc-kpgc7\" (UID: \"cc8d9d95-eaea-499f-8cc5-3db2c3b3ca8d\") " pod="kuadrant-system/authorino-7ff4d677dc-kpgc7" May 29 17:06:19.059742 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:19.059720 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cc8d9d95-eaea-499f-8cc5-3db2c3b3ca8d-tls-cert\") pod \"authorino-7ff4d677dc-kpgc7\" (UID: \"cc8d9d95-eaea-499f-8cc5-3db2c3b3ca8d\") " pod="kuadrant-system/authorino-7ff4d677dc-kpgc7" May 29 17:06:19.061533 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:19.061506 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1fd9f41-c6f9-4458-a72f-c851d2426b02-kube-api-access-pqk2z" (OuterVolumeSpecName: "kube-api-access-pqk2z") pod "f1fd9f41-c6f9-4458-a72f-c851d2426b02" (UID: "f1fd9f41-c6f9-4458-a72f-c851d2426b02"). InnerVolumeSpecName "kube-api-access-pqk2z". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 29 17:06:19.061636 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:19.061507 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4b2dd6a-ff07-4ace-bf95-6a82a8e0933c-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "a4b2dd6a-ff07-4ace-bf95-6a82a8e0933c" (UID: "a4b2dd6a-ff07-4ace-bf95-6a82a8e0933c"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 29 17:06:19.061636 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:19.061584 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4b2dd6a-ff07-4ace-bf95-6a82a8e0933c-kube-api-access-pplr4" (OuterVolumeSpecName: "kube-api-access-pplr4") pod "a4b2dd6a-ff07-4ace-bf95-6a82a8e0933c" (UID: "a4b2dd6a-ff07-4ace-bf95-6a82a8e0933c"). InnerVolumeSpecName "kube-api-access-pplr4". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 29 17:06:19.062152 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:19.062127 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cc8d9d95-eaea-499f-8cc5-3db2c3b3ca8d-tls-cert\") pod \"authorino-7ff4d677dc-kpgc7\" (UID: \"cc8d9d95-eaea-499f-8cc5-3db2c3b3ca8d\") " pod="kuadrant-system/authorino-7ff4d677dc-kpgc7" May 29 17:06:19.066626 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:19.066604 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltn4s\" (UniqueName: \"kubernetes.io/projected/cc8d9d95-eaea-499f-8cc5-3db2c3b3ca8d-kube-api-access-ltn4s\") pod \"authorino-7ff4d677dc-kpgc7\" (UID: \"cc8d9d95-eaea-499f-8cc5-3db2c3b3ca8d\") " pod="kuadrant-system/authorino-7ff4d677dc-kpgc7" May 29 17:06:19.161314 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:19.161216 2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqk2z\" (UniqueName: \"kubernetes.io/projected/f1fd9f41-c6f9-4458-a72f-c851d2426b02-kube-api-access-pqk2z\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 17:06:19.161314 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:19.161252 2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pplr4\" (UniqueName: \"kubernetes.io/projected/a4b2dd6a-ff07-4ace-bf95-6a82a8e0933c-kube-api-access-pplr4\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 17:06:19.161314 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:19.161267 2549 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a4b2dd6a-ff07-4ace-bf95-6a82a8e0933c-tls-cert\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 17:06:19.165001 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:19.164982 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7ff4d677dc-kpgc7" May 29 17:06:19.286069 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:19.286046 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7ff4d677dc-kpgc7"] May 29 17:06:19.287947 ip-10-0-141-197 kubenswrapper[2549]: W0529 17:06:19.287924 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcc8d9d95_eaea_499f_8cc5_3db2c3b3ca8d.slice/crio-62dda056e071cb074a7074e51033b579a78ac0c51380f08bc4db34d5231c760f WatchSource:0}: Error finding container 62dda056e071cb074a7074e51033b579a78ac0c51380f08bc4db34d5231c760f: Status 404 returned error can't find the container with id 62dda056e071cb074a7074e51033b579a78ac0c51380f08bc4db34d5231c760f May 29 17:06:19.954699 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:19.954666 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-w2wbb" May 29 17:06:19.955083 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:19.954691 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7ff4d677dc-kpgc7" event={"ID":"cc8d9d95-eaea-499f-8cc5-3db2c3b3ca8d","Type":"ContainerStarted","Data":"de7667a7009ee6a4395a492fd8882508034e99b264eac5cd3fea3f04714cd0a1"} May 29 17:06:19.955083 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:19.954729 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7ff4d677dc-kpgc7" event={"ID":"cc8d9d95-eaea-499f-8cc5-3db2c3b3ca8d","Type":"ContainerStarted","Data":"62dda056e071cb074a7074e51033b579a78ac0c51380f08bc4db34d5231c760f"} May 29 17:06:19.955083 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:19.954870 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56fdd757f5-w8flk" May 29 17:06:19.969258 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:19.969218 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-7ff4d677dc-kpgc7" podStartSLOduration=1.47064952 podStartE2EDuration="1.969205572s" podCreationTimestamp="2026-05-29 17:06:18 +0000 UTC" firstStartedPulling="2026-05-29 17:06:19.289138412 +0000 UTC m=+569.775567596" lastFinishedPulling="2026-05-29 17:06:19.787694455 +0000 UTC m=+570.274123648" observedRunningTime="2026-05-29 17:06:19.967691412 +0000 UTC m=+570.454120672" watchObservedRunningTime="2026-05-29 17:06:19.969205572 +0000 UTC m=+570.455634768" May 29 17:06:20.004036 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:20.004003 2549 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-w2wbb"] May 29 17:06:20.008654 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:20.008630 2549 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-w2wbb"] May 29 17:06:20.024891 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:20.024867 2549 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56fdd757f5-w8flk"] May 29 17:06:20.026711 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:20.026689 2549 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-56fdd757f5-w8flk"] May 29 17:06:20.085142 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:20.085076 2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4b2dd6a-ff07-4ace-bf95-6a82a8e0933c" path="/var/lib/kubelet/pods/a4b2dd6a-ff07-4ace-bf95-6a82a8e0933c/volumes" May 29 17:06:20.085300 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:06:20.085288 2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1fd9f41-c6f9-4458-a72f-c851d2426b02" path="/var/lib/kubelet/pods/f1fd9f41-c6f9-4458-a72f-c851d2426b02/volumes" May 29 17:07:47.796694 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:07:47.796660 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5cf79b98d9-x7m6w"] May 29 17:07:47.800001 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:07:47.799985 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cf79b98d9-x7m6w" May 29 17:07:47.802047 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:07:47.802024 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"authorino-oidc-ca\"" May 29 17:07:47.807101 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:07:47.807022 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5cf79b98d9-x7m6w"] May 29 17:07:47.922738 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:07:47.922706 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f-oidc-ca\") pod \"authorino-5cf79b98d9-x7m6w\" (UID: \"8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f\") " pod="kuadrant-system/authorino-5cf79b98d9-x7m6w" May 29 17:07:47.922738 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:07:47.922744 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxvbw\" (UniqueName: \"kubernetes.io/projected/8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f-kube-api-access-zxvbw\") pod \"authorino-5cf79b98d9-x7m6w\" (UID: \"8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f\") " pod="kuadrant-system/authorino-5cf79b98d9-x7m6w" May 29 17:07:47.922973 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:07:47.922763 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f-tls-cert\") pod \"authorino-5cf79b98d9-x7m6w\" (UID: \"8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f\") " pod="kuadrant-system/authorino-5cf79b98d9-x7m6w" May 29 17:07:48.023414 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:07:48.023380 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f-oidc-ca\") pod \"authorino-5cf79b98d9-x7m6w\" (UID: \"8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f\") " pod="kuadrant-system/authorino-5cf79b98d9-x7m6w" May 29 17:07:48.023414 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:07:48.023421 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zxvbw\" (UniqueName: \"kubernetes.io/projected/8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f-kube-api-access-zxvbw\") pod \"authorino-5cf79b98d9-x7m6w\" (UID: \"8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f\") " pod="kuadrant-system/authorino-5cf79b98d9-x7m6w" May 29 17:07:48.023630 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:07:48.023442 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f-tls-cert\") pod \"authorino-5cf79b98d9-x7m6w\" (UID: \"8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f\") " pod="kuadrant-system/authorino-5cf79b98d9-x7m6w" May 29 17:07:48.024053 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:07:48.024032 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f-oidc-ca\") pod \"authorino-5cf79b98d9-x7m6w\" (UID: \"8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f\") " pod="kuadrant-system/authorino-5cf79b98d9-x7m6w" May 29 17:07:48.025856 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:07:48.025819 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f-tls-cert\") pod \"authorino-5cf79b98d9-x7m6w\" (UID: \"8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f\") " pod="kuadrant-system/authorino-5cf79b98d9-x7m6w" May 29 17:07:48.030521 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:07:48.030497 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxvbw\" (UniqueName: \"kubernetes.io/projected/8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f-kube-api-access-zxvbw\") pod \"authorino-5cf79b98d9-x7m6w\" (UID: \"8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f\") " pod="kuadrant-system/authorino-5cf79b98d9-x7m6w" May 29 17:07:48.110359 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:07:48.110267 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cf79b98d9-x7m6w" May 29 17:07:48.230387 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:07:48.230357 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5cf79b98d9-x7m6w"] May 29 17:07:48.232946 ip-10-0-141-197 kubenswrapper[2549]: W0529 17:07:48.232914 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8c8bbd1e_dd5a_4d8f_92d5_37576b89e65f.slice/crio-b10ee452a7ba3fa869ce1c4baab7f9b2cdc59dee848341a63f97246122a0fb85 WatchSource:0}: Error finding container b10ee452a7ba3fa869ce1c4baab7f9b2cdc59dee848341a63f97246122a0fb85: Status 404 returned error can't find the container with id b10ee452a7ba3fa869ce1c4baab7f9b2cdc59dee848341a63f97246122a0fb85 May 29 17:07:48.234126 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:07:48.234108 2549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 29 17:07:48.249362 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:07:48.249328 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5cf79b98d9-x7m6w" event={"ID":"8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f","Type":"ContainerStarted","Data":"b10ee452a7ba3fa869ce1c4baab7f9b2cdc59dee848341a63f97246122a0fb85"} May 29 17:07:49.254785 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:07:49.254742 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5cf79b98d9-x7m6w" event={"ID":"8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f","Type":"ContainerStarted","Data":"8e58c608e1acce0d58f6181e1134cb1105967ed909c4a1bee14b822b270c1b34"} May 29 17:07:49.269601 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:07:49.269533 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-5cf79b98d9-x7m6w" podStartSLOduration=1.8945304090000001 podStartE2EDuration="2.269514285s" podCreationTimestamp="2026-05-29 17:07:47 +0000 UTC" firstStartedPulling="2026-05-29 17:07:48.234227597 +0000 UTC m=+658.720656778" lastFinishedPulling="2026-05-29 17:07:48.609211475 +0000 UTC m=+659.095640654" observedRunningTime="2026-05-29 17:07:49.26902423 +0000 UTC m=+659.755453438" watchObservedRunningTime="2026-05-29 17:07:49.269514285 +0000 UTC m=+659.755943488" May 29 17:07:49.297540 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:07:49.297504 2549 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7ff4d677dc-kpgc7"] May 29 17:07:49.297795 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:07:49.297767 2549 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-7ff4d677dc-kpgc7" podUID="cc8d9d95-eaea-499f-8cc5-3db2c3b3ca8d" containerName="authorino" containerID="cri-o://de7667a7009ee6a4395a492fd8882508034e99b264eac5cd3fea3f04714cd0a1" gracePeriod=30 May 29 17:07:49.552874 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:07:49.552833 2549 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7ff4d677dc-kpgc7" May 29 17:07:49.638235 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:07:49.638200 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cc8d9d95-eaea-499f-8cc5-3db2c3b3ca8d-tls-cert\") pod \"cc8d9d95-eaea-499f-8cc5-3db2c3b3ca8d\" (UID: \"cc8d9d95-eaea-499f-8cc5-3db2c3b3ca8d\") " May 29 17:07:49.638403 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:07:49.638272 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ltn4s\" (UniqueName: \"kubernetes.io/projected/cc8d9d95-eaea-499f-8cc5-3db2c3b3ca8d-kube-api-access-ltn4s\") pod \"cc8d9d95-eaea-499f-8cc5-3db2c3b3ca8d\" (UID: \"cc8d9d95-eaea-499f-8cc5-3db2c3b3ca8d\") " May 29 17:07:49.640293 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:07:49.640258 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc8d9d95-eaea-499f-8cc5-3db2c3b3ca8d-kube-api-access-ltn4s" (OuterVolumeSpecName: "kube-api-access-ltn4s") pod "cc8d9d95-eaea-499f-8cc5-3db2c3b3ca8d" (UID: "cc8d9d95-eaea-499f-8cc5-3db2c3b3ca8d"). InnerVolumeSpecName "kube-api-access-ltn4s". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 29 17:07:49.648029 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:07:49.648003 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc8d9d95-eaea-499f-8cc5-3db2c3b3ca8d-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "cc8d9d95-eaea-499f-8cc5-3db2c3b3ca8d" (UID: "cc8d9d95-eaea-499f-8cc5-3db2c3b3ca8d"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 29 17:07:49.739418 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:07:49.739386 2549 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cc8d9d95-eaea-499f-8cc5-3db2c3b3ca8d-tls-cert\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 17:07:49.739418 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:07:49.739416 2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ltn4s\" (UniqueName: \"kubernetes.io/projected/cc8d9d95-eaea-499f-8cc5-3db2c3b3ca8d-kube-api-access-ltn4s\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 17:07:50.259307 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:07:50.259270 2549 generic.go:358] "Generic (PLEG): container finished" podID="cc8d9d95-eaea-499f-8cc5-3db2c3b3ca8d" containerID="de7667a7009ee6a4395a492fd8882508034e99b264eac5cd3fea3f04714cd0a1" exitCode=0 May 29 17:07:50.259718 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:07:50.259317 2549 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7ff4d677dc-kpgc7" May 29 17:07:50.259718 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:07:50.259355 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7ff4d677dc-kpgc7" event={"ID":"cc8d9d95-eaea-499f-8cc5-3db2c3b3ca8d","Type":"ContainerDied","Data":"de7667a7009ee6a4395a492fd8882508034e99b264eac5cd3fea3f04714cd0a1"} May 29 17:07:50.259718 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:07:50.259389 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7ff4d677dc-kpgc7" event={"ID":"cc8d9d95-eaea-499f-8cc5-3db2c3b3ca8d","Type":"ContainerDied","Data":"62dda056e071cb074a7074e51033b579a78ac0c51380f08bc4db34d5231c760f"} May 29 17:07:50.259718 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:07:50.259405 2549 scope.go:117] "RemoveContainer" containerID="de7667a7009ee6a4395a492fd8882508034e99b264eac5cd3fea3f04714cd0a1" May 29 17:07:50.267426 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:07:50.267408 2549 scope.go:117] "RemoveContainer" containerID="de7667a7009ee6a4395a492fd8882508034e99b264eac5cd3fea3f04714cd0a1" May 29 17:07:50.267664 ip-10-0-141-197 kubenswrapper[2549]: E0529 17:07:50.267648 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de7667a7009ee6a4395a492fd8882508034e99b264eac5cd3fea3f04714cd0a1\": container with ID starting with de7667a7009ee6a4395a492fd8882508034e99b264eac5cd3fea3f04714cd0a1 not found: ID does not exist" containerID="de7667a7009ee6a4395a492fd8882508034e99b264eac5cd3fea3f04714cd0a1" May 29 17:07:50.267708 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:07:50.267674 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de7667a7009ee6a4395a492fd8882508034e99b264eac5cd3fea3f04714cd0a1"} err="failed to get container status \"de7667a7009ee6a4395a492fd8882508034e99b264eac5cd3fea3f04714cd0a1\": rpc error: code = NotFound desc = could not find container \"de7667a7009ee6a4395a492fd8882508034e99b264eac5cd3fea3f04714cd0a1\": container with ID starting with de7667a7009ee6a4395a492fd8882508034e99b264eac5cd3fea3f04714cd0a1 not found: ID does not exist" May 29 17:07:50.285712 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:07:50.285647 2549 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7ff4d677dc-kpgc7"] May 29 17:07:50.292717 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:07:50.292684 2549 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7ff4d677dc-kpgc7"] May 29 17:07:52.084738 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:07:52.084693 2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc8d9d95-eaea-499f-8cc5-3db2c3b3ca8d" path="/var/lib/kubelet/pods/cc8d9d95-eaea-499f-8cc5-3db2c3b3ca8d/volumes" May 29 17:10:16.462052 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:10:16.461974 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-867d7dffb-6mfvd"] May 29 17:10:16.462476 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:10:16.462296 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc8d9d95-eaea-499f-8cc5-3db2c3b3ca8d" containerName="authorino" May 29 17:10:16.462476 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:10:16.462306 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc8d9d95-eaea-499f-8cc5-3db2c3b3ca8d" containerName="authorino" May 29 17:10:16.462476 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:10:16.462362 2549 memory_manager.go:356] "RemoveStaleState removing state" podUID="cc8d9d95-eaea-499f-8cc5-3db2c3b3ca8d" containerName="authorino" May 29 17:10:16.465252 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:10:16.465235 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-867d7dffb-6mfvd" May 29 17:10:16.472434 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:10:16.472412 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-867d7dffb-6mfvd"] May 29 17:10:16.621533 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:10:16.621497 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2l4nt\" (UniqueName: \"kubernetes.io/projected/164aed9a-1976-4ec0-b6d1-c3b16f8631fa-kube-api-access-2l4nt\") pod \"authorino-867d7dffb-6mfvd\" (UID: \"164aed9a-1976-4ec0-b6d1-c3b16f8631fa\") " pod="kuadrant-system/authorino-867d7dffb-6mfvd" May 29 17:10:16.621533 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:10:16.621541 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/164aed9a-1976-4ec0-b6d1-c3b16f8631fa-oidc-ca\") pod \"authorino-867d7dffb-6mfvd\" (UID: \"164aed9a-1976-4ec0-b6d1-c3b16f8631fa\") " pod="kuadrant-system/authorino-867d7dffb-6mfvd" May 29 17:10:16.621727 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:10:16.621574 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/164aed9a-1976-4ec0-b6d1-c3b16f8631fa-tls-cert\") pod \"authorino-867d7dffb-6mfvd\" (UID: \"164aed9a-1976-4ec0-b6d1-c3b16f8631fa\") " pod="kuadrant-system/authorino-867d7dffb-6mfvd" May 29 17:10:16.722882 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:10:16.722791 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2l4nt\" (UniqueName: \"kubernetes.io/projected/164aed9a-1976-4ec0-b6d1-c3b16f8631fa-kube-api-access-2l4nt\") pod \"authorino-867d7dffb-6mfvd\" (UID: \"164aed9a-1976-4ec0-b6d1-c3b16f8631fa\") " pod="kuadrant-system/authorino-867d7dffb-6mfvd" May 29 17:10:16.722882 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:10:16.722834 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/164aed9a-1976-4ec0-b6d1-c3b16f8631fa-oidc-ca\") pod \"authorino-867d7dffb-6mfvd\" (UID: \"164aed9a-1976-4ec0-b6d1-c3b16f8631fa\") " pod="kuadrant-system/authorino-867d7dffb-6mfvd" May 29 17:10:16.723048 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:10:16.722901 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/164aed9a-1976-4ec0-b6d1-c3b16f8631fa-tls-cert\") pod \"authorino-867d7dffb-6mfvd\" (UID: \"164aed9a-1976-4ec0-b6d1-c3b16f8631fa\") " pod="kuadrant-system/authorino-867d7dffb-6mfvd" May 29 17:10:16.723474 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:10:16.723455 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/164aed9a-1976-4ec0-b6d1-c3b16f8631fa-oidc-ca\") pod \"authorino-867d7dffb-6mfvd\" (UID: \"164aed9a-1976-4ec0-b6d1-c3b16f8631fa\") " pod="kuadrant-system/authorino-867d7dffb-6mfvd" May 29 17:10:16.725360 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:10:16.725340 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/164aed9a-1976-4ec0-b6d1-c3b16f8631fa-tls-cert\") pod \"authorino-867d7dffb-6mfvd\" (UID: \"164aed9a-1976-4ec0-b6d1-c3b16f8631fa\") " pod="kuadrant-system/authorino-867d7dffb-6mfvd" May 29 17:10:16.729415 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:10:16.729392 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l4nt\" (UniqueName: \"kubernetes.io/projected/164aed9a-1976-4ec0-b6d1-c3b16f8631fa-kube-api-access-2l4nt\") pod \"authorino-867d7dffb-6mfvd\" (UID: \"164aed9a-1976-4ec0-b6d1-c3b16f8631fa\") " pod="kuadrant-system/authorino-867d7dffb-6mfvd" May 29 17:10:16.774755 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:10:16.774716 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-867d7dffb-6mfvd" May 29 17:10:16.899140 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:10:16.899106 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-867d7dffb-6mfvd"] May 29 17:10:16.902412 ip-10-0-141-197 kubenswrapper[2549]: W0529 17:10:16.902364 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod164aed9a_1976_4ec0_b6d1_c3b16f8631fa.slice/crio-d4d964f8c23b381c348ca2c03a713776596ad26da773d3e59ff2fbfa9164cc25 WatchSource:0}: Error finding container d4d964f8c23b381c348ca2c03a713776596ad26da773d3e59ff2fbfa9164cc25: Status 404 returned error can't find the container with id d4d964f8c23b381c348ca2c03a713776596ad26da773d3e59ff2fbfa9164cc25 May 29 17:10:17.735963 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:10:17.735878 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-867d7dffb-6mfvd" event={"ID":"164aed9a-1976-4ec0-b6d1-c3b16f8631fa","Type":"ContainerStarted","Data":"576b257c021c1e7e02058f63bf832753037067f6f3c8fe364747b46c75c6e420"} May 29 17:10:17.736468 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:10:17.736428 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-867d7dffb-6mfvd" event={"ID":"164aed9a-1976-4ec0-b6d1-c3b16f8631fa","Type":"ContainerStarted","Data":"d4d964f8c23b381c348ca2c03a713776596ad26da773d3e59ff2fbfa9164cc25"} May 29 17:10:17.755629 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:10:17.755572 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-867d7dffb-6mfvd" podStartSLOduration=1.289472961 podStartE2EDuration="1.75555458s" podCreationTimestamp="2026-05-29 17:10:16 +0000 UTC" firstStartedPulling="2026-05-29 17:10:16.903708243 +0000 UTC m=+807.390137425" lastFinishedPulling="2026-05-29 17:10:17.369789848 +0000 UTC m=+807.856219044" observedRunningTime="2026-05-29 17:10:17.753509293 +0000 UTC m=+808.239938494" watchObservedRunningTime="2026-05-29 17:10:17.75555458 +0000 UTC m=+808.241983783" May 29 17:10:17.782112 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:10:17.781481 2549 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5cf79b98d9-x7m6w"] May 29 17:10:17.782112 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:10:17.781681 2549 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-5cf79b98d9-x7m6w" podUID="8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f" containerName="authorino" containerID="cri-o://8e58c608e1acce0d58f6181e1134cb1105967ed909c4a1bee14b822b270c1b34" gracePeriod=30 May 29 17:10:18.044779 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:10:18.044757 2549 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cf79b98d9-x7m6w" May 29 17:10:18.134572 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:10:18.134539 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zxvbw\" (UniqueName: \"kubernetes.io/projected/8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f-kube-api-access-zxvbw\") pod \"8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f\" (UID: \"8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f\") " May 29 17:10:18.134731 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:10:18.134607 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f-oidc-ca\") pod \"8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f\" (UID: \"8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f\") " May 29 17:10:18.134731 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:10:18.134660 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f-tls-cert\") pod \"8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f\" (UID: \"8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f\") " May 29 17:10:18.136626 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:10:18.136590 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f-kube-api-access-zxvbw" (OuterVolumeSpecName: "kube-api-access-zxvbw") pod "8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f" (UID: "8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f"). InnerVolumeSpecName "kube-api-access-zxvbw". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 29 17:10:18.140515 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:10:18.140488 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f-oidc-ca" (OuterVolumeSpecName: "oidc-ca") pod "8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f" (UID: "8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f"). InnerVolumeSpecName "oidc-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 29 17:10:18.144107 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:10:18.144085 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f" (UID: "8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 29 17:10:18.235804 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:10:18.235759 2549 reconciler_common.go:299] "Volume detached for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f-oidc-ca\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 17:10:18.235804 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:10:18.235800 2549 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f-tls-cert\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 17:10:18.235804 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:10:18.235811 2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zxvbw\" (UniqueName: \"kubernetes.io/projected/8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f-kube-api-access-zxvbw\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 17:10:18.740463 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:10:18.740435 2549 generic.go:358] "Generic (PLEG): container finished" podID="8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f" containerID="8e58c608e1acce0d58f6181e1134cb1105967ed909c4a1bee14b822b270c1b34" exitCode=0 May 29 17:10:18.740895 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:10:18.740487 2549 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cf79b98d9-x7m6w" May 29 17:10:18.740895 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:10:18.740521 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5cf79b98d9-x7m6w" event={"ID":"8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f","Type":"ContainerDied","Data":"8e58c608e1acce0d58f6181e1134cb1105967ed909c4a1bee14b822b270c1b34"} May 29 17:10:18.740895 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:10:18.740571 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5cf79b98d9-x7m6w" event={"ID":"8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f","Type":"ContainerDied","Data":"b10ee452a7ba3fa869ce1c4baab7f9b2cdc59dee848341a63f97246122a0fb85"} May 29 17:10:18.740895 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:10:18.740596 2549 scope.go:117] "RemoveContainer" containerID="8e58c608e1acce0d58f6181e1134cb1105967ed909c4a1bee14b822b270c1b34" May 29 17:10:18.748605 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:10:18.748590 2549 scope.go:117] "RemoveContainer" containerID="8e58c608e1acce0d58f6181e1134cb1105967ed909c4a1bee14b822b270c1b34" May 29 17:10:18.748920 ip-10-0-141-197 kubenswrapper[2549]: E0529 17:10:18.748902 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e58c608e1acce0d58f6181e1134cb1105967ed909c4a1bee14b822b270c1b34\": container with ID starting with 8e58c608e1acce0d58f6181e1134cb1105967ed909c4a1bee14b822b270c1b34 not found: ID does not exist" containerID="8e58c608e1acce0d58f6181e1134cb1105967ed909c4a1bee14b822b270c1b34" May 29 17:10:18.748991 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:10:18.748931 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e58c608e1acce0d58f6181e1134cb1105967ed909c4a1bee14b822b270c1b34"} err="failed to get container status \"8e58c608e1acce0d58f6181e1134cb1105967ed909c4a1bee14b822b270c1b34\": rpc error: code = NotFound desc = could not find container \"8e58c608e1acce0d58f6181e1134cb1105967ed909c4a1bee14b822b270c1b34\": container with ID starting with 8e58c608e1acce0d58f6181e1134cb1105967ed909c4a1bee14b822b270c1b34 not found: ID does not exist" May 29 17:10:18.758797 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:10:18.758772 2549 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5cf79b98d9-x7m6w"] May 29 17:10:18.765019 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:10:18.764990 2549 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5cf79b98d9-x7m6w"] May 29 17:10:20.084862 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:10:20.084809 2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f" path="/var/lib/kubelet/pods/8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f/volumes" May 29 17:15:00.142600 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:15:00.142570 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29667915-nhfrp"] May 29 17:15:00.144968 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:15:00.142908 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f" containerName="authorino" May 29 17:15:00.144968 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:15:00.142920 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f" containerName="authorino" May 29 17:15:00.144968 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:15:00.142985 2549 memory_manager.go:356] "RemoveStaleState removing state" podUID="8c8bbd1e-dd5a-4d8f-92d5-37576b89e65f" containerName="authorino" May 29 17:15:00.145875 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:15:00.145853 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29667915-nhfrp" May 29 17:15:00.156806 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:15:00.156767 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-8mm95\"" May 29 17:15:00.195281 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:15:00.195250 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29667915-nhfrp"] May 29 17:15:00.233885 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:15:00.233852 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmtcl\" (UniqueName: \"kubernetes.io/projected/61b01b12-f9a9-473a-811c-89ba117617d6-kube-api-access-dmtcl\") pod \"maas-api-key-cleanup-29667915-nhfrp\" (UID: \"61b01b12-f9a9-473a-811c-89ba117617d6\") " pod="opendatahub/maas-api-key-cleanup-29667915-nhfrp" May 29 17:15:00.334952 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:15:00.334923 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dmtcl\" (UniqueName: \"kubernetes.io/projected/61b01b12-f9a9-473a-811c-89ba117617d6-kube-api-access-dmtcl\") pod \"maas-api-key-cleanup-29667915-nhfrp\" (UID: \"61b01b12-f9a9-473a-811c-89ba117617d6\") " pod="opendatahub/maas-api-key-cleanup-29667915-nhfrp" May 29 17:15:00.342998 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:15:00.342979 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmtcl\" (UniqueName: \"kubernetes.io/projected/61b01b12-f9a9-473a-811c-89ba117617d6-kube-api-access-dmtcl\") pod \"maas-api-key-cleanup-29667915-nhfrp\" (UID: \"61b01b12-f9a9-473a-811c-89ba117617d6\") " pod="opendatahub/maas-api-key-cleanup-29667915-nhfrp" May 29 17:15:00.456061 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:15:00.455991 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29667915-nhfrp" May 29 17:15:00.577381 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:15:00.577353 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29667915-nhfrp"] May 29 17:15:00.580132 ip-10-0-141-197 kubenswrapper[2549]: W0529 17:15:00.580107 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61b01b12_f9a9_473a_811c_89ba117617d6.slice/crio-861a8e706ad792d260439762b8aca6b175b4e707da054bbd44a5ed23c3e91de2 WatchSource:0}: Error finding container 861a8e706ad792d260439762b8aca6b175b4e707da054bbd44a5ed23c3e91de2: Status 404 returned error can't find the container with id 861a8e706ad792d260439762b8aca6b175b4e707da054bbd44a5ed23c3e91de2 May 29 17:15:00.582225 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:15:00.582210 2549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 29 17:15:00.659086 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:15:00.659058 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29667915-nhfrp" event={"ID":"61b01b12-f9a9-473a-811c-89ba117617d6","Type":"ContainerStarted","Data":"861a8e706ad792d260439762b8aca6b175b4e707da054bbd44a5ed23c3e91de2"} May 29 17:15:03.673275 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:15:03.673187 2549 generic.go:358] "Generic (PLEG): container finished" podID="61b01b12-f9a9-473a-811c-89ba117617d6" containerID="f06a57ed68a0d862712679ccf85b0471f80098eea0a189c70dc3ddeb52f2e0d7" exitCode=0 May 29 17:15:03.673636 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:15:03.673275 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29667915-nhfrp" event={"ID":"61b01b12-f9a9-473a-811c-89ba117617d6","Type":"ContainerDied","Data":"f06a57ed68a0d862712679ccf85b0471f80098eea0a189c70dc3ddeb52f2e0d7"} May 29 17:15:04.804739 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:15:04.804715 2549 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29667915-nhfrp" May 29 17:15:04.976761 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:15:04.976678 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dmtcl\" (UniqueName: \"kubernetes.io/projected/61b01b12-f9a9-473a-811c-89ba117617d6-kube-api-access-dmtcl\") pod \"61b01b12-f9a9-473a-811c-89ba117617d6\" (UID: \"61b01b12-f9a9-473a-811c-89ba117617d6\") " May 29 17:15:04.978771 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:15:04.978748 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61b01b12-f9a9-473a-811c-89ba117617d6-kube-api-access-dmtcl" (OuterVolumeSpecName: "kube-api-access-dmtcl") pod "61b01b12-f9a9-473a-811c-89ba117617d6" (UID: "61b01b12-f9a9-473a-811c-89ba117617d6"). InnerVolumeSpecName "kube-api-access-dmtcl". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 29 17:15:05.077281 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:15:05.077249 2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmtcl\" (UniqueName: \"kubernetes.io/projected/61b01b12-f9a9-473a-811c-89ba117617d6-kube-api-access-dmtcl\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 17:15:05.680658 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:15:05.680632 2549 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29667915-nhfrp" May 29 17:15:05.680828 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:15:05.680630 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29667915-nhfrp" event={"ID":"61b01b12-f9a9-473a-811c-89ba117617d6","Type":"ContainerDied","Data":"861a8e706ad792d260439762b8aca6b175b4e707da054bbd44a5ed23c3e91de2"} May 29 17:15:05.680828 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:15:05.680735 2549 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="861a8e706ad792d260439762b8aca6b175b4e707da054bbd44a5ed23c3e91de2" May 29 17:30:00.138062 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:30:00.138023 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29667930-v2l8r"] May 29 17:30:00.140176 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:30:00.138361 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61b01b12-f9a9-473a-811c-89ba117617d6" containerName="cleanup" May 29 17:30:00.140176 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:30:00.138376 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="61b01b12-f9a9-473a-811c-89ba117617d6" containerName="cleanup" May 29 17:30:00.140176 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:30:00.138439 2549 memory_manager.go:356] "RemoveStaleState removing state" podUID="61b01b12-f9a9-473a-811c-89ba117617d6" containerName="cleanup" May 29 17:30:00.141063 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:30:00.141044 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29667930-v2l8r" May 29 17:30:00.143019 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:30:00.142998 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-8mm95\"" May 29 17:30:00.154339 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:30:00.154316 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29667930-v2l8r"] May 29 17:30:00.165867 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:30:00.165829 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4r87\" (UniqueName: \"kubernetes.io/projected/416761f5-34cd-4e42-9498-dcb4a95ea516-kube-api-access-j4r87\") pod \"maas-api-key-cleanup-29667930-v2l8r\" (UID: \"416761f5-34cd-4e42-9498-dcb4a95ea516\") " pod="opendatahub/maas-api-key-cleanup-29667930-v2l8r" May 29 17:30:00.266414 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:30:00.266389 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j4r87\" (UniqueName: \"kubernetes.io/projected/416761f5-34cd-4e42-9498-dcb4a95ea516-kube-api-access-j4r87\") pod \"maas-api-key-cleanup-29667930-v2l8r\" (UID: \"416761f5-34cd-4e42-9498-dcb4a95ea516\") " pod="opendatahub/maas-api-key-cleanup-29667930-v2l8r" May 29 17:30:00.274003 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:30:00.273975 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4r87\" (UniqueName: \"kubernetes.io/projected/416761f5-34cd-4e42-9498-dcb4a95ea516-kube-api-access-j4r87\") pod \"maas-api-key-cleanup-29667930-v2l8r\" (UID: \"416761f5-34cd-4e42-9498-dcb4a95ea516\") " pod="opendatahub/maas-api-key-cleanup-29667930-v2l8r" May 29 17:30:00.451937 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:30:00.451861 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29667930-v2l8r" May 29 17:30:00.575074 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:30:00.575042 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29667930-v2l8r"] May 29 17:30:00.578685 ip-10-0-141-197 kubenswrapper[2549]: W0529 17:30:00.578658 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod416761f5_34cd_4e42_9498_dcb4a95ea516.slice/crio-e8a6006afad37c0fac74e4413fa8583f3c01201400449aa193362e2cb6012d4f WatchSource:0}: Error finding container e8a6006afad37c0fac74e4413fa8583f3c01201400449aa193362e2cb6012d4f: Status 404 returned error can't find the container with id e8a6006afad37c0fac74e4413fa8583f3c01201400449aa193362e2cb6012d4f May 29 17:30:00.580313 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:30:00.580298 2549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 29 17:30:01.540984 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:30:01.540947 2549 generic.go:358] "Generic (PLEG): container finished" podID="416761f5-34cd-4e42-9498-dcb4a95ea516" containerID="70f77f95ddfda248a66d5a935f25d9bf1e5d6013a736a4e3f1c1e5e09fa99227" exitCode=0 May 29 17:30:01.541413 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:30:01.541030 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29667930-v2l8r" event={"ID":"416761f5-34cd-4e42-9498-dcb4a95ea516","Type":"ContainerDied","Data":"70f77f95ddfda248a66d5a935f25d9bf1e5d6013a736a4e3f1c1e5e09fa99227"} May 29 17:30:01.541413 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:30:01.541062 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29667930-v2l8r" event={"ID":"416761f5-34cd-4e42-9498-dcb4a95ea516","Type":"ContainerStarted","Data":"e8a6006afad37c0fac74e4413fa8583f3c01201400449aa193362e2cb6012d4f"} May 29 17:30:02.660711 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:30:02.660687 2549 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29667930-v2l8r" May 29 17:30:02.786944 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:30:02.786911 2549 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j4r87\" (UniqueName: \"kubernetes.io/projected/416761f5-34cd-4e42-9498-dcb4a95ea516-kube-api-access-j4r87\") pod \"416761f5-34cd-4e42-9498-dcb4a95ea516\" (UID: \"416761f5-34cd-4e42-9498-dcb4a95ea516\") " May 29 17:30:02.788997 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:30:02.788967 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/416761f5-34cd-4e42-9498-dcb4a95ea516-kube-api-access-j4r87" (OuterVolumeSpecName: "kube-api-access-j4r87") pod "416761f5-34cd-4e42-9498-dcb4a95ea516" (UID: "416761f5-34cd-4e42-9498-dcb4a95ea516"). InnerVolumeSpecName "kube-api-access-j4r87". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 29 17:30:02.887554 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:30:02.887526 2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4r87\" (UniqueName: \"kubernetes.io/projected/416761f5-34cd-4e42-9498-dcb4a95ea516-kube-api-access-j4r87\") on node \"ip-10-0-141-197.ec2.internal\" DevicePath \"\"" May 29 17:30:03.548551 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:30:03.548516 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29667930-v2l8r" event={"ID":"416761f5-34cd-4e42-9498-dcb4a95ea516","Type":"ContainerDied","Data":"e8a6006afad37c0fac74e4413fa8583f3c01201400449aa193362e2cb6012d4f"} May 29 17:30:03.548551 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:30:03.548548 2549 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e8a6006afad37c0fac74e4413fa8583f3c01201400449aa193362e2cb6012d4f" May 29 17:30:03.548754 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:30:03.548573 2549 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29667930-v2l8r" May 29 17:33:20.655307 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:20.655261 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-867d7dffb-6mfvd_164aed9a-1976-4ec0-b6d1-c3b16f8631fa/authorino/0.log" May 29 17:33:25.973357 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:25.973331 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29667915-nhfrp_61b01b12-f9a9-473a-811c-89ba117617d6/cleanup/0.log" May 29 17:33:26.085266 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:26.085240 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29667930-v2l8r_416761f5-34cd-4e42-9498-dcb4a95ea516/cleanup/0.log" May 29 17:33:26.839268 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:26.839239 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-598f575fbf-xfn72_c2f55f24-6370-45fd-8cba-77688e4778ba/copy-manifests/0.log" May 29 17:33:26.845461 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:26.845442 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-598f575fbf-xfn72_c2f55f24-6370-45fd-8cba-77688e4778ba/manager/0.log" May 29 17:33:26.959357 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:26.959318 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-68c4fbbd6f-qt7tp_b2f2adcc-f87e-4153-aacb-f6a57b9a3ea4/postgres/0.log" May 29 17:33:28.306608 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:28.306572 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-867d7dffb-6mfvd_164aed9a-1976-4ec0-b6d1-c3b16f8631fa/authorino/0.log" May 29 17:33:28.671148 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:28.671072 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-6cb54b5c86-4xnfw_61542e3a-fe67-47f8-8bd5-adb7b9b93def/kuadrant-console-plugin/0.log" May 29 17:33:29.886853 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:29.886825 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6fbdcf5ddf-w5vnb_fcaa6078-0d3a-4ae6-b47f-eaf5b897c091/kube-auth-proxy/0.log" May 29 17:33:36.066614 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:36.066578 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7r7t9/must-gather-fmgqv"] May 29 17:33:36.067088 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:36.066921 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="416761f5-34cd-4e42-9498-dcb4a95ea516" containerName="cleanup" May 29 17:33:36.067088 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:36.066932 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="416761f5-34cd-4e42-9498-dcb4a95ea516" containerName="cleanup" May 29 17:33:36.067088 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:36.066986 2549 memory_manager.go:356] "RemoveStaleState removing state" podUID="416761f5-34cd-4e42-9498-dcb4a95ea516" containerName="cleanup" May 29 17:33:36.070143 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:36.070124 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7r7t9/must-gather-fmgqv" May 29 17:33:36.072170 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:36.072149 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7r7t9\"/\"openshift-service-ca.crt\"" May 29 17:33:36.072616 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:36.072589 2549 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-7r7t9\"/\"default-dockercfg-52q9n\"" May 29 17:33:36.072616 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:36.072589 2549 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7r7t9\"/\"kube-root-ca.crt\"" May 29 17:33:36.085610 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:36.085584 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7r7t9/must-gather-fmgqv"] May 29 17:33:36.237379 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:36.237346 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bb7pz\" (UniqueName: \"kubernetes.io/projected/2c941900-961c-4238-9719-d5b741830a6f-kube-api-access-bb7pz\") pod \"must-gather-fmgqv\" (UID: \"2c941900-961c-4238-9719-d5b741830a6f\") " pod="openshift-must-gather-7r7t9/must-gather-fmgqv" May 29 17:33:36.237554 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:36.237424 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2c941900-961c-4238-9719-d5b741830a6f-must-gather-output\") pod \"must-gather-fmgqv\" (UID: \"2c941900-961c-4238-9719-d5b741830a6f\") " pod="openshift-must-gather-7r7t9/must-gather-fmgqv" May 29 17:33:36.338139 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:36.338060 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2c941900-961c-4238-9719-d5b741830a6f-must-gather-output\") pod \"must-gather-fmgqv\" (UID: \"2c941900-961c-4238-9719-d5b741830a6f\") " pod="openshift-must-gather-7r7t9/must-gather-fmgqv" May 29 17:33:36.338139 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:36.338111 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bb7pz\" (UniqueName: \"kubernetes.io/projected/2c941900-961c-4238-9719-d5b741830a6f-kube-api-access-bb7pz\") pod \"must-gather-fmgqv\" (UID: \"2c941900-961c-4238-9719-d5b741830a6f\") " pod="openshift-must-gather-7r7t9/must-gather-fmgqv" May 29 17:33:36.338416 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:36.338394 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2c941900-961c-4238-9719-d5b741830a6f-must-gather-output\") pod \"must-gather-fmgqv\" (UID: \"2c941900-961c-4238-9719-d5b741830a6f\") " pod="openshift-must-gather-7r7t9/must-gather-fmgqv" May 29 17:33:36.345809 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:36.345787 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bb7pz\" (UniqueName: \"kubernetes.io/projected/2c941900-961c-4238-9719-d5b741830a6f-kube-api-access-bb7pz\") pod \"must-gather-fmgqv\" (UID: \"2c941900-961c-4238-9719-d5b741830a6f\") " pod="openshift-must-gather-7r7t9/must-gather-fmgqv" May 29 17:33:36.379828 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:36.379802 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7r7t9/must-gather-fmgqv" May 29 17:33:36.501762 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:36.501722 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7r7t9/must-gather-fmgqv"] May 29 17:33:36.504794 ip-10-0-141-197 kubenswrapper[2549]: W0529 17:33:36.504763 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2c941900_961c_4238_9719_d5b741830a6f.slice/crio-75f3738c0c6e202ef9dae03026bb8bc6d329486eb8e1bce5b4e794e5cb1662d5 WatchSource:0}: Error finding container 75f3738c0c6e202ef9dae03026bb8bc6d329486eb8e1bce5b4e794e5cb1662d5: Status 404 returned error can't find the container with id 75f3738c0c6e202ef9dae03026bb8bc6d329486eb8e1bce5b4e794e5cb1662d5 May 29 17:33:37.240104 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:37.240079 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7r7t9/must-gather-fmgqv" event={"ID":"2c941900-961c-4238-9719-d5b741830a6f","Type":"ContainerStarted","Data":"75f3738c0c6e202ef9dae03026bb8bc6d329486eb8e1bce5b4e794e5cb1662d5"} May 29 17:33:38.245953 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:38.245918 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7r7t9/must-gather-fmgqv" event={"ID":"2c941900-961c-4238-9719-d5b741830a6f","Type":"ContainerStarted","Data":"b90dbebc1349d8c31f8c58b13c5237bd9d512eb2bf720a6a805fc9a22210bda4"} May 29 17:33:38.246335 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:38.245959 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7r7t9/must-gather-fmgqv" event={"ID":"2c941900-961c-4238-9719-d5b741830a6f","Type":"ContainerStarted","Data":"db1ced64a121656b4dedead725a799d13916794cc8467ba3916675cc46d6a9b2"} May 29 17:33:38.260800 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:38.260744 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-7r7t9/must-gather-fmgqv" podStartSLOduration=1.61266467 podStartE2EDuration="2.260730931s" podCreationTimestamp="2026-05-29 17:33:36 +0000 UTC" firstStartedPulling="2026-05-29 17:33:36.506486395 +0000 UTC m=+2206.992915574" lastFinishedPulling="2026-05-29 17:33:37.154552651 +0000 UTC m=+2207.640981835" observedRunningTime="2026-05-29 17:33:38.258710399 +0000 UTC m=+2208.745139600" watchObservedRunningTime="2026-05-29 17:33:38.260730931 +0000 UTC m=+2208.747160132" May 29 17:33:38.954359 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:38.954328 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-dhvms_eb980098-9793-4340-b57d-b0600c17b07b/global-pull-secret-syncer/0.log" May 29 17:33:39.061683 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:39.061650 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-w9jdd_54c6a295-b9b0-4cda-9a11-33a3b53a7dfb/konnectivity-agent/0.log" May 29 17:33:39.147919 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:39.147886 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-197.ec2.internal_62bc43ff86f9d08efd1447f9d66d955f/haproxy/0.log" May 29 17:33:43.716031 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:43.715990 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-867d7dffb-6mfvd_164aed9a-1976-4ec0-b6d1-c3b16f8631fa/authorino/0.log" May 29 17:33:43.790146 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:43.790112 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-6cb54b5c86-4xnfw_61542e3a-fe67-47f8-8bd5-adb7b9b93def/kuadrant-console-plugin/0.log" May 29 17:33:45.428258 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:45.428130 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-jpxcp_67c4e63f-21b5-4b12-8a2c-915a4d64852b/kube-state-metrics/0.log" May 29 17:33:45.457501 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:45.457434 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-jpxcp_67c4e63f-21b5-4b12-8a2c-915a4d64852b/kube-rbac-proxy-main/0.log" May 29 17:33:45.519282 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:45.519246 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-jpxcp_67c4e63f-21b5-4b12-8a2c-915a4d64852b/kube-rbac-proxy-self/0.log" May 29 17:33:45.606357 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:45.606275 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9ddvs_dfd131b4-94e6-4910-8548-de6c489f64ac/node-exporter/0.log" May 29 17:33:45.627411 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:45.627373 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9ddvs_dfd131b4-94e6-4910-8548-de6c489f64ac/kube-rbac-proxy/0.log" May 29 17:33:45.649214 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:45.649184 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9ddvs_dfd131b4-94e6-4910-8548-de6c489f64ac/init-textfile/0.log" May 29 17:33:45.815808 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:45.815780 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-zn8jx_0607511d-a31e-4794-b8b8-6a58179aabd0/kube-rbac-proxy-main/0.log" May 29 17:33:45.836567 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:45.836520 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-zn8jx_0607511d-a31e-4794-b8b8-6a58179aabd0/kube-rbac-proxy-self/0.log" May 29 17:33:45.855569 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:45.855503 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-zn8jx_0607511d-a31e-4794-b8b8-6a58179aabd0/openshift-state-metrics/0.log" May 29 17:33:45.890574 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:45.890496 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_515b776d-6740-4741-80a7-8759378a7e42/prometheus/0.log" May 29 17:33:45.910737 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:45.910703 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_515b776d-6740-4741-80a7-8759378a7e42/config-reloader/0.log" May 29 17:33:45.948686 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:45.948636 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_515b776d-6740-4741-80a7-8759378a7e42/thanos-sidecar/0.log" May 29 17:33:45.969035 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:45.968992 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_515b776d-6740-4741-80a7-8759378a7e42/kube-rbac-proxy-web/0.log" May 29 17:33:45.988196 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:45.988166 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_515b776d-6740-4741-80a7-8759378a7e42/kube-rbac-proxy/0.log" May 29 17:33:46.008047 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:46.008018 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_515b776d-6740-4741-80a7-8759378a7e42/kube-rbac-proxy-thanos/0.log" May 29 17:33:46.034914 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:46.034889 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_515b776d-6740-4741-80a7-8759378a7e42/init-config-reloader/0.log" May 29 17:33:46.209441 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:46.209362 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7594d44679-mkqlc_3686b4d1-9720-4c73-95ce-177c869d1c5f/thanos-query/0.log" May 29 17:33:46.228766 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:46.228738 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7594d44679-mkqlc_3686b4d1-9720-4c73-95ce-177c869d1c5f/kube-rbac-proxy-web/0.log" May 29 17:33:46.248835 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:46.248806 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7594d44679-mkqlc_3686b4d1-9720-4c73-95ce-177c869d1c5f/kube-rbac-proxy/0.log" May 29 17:33:46.268231 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:46.268154 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7594d44679-mkqlc_3686b4d1-9720-4c73-95ce-177c869d1c5f/prom-label-proxy/0.log" May 29 17:33:46.288991 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:46.288947 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7594d44679-mkqlc_3686b4d1-9720-4c73-95ce-177c869d1c5f/kube-rbac-proxy-rules/0.log" May 29 17:33:46.308575 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:46.308527 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7594d44679-mkqlc_3686b4d1-9720-4c73-95ce-177c869d1c5f/kube-rbac-proxy-metrics/0.log" May 29 17:33:47.363217 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:47.363190 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-76c9bb5fdf-n6n9z_8921009e-fb12-4b6b-9bef-9120a210fb34/networking-console-plugin/0.log" May 29 17:33:48.444574 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:48.444544 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-d67978758-jpn66_f259edd2-e076-4195-a908-27bec492bd8e/console/0.log" May 29 17:33:49.638610 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:49.638570 2549 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7r7t9/perf-node-gather-daemonset-7x88b"] May 29 17:33:49.646484 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:49.646455 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7r7t9/perf-node-gather-daemonset-7x88b" May 29 17:33:49.650858 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:49.650811 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7r7t9/perf-node-gather-daemonset-7x88b"] May 29 17:33:49.778572 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:49.778531 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f7699158-77c1-4376-bc96-6c2a40f0f95b-lib-modules\") pod \"perf-node-gather-daemonset-7x88b\" (UID: \"f7699158-77c1-4376-bc96-6c2a40f0f95b\") " pod="openshift-must-gather-7r7t9/perf-node-gather-daemonset-7x88b" May 29 17:33:49.778894 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:49.778871 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vptps\" (UniqueName: \"kubernetes.io/projected/f7699158-77c1-4376-bc96-6c2a40f0f95b-kube-api-access-vptps\") pod \"perf-node-gather-daemonset-7x88b\" (UID: \"f7699158-77c1-4376-bc96-6c2a40f0f95b\") " pod="openshift-must-gather-7r7t9/perf-node-gather-daemonset-7x88b" May 29 17:33:49.779036 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:49.779022 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f7699158-77c1-4376-bc96-6c2a40f0f95b-podres\") pod \"perf-node-gather-daemonset-7x88b\" (UID: \"f7699158-77c1-4376-bc96-6c2a40f0f95b\") " pod="openshift-must-gather-7r7t9/perf-node-gather-daemonset-7x88b" May 29 17:33:49.779154 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:49.779142 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f7699158-77c1-4376-bc96-6c2a40f0f95b-proc\") pod \"perf-node-gather-daemonset-7x88b\" (UID: \"f7699158-77c1-4376-bc96-6c2a40f0f95b\") " pod="openshift-must-gather-7r7t9/perf-node-gather-daemonset-7x88b" May 29 17:33:49.779327 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:49.779313 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f7699158-77c1-4376-bc96-6c2a40f0f95b-sys\") pod \"perf-node-gather-daemonset-7x88b\" (UID: \"f7699158-77c1-4376-bc96-6c2a40f0f95b\") " pod="openshift-must-gather-7r7t9/perf-node-gather-daemonset-7x88b" May 29 17:33:49.880252 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:49.880204 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f7699158-77c1-4376-bc96-6c2a40f0f95b-sys\") pod \"perf-node-gather-daemonset-7x88b\" (UID: \"f7699158-77c1-4376-bc96-6c2a40f0f95b\") " pod="openshift-must-gather-7r7t9/perf-node-gather-daemonset-7x88b" May 29 17:33:49.880427 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:49.880269 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f7699158-77c1-4376-bc96-6c2a40f0f95b-lib-modules\") pod \"perf-node-gather-daemonset-7x88b\" (UID: \"f7699158-77c1-4376-bc96-6c2a40f0f95b\") " pod="openshift-must-gather-7r7t9/perf-node-gather-daemonset-7x88b" May 29 17:33:49.880427 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:49.880324 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vptps\" (UniqueName: \"kubernetes.io/projected/f7699158-77c1-4376-bc96-6c2a40f0f95b-kube-api-access-vptps\") pod \"perf-node-gather-daemonset-7x88b\" (UID: \"f7699158-77c1-4376-bc96-6c2a40f0f95b\") " pod="openshift-must-gather-7r7t9/perf-node-gather-daemonset-7x88b" May 29 17:33:49.880427 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:49.880355 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f7699158-77c1-4376-bc96-6c2a40f0f95b-podres\") pod \"perf-node-gather-daemonset-7x88b\" (UID: \"f7699158-77c1-4376-bc96-6c2a40f0f95b\") " pod="openshift-must-gather-7r7t9/perf-node-gather-daemonset-7x88b" May 29 17:33:49.880427 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:49.880386 2549 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f7699158-77c1-4376-bc96-6c2a40f0f95b-proc\") pod \"perf-node-gather-daemonset-7x88b\" (UID: \"f7699158-77c1-4376-bc96-6c2a40f0f95b\") " pod="openshift-must-gather-7r7t9/perf-node-gather-daemonset-7x88b" May 29 17:33:49.880631 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:49.880481 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f7699158-77c1-4376-bc96-6c2a40f0f95b-proc\") pod \"perf-node-gather-daemonset-7x88b\" (UID: \"f7699158-77c1-4376-bc96-6c2a40f0f95b\") " pod="openshift-must-gather-7r7t9/perf-node-gather-daemonset-7x88b" May 29 17:33:49.880631 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:49.880542 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f7699158-77c1-4376-bc96-6c2a40f0f95b-sys\") pod \"perf-node-gather-daemonset-7x88b\" (UID: \"f7699158-77c1-4376-bc96-6c2a40f0f95b\") " pod="openshift-must-gather-7r7t9/perf-node-gather-daemonset-7x88b" May 29 17:33:49.880730 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:49.880659 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f7699158-77c1-4376-bc96-6c2a40f0f95b-lib-modules\") pod \"perf-node-gather-daemonset-7x88b\" (UID: \"f7699158-77c1-4376-bc96-6c2a40f0f95b\") " pod="openshift-must-gather-7r7t9/perf-node-gather-daemonset-7x88b" May 29 17:33:49.880982 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:49.880952 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f7699158-77c1-4376-bc96-6c2a40f0f95b-podres\") pod \"perf-node-gather-daemonset-7x88b\" (UID: \"f7699158-77c1-4376-bc96-6c2a40f0f95b\") " pod="openshift-must-gather-7r7t9/perf-node-gather-daemonset-7x88b" May 29 17:33:49.887731 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:49.887708 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vptps\" (UniqueName: \"kubernetes.io/projected/f7699158-77c1-4376-bc96-6c2a40f0f95b-kube-api-access-vptps\") pod \"perf-node-gather-daemonset-7x88b\" (UID: \"f7699158-77c1-4376-bc96-6c2a40f0f95b\") " pod="openshift-must-gather-7r7t9/perf-node-gather-daemonset-7x88b" May 29 17:33:49.945565 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:49.945498 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-kzj26_bca913b1-2d98-44c7-906e-bdb4c8450658/dns/0.log" May 29 17:33:49.960488 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:49.960460 2549 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7r7t9/perf-node-gather-daemonset-7x88b" May 29 17:33:49.963502 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:49.963467 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-kzj26_bca913b1-2d98-44c7-906e-bdb4c8450658/kube-rbac-proxy/0.log" May 29 17:33:50.092991 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:50.092964 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-z94bv_5c547c68-54a9-4af7-afb9-494d1a1b6a03/dns-node-resolver/0.log" May 29 17:33:50.104547 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:50.104523 2549 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7r7t9/perf-node-gather-daemonset-7x88b"] May 29 17:33:50.106881 ip-10-0-141-197 kubenswrapper[2549]: W0529 17:33:50.106853 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf7699158_77c1_4376_bc96_6c2a40f0f95b.slice/crio-37133cf8bbcf485cb6267d378b6c51146ddf4e3a4d0c8ebd1e33ac71d645c29e WatchSource:0}: Error finding container 37133cf8bbcf485cb6267d378b6c51146ddf4e3a4d0c8ebd1e33ac71d645c29e: Status 404 returned error can't find the container with id 37133cf8bbcf485cb6267d378b6c51146ddf4e3a4d0c8ebd1e33ac71d645c29e May 29 17:33:50.297579 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:50.297543 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7r7t9/perf-node-gather-daemonset-7x88b" event={"ID":"f7699158-77c1-4376-bc96-6c2a40f0f95b","Type":"ContainerStarted","Data":"b424ab7d862a21d9e04be47ba3d448270300ac701bf086fbce0185693888c827"} May 29 17:33:50.297579 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:50.297582 2549 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7r7t9/perf-node-gather-daemonset-7x88b" event={"ID":"f7699158-77c1-4376-bc96-6c2a40f0f95b","Type":"ContainerStarted","Data":"37133cf8bbcf485cb6267d378b6c51146ddf4e3a4d0c8ebd1e33ac71d645c29e"} May 29 17:33:50.297820 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:50.297605 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-7r7t9/perf-node-gather-daemonset-7x88b" May 29 17:33:50.312139 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:50.312069 2549 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-7r7t9/perf-node-gather-daemonset-7x88b" podStartSLOduration=1.312054627 podStartE2EDuration="1.312054627s" podCreationTimestamp="2026-05-29 17:33:49 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 17:33:50.310464914 +0000 UTC m=+2220.796894116" watchObservedRunningTime="2026-05-29 17:33:50.312054627 +0000 UTC m=+2220.798483830" May 29 17:33:50.574075 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:50.574040 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-56d4bd47c8-2rw2c_34245c5b-a05b-46f7-817f-5da3e5af1e42/registry/0.log" May 29 17:33:50.642858 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:50.642812 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qdvbn_962b0454-8569-4239-b4b3-b88b0fe86570/node-ca/0.log" May 29 17:33:51.552631 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:51.552601 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6fbdcf5ddf-w5vnb_fcaa6078-0d3a-4ae6-b47f-eaf5b897c091/kube-auth-proxy/0.log" May 29 17:33:52.237144 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:52.237112 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wnqbv_17e14531-cc91-412b-9804-9c1c9fe52eb8/serve-healthcheck-canary/0.log" May 29 17:33:52.738281 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:52.738249 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n9wt9_d59b7498-802e-4d3b-85ed-54913e4bc2ea/kube-rbac-proxy/0.log" May 29 17:33:52.758272 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:52.758239 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n9wt9_d59b7498-802e-4d3b-85ed-54913e4bc2ea/exporter/0.log" May 29 17:33:52.778598 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:52.778566 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n9wt9_d59b7498-802e-4d3b-85ed-54913e4bc2ea/extractor/0.log" May 29 17:33:54.945418 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:54.945384 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29667915-nhfrp_61b01b12-f9a9-473a-811c-89ba117617d6/cleanup/0.log" May 29 17:33:54.965915 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:54.965889 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29667930-v2l8r_416761f5-34cd-4e42-9498-dcb4a95ea516/cleanup/0.log" May 29 17:33:55.288588 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:55.288561 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-598f575fbf-xfn72_c2f55f24-6370-45fd-8cba-77688e4778ba/manager/0.log" May 29 17:33:55.306664 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:55.306641 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-598f575fbf-xfn72_c2f55f24-6370-45fd-8cba-77688e4778ba/copy-manifests/0.log" May 29 17:33:55.328861 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:55.328816 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-68c4fbbd6f-qt7tp_b2f2adcc-f87e-4153-aacb-f6a57b9a3ea4/postgres/0.log" May 29 17:33:56.311515 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:56.311486 2549 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-7r7t9/perf-node-gather-daemonset-7x88b" May 29 17:33:56.633875 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:33:56.633768 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-674dbbfb6b-bqv25_65c6a255-c62f-47c2-8a95-af480b16ed10/manager/0.log" May 29 17:34:02.884768 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:34:02.884727 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-4fwqn_e7bc2100-fd8c-4fe2-8ad5-3e9a3fe25101/kube-multus/0.log" May 29 17:34:03.056638 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:34:03.056609 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-52zq6_56335f96-b27f-4a52-beb7-656001cdea57/kube-multus-additional-cni-plugins/0.log" May 29 17:34:03.075807 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:34:03.075775 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-52zq6_56335f96-b27f-4a52-beb7-656001cdea57/egress-router-binary-copy/0.log" May 29 17:34:03.114743 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:34:03.114716 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-52zq6_56335f96-b27f-4a52-beb7-656001cdea57/cni-plugins/0.log" May 29 17:34:03.173865 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:34:03.173770 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-52zq6_56335f96-b27f-4a52-beb7-656001cdea57/bond-cni-plugin/0.log" May 29 17:34:03.231593 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:34:03.231557 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-52zq6_56335f96-b27f-4a52-beb7-656001cdea57/routeoverride-cni/0.log" May 29 17:34:03.259119 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:34:03.259088 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-52zq6_56335f96-b27f-4a52-beb7-656001cdea57/whereabouts-cni-bincopy/0.log" May 29 17:34:03.280738 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:34:03.280711 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-52zq6_56335f96-b27f-4a52-beb7-656001cdea57/whereabouts-cni/0.log" May 29 17:34:03.482772 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:34:03.482700 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ps265_7d457970-4377-4157-9d77-81db07916925/network-metrics-daemon/0.log" May 29 17:34:03.500182 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:34:03.500155 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ps265_7d457970-4377-4157-9d77-81db07916925/kube-rbac-proxy/0.log" May 29 17:34:04.909431 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:34:04.909393 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nfv6s_f6995bff-b11d-42e1-87ea-ecc131cd8810/ovn-controller/0.log" May 29 17:34:04.939478 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:34:04.939433 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nfv6s_f6995bff-b11d-42e1-87ea-ecc131cd8810/ovn-acl-logging/0.log" May 29 17:34:04.960381 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:34:04.960353 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nfv6s_f6995bff-b11d-42e1-87ea-ecc131cd8810/kube-rbac-proxy-node/0.log" May 29 17:34:04.979404 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:34:04.979374 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nfv6s_f6995bff-b11d-42e1-87ea-ecc131cd8810/kube-rbac-proxy-ovn-metrics/0.log" May 29 17:34:04.995496 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:34:04.995469 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nfv6s_f6995bff-b11d-42e1-87ea-ecc131cd8810/northd/0.log" May 29 17:34:05.015097 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:34:05.015064 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nfv6s_f6995bff-b11d-42e1-87ea-ecc131cd8810/nbdb/0.log" May 29 17:34:05.034009 ip-10-0-141-197 kubenswrapper[2549]: I0529 17:34:05.033982 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nfv6s_f6995bff-b11d-42e1-87ea-ecc131cd8810/sbdb/0.log"