Jul 06 01:18:22.758701 ip-10-0-143-159 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 06 01:18:22.758713 ip-10-0-143-159 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 06 01:18:22.758720 ip-10-0-143-159 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 06 01:18:22.758939 ip-10-0-143-159 systemd[1]: Failed to start Kubernetes Kubelet. Jul 06 01:18:32.768250 ip-10-0-143-159 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 06 01:18:32.768266 ip-10-0-143-159 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 1e4775ca71a74430b17efc908fd91954 -- Jul 06 01:20:25.996728 ip-10-0-143-159 systemd[1]: Starting Kubernetes Kubelet... Jul 06 01:20:26.377597 ip-10-0-143-159 kubenswrapper[2565]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 01:20:26.377597 ip-10-0-143-159 kubenswrapper[2565]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 01:20:26.377597 ip-10-0-143-159 kubenswrapper[2565]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 01:20:26.377597 ip-10-0-143-159 kubenswrapper[2565]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 01:20:26.377597 ip-10-0-143-159 kubenswrapper[2565]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 01:20:26.379186 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.379089 2565 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 01:20:26.386056 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386025 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 01:20:26.386056 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386054 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 01:20:26.386056 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386058 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 01:20:26.386056 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386062 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 01:20:26.386190 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386065 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 01:20:26.386190 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386068 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 01:20:26.386190 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386071 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 01:20:26.386190 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386074 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 01:20:26.386190 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386077 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 01:20:26.386190 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386079 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 01:20:26.386190 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386083 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 01:20:26.386190 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386086 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 01:20:26.386190 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386089 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 01:20:26.386190 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386092 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 01:20:26.386190 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386094 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 01:20:26.386190 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386097 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 01:20:26.386190 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386099 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 01:20:26.386190 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386101 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 01:20:26.386190 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386104 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 01:20:26.386190 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386106 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 01:20:26.386190 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386109 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 01:20:26.386190 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386111 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 01:20:26.386190 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386113 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 01:20:26.386190 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386116 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 01:20:26.386686 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386118 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 01:20:26.386686 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386121 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 01:20:26.386686 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386123 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 01:20:26.386686 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386125 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 01:20:26.386686 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386128 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 01:20:26.386686 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386131 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 01:20:26.386686 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386133 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 01:20:26.386686 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386135 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 01:20:26.386686 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386138 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 01:20:26.386686 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386140 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 01:20:26.386686 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386143 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 01:20:26.386686 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386145 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 01:20:26.386686 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386147 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 01:20:26.386686 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386151 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 01:20:26.386686 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386153 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 01:20:26.386686 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386157 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 01:20:26.386686 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386159 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 01:20:26.386686 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386162 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 01:20:26.386686 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386165 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 01:20:26.387158 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386169 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 01:20:26.387158 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386173 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 01:20:26.387158 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386176 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 01:20:26.387158 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386179 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 01:20:26.387158 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386181 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 01:20:26.387158 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386185 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 01:20:26.387158 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386188 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 01:20:26.387158 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386191 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 01:20:26.387158 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386194 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 01:20:26.387158 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386196 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 01:20:26.387158 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386199 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 01:20:26.387158 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386201 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 01:20:26.387158 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386204 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 01:20:26.387158 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386207 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 01:20:26.387158 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386210 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 01:20:26.387158 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386213 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 01:20:26.387158 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386216 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 01:20:26.387158 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386219 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 01:20:26.387158 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386221 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 01:20:26.387158 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386223 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 01:20:26.387662 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386227 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 01:20:26.387662 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386231 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 01:20:26.387662 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386233 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 01:20:26.387662 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386236 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 01:20:26.387662 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386239 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 01:20:26.387662 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386242 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 01:20:26.387662 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386247 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 01:20:26.387662 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386249 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 06 01:20:26.387662 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386260 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 01:20:26.387662 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386262 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 01:20:26.387662 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386265 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 01:20:26.387662 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386267 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 01:20:26.387662 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386270 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 01:20:26.387662 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386272 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 01:20:26.387662 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386274 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 01:20:26.387662 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386277 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 01:20:26.387662 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386279 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 01:20:26.387662 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386282 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 01:20:26.387662 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386284 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 01:20:26.387662 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386287 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 01:20:26.388156 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386289 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 01:20:26.388156 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386292 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 01:20:26.388156 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386295 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 01:20:26.388156 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386297 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 01:20:26.388156 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386300 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 01:20:26.388156 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386302 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 01:20:26.388156 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386306 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 01:20:26.388156 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386310 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 01:20:26.388156 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386312 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 01:20:26.388156 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386315 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 01:20:26.388156 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386318 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 01:20:26.388156 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386320 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 01:20:26.388156 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386337 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 01:20:26.388156 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386340 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 01:20:26.388156 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386344 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 01:20:26.388156 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.386348 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 01:20:26.388156 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387688 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 01:20:26.388156 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387697 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 01:20:26.388156 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387701 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 01:20:26.388630 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387706 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 01:20:26.388630 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387710 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 01:20:26.388630 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387714 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 01:20:26.388630 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387717 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 01:20:26.388630 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387720 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 01:20:26.388630 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387722 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 01:20:26.388630 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387725 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 01:20:26.388630 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387727 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 01:20:26.388630 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387730 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 01:20:26.388630 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387732 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 01:20:26.388630 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387735 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 01:20:26.388630 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387737 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 01:20:26.388630 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387740 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 01:20:26.388630 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387742 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 01:20:26.388630 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387745 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 01:20:26.388630 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387748 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 01:20:26.388630 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387751 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 01:20:26.388630 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387754 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 01:20:26.388630 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387756 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 01:20:26.389161 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387759 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 01:20:26.389161 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387762 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 01:20:26.389161 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387765 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 01:20:26.389161 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387767 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 01:20:26.389161 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387770 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 01:20:26.389161 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387773 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 01:20:26.389161 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387775 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 01:20:26.389161 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387778 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 01:20:26.389161 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387780 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 01:20:26.389161 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387783 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 01:20:26.389161 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387785 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 01:20:26.389161 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387788 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 01:20:26.389161 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387790 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 01:20:26.389161 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387792 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 01:20:26.389161 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387795 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 01:20:26.389161 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387797 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 01:20:26.389161 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387800 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 01:20:26.389161 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387804 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 01:20:26.389161 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387806 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 01:20:26.389161 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387808 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 01:20:26.389686 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387811 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 06 01:20:26.389686 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387814 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 01:20:26.389686 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387816 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 01:20:26.389686 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387819 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 01:20:26.389686 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387821 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 01:20:26.389686 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387823 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 01:20:26.389686 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387826 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 01:20:26.389686 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387830 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 01:20:26.389686 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387832 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 01:20:26.389686 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387835 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 01:20:26.389686 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387838 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 01:20:26.389686 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387842 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 01:20:26.389686 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387845 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 01:20:26.389686 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387848 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 01:20:26.389686 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387851 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 01:20:26.389686 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387853 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 01:20:26.389686 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387856 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 01:20:26.389686 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387859 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 01:20:26.389686 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387862 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 01:20:26.389686 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387864 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 01:20:26.390179 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387867 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 01:20:26.390179 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387871 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 01:20:26.390179 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387873 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 01:20:26.390179 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387876 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 01:20:26.390179 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387878 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 01:20:26.390179 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387881 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 01:20:26.390179 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387883 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 01:20:26.390179 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387886 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 01:20:26.390179 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387888 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 01:20:26.390179 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387891 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 01:20:26.390179 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387893 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 01:20:26.390179 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387896 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 01:20:26.390179 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387898 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 01:20:26.390179 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387900 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 01:20:26.390179 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387903 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 01:20:26.390179 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387905 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 01:20:26.390179 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387908 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 01:20:26.390179 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387910 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 01:20:26.390179 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387913 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 01:20:26.390179 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387916 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 01:20:26.390679 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387919 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 01:20:26.390679 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387922 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 01:20:26.390679 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387924 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 01:20:26.390679 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387927 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 01:20:26.390679 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387929 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 01:20:26.390679 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387932 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 01:20:26.390679 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387934 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 01:20:26.390679 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387937 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 01:20:26.390679 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387939 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 01:20:26.390679 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387942 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 01:20:26.390679 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387946 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 01:20:26.390679 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387949 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 01:20:26.390679 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387952 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 01:20:26.390679 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387954 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 01:20:26.390679 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387956 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 01:20:26.390679 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387959 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 01:20:26.390679 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.387961 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 01:20:26.390679 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388431 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 01:20:26.390679 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388437 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 01:20:26.390679 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388440 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 01:20:26.391168 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388443 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 01:20:26.391168 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388452 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 01:20:26.391168 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388455 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 01:20:26.391168 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388457 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 01:20:26.391168 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388460 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 01:20:26.391168 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388463 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 01:20:26.391168 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388465 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 01:20:26.391168 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388469 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 01:20:26.391168 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388471 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 01:20:26.391168 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388473 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 01:20:26.391168 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388476 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 01:20:26.391168 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388479 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 01:20:26.391168 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388481 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 01:20:26.391168 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388484 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 01:20:26.391168 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388486 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 01:20:26.391168 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388489 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 01:20:26.391168 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388491 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 01:20:26.391168 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388493 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 01:20:26.391168 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388496 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 01:20:26.391168 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388498 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 01:20:26.391676 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388501 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 01:20:26.391676 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388504 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 01:20:26.391676 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388507 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 01:20:26.391676 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388509 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 01:20:26.391676 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388512 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 01:20:26.391676 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388514 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 01:20:26.391676 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388517 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 01:20:26.391676 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388519 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 01:20:26.391676 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388521 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 06 01:20:26.391676 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388524 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 01:20:26.391676 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388526 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 01:20:26.391676 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388528 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 01:20:26.391676 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388531 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 01:20:26.391676 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388533 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 01:20:26.391676 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388536 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 01:20:26.391676 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388538 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 01:20:26.391676 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388541 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 01:20:26.391676 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388543 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 01:20:26.391676 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388545 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 01:20:26.391676 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388548 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 01:20:26.392196 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388551 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 01:20:26.392196 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388555 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 01:20:26.392196 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388559 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 01:20:26.392196 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388562 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 01:20:26.392196 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388565 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 01:20:26.392196 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388568 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 01:20:26.392196 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388570 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 01:20:26.392196 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388573 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 01:20:26.392196 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388576 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 01:20:26.392196 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388579 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 01:20:26.392196 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388581 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 01:20:26.392196 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388584 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 01:20:26.392196 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388587 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 01:20:26.392196 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388590 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 01:20:26.392196 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388592 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 01:20:26.392196 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388595 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 01:20:26.392196 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388599 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 01:20:26.392196 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388602 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 01:20:26.392196 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388605 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 01:20:26.392675 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388607 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 01:20:26.392675 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388610 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 01:20:26.392675 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388613 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 01:20:26.392675 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388615 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 01:20:26.392675 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388635 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 01:20:26.392675 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388639 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 01:20:26.392675 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388642 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 01:20:26.392675 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388645 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 01:20:26.392675 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388649 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 01:20:26.392675 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388652 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 01:20:26.392675 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388655 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 01:20:26.392675 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388657 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 01:20:26.392675 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388660 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 01:20:26.392675 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388663 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 01:20:26.392675 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388666 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 01:20:26.392675 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388668 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 01:20:26.392675 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388671 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 01:20:26.392675 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388673 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 01:20:26.392675 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388676 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 01:20:26.392675 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388678 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 01:20:26.393172 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388681 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 01:20:26.393172 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388684 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 01:20:26.393172 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388686 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 01:20:26.393172 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388689 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 01:20:26.393172 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388692 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 01:20:26.393172 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388695 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 01:20:26.393172 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388698 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 01:20:26.393172 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388700 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 01:20:26.393172 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388703 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 01:20:26.393172 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388705 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 01:20:26.393172 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388708 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 01:20:26.393172 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388710 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 01:20:26.393172 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388713 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 01:20:26.393172 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388715 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 01:20:26.393172 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388717 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 01:20:26.393172 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388720 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 01:20:26.393172 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.388722 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 01:20:26.393172 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.388809 2565 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 01:20:26.393172 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.388820 2565 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 01:20:26.393172 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.388830 2565 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 01:20:26.393718 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.388836 2565 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 01:20:26.393718 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.388846 2565 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 01:20:26.393718 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.388849 2565 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 01:20:26.393718 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.388854 2565 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 01:20:26.393718 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.388858 2565 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 01:20:26.393718 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.388861 2565 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 01:20:26.393718 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.388865 2565 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 01:20:26.393718 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.388869 2565 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 01:20:26.393718 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.388872 2565 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 01:20:26.393718 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.388875 2565 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 01:20:26.393718 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.388878 2565 flags.go:64] FLAG: --cgroup-root="" Jul 06 01:20:26.393718 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.388881 2565 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 01:20:26.393718 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.388884 2565 flags.go:64] FLAG: --client-ca-file="" Jul 06 01:20:26.393718 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.388887 2565 flags.go:64] FLAG: --cloud-provider="external" Jul 06 01:20:26.393718 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.388890 2565 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 01:20:26.393718 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.388898 2565 flags.go:64] FLAG: --cluster-domain="" Jul 06 01:20:26.393718 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.388901 2565 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 01:20:26.393718 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.388904 2565 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 01:20:26.393718 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.388907 2565 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 01:20:26.393718 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.388912 2565 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 01:20:26.393718 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.388916 2565 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 01:20:26.393718 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.388919 2565 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 01:20:26.393718 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.388922 2565 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 01:20:26.394305 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.388925 2565 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 01:20:26.394305 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.388928 2565 flags.go:64] FLAG: --contention-profiling="false" Jul 06 01:20:26.394305 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.388931 2565 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 01:20:26.394305 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.388934 2565 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 01:20:26.394305 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.388937 2565 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 01:20:26.394305 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.388940 2565 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 01:20:26.394305 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.388944 2565 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 01:20:26.394305 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.388947 2565 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 01:20:26.394305 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.388956 2565 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 01:20:26.394305 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.388959 2565 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 01:20:26.394305 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.388962 2565 flags.go:64] FLAG: --enable-server="true" Jul 06 01:20:26.394305 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.388977 2565 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 01:20:26.394305 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.388984 2565 flags.go:64] FLAG: --event-burst="100" Jul 06 01:20:26.394305 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.388990 2565 flags.go:64] FLAG: --event-qps="50" Jul 06 01:20:26.394305 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.388993 2565 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 01:20:26.394305 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.388998 2565 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 01:20:26.394305 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389001 2565 flags.go:64] FLAG: --eviction-hard="" Jul 06 01:20:26.394305 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389004 2565 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 01:20:26.394305 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389007 2565 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 01:20:26.394305 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389010 2565 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 01:20:26.394305 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389013 2565 flags.go:64] FLAG: --eviction-soft="" Jul 06 01:20:26.394305 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389015 2565 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 01:20:26.394305 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389018 2565 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 01:20:26.394305 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389021 2565 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 01:20:26.394305 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389023 2565 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 01:20:26.394971 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389026 2565 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 01:20:26.394971 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389029 2565 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 01:20:26.394971 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389032 2565 flags.go:64] FLAG: --feature-gates="" Jul 06 01:20:26.394971 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389040 2565 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 01:20:26.394971 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389043 2565 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 01:20:26.394971 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389047 2565 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 01:20:26.394971 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389049 2565 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 01:20:26.394971 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389052 2565 flags.go:64] FLAG: --healthz-port="10248" Jul 06 01:20:26.394971 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389055 2565 flags.go:64] FLAG: --help="false" Jul 06 01:20:26.394971 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389058 2565 flags.go:64] FLAG: --hostname-override="ip-10-0-143-159.ec2.internal" Jul 06 01:20:26.394971 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389061 2565 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 01:20:26.394971 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389064 2565 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 01:20:26.394971 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389067 2565 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 01:20:26.394971 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389070 2565 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 01:20:26.394971 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389073 2565 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 01:20:26.394971 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389076 2565 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 01:20:26.394971 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389079 2565 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 01:20:26.394971 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389081 2565 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 01:20:26.394971 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389084 2565 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 01:20:26.394971 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389093 2565 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 01:20:26.394971 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389097 2565 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 01:20:26.394971 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389100 2565 flags.go:64] FLAG: --kube-reserved="" Jul 06 01:20:26.394971 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389104 2565 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 01:20:26.394971 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389107 2565 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 01:20:26.395680 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389110 2565 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 01:20:26.395680 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389113 2565 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 01:20:26.395680 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389115 2565 flags.go:64] FLAG: --lock-file="" Jul 06 01:20:26.395680 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389118 2565 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 01:20:26.395680 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389121 2565 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 01:20:26.395680 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389124 2565 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 01:20:26.395680 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389129 2565 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 01:20:26.395680 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389132 2565 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 01:20:26.395680 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389135 2565 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 01:20:26.395680 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389138 2565 flags.go:64] FLAG: --logging-format="text" Jul 06 01:20:26.395680 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389141 2565 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 01:20:26.395680 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389144 2565 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 01:20:26.395680 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389147 2565 flags.go:64] FLAG: --manifest-url="" Jul 06 01:20:26.395680 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389149 2565 flags.go:64] FLAG: --manifest-url-header="" Jul 06 01:20:26.395680 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389154 2565 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 01:20:26.395680 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389157 2565 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 01:20:26.395680 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389161 2565 flags.go:64] FLAG: --max-pods="110" Jul 06 01:20:26.395680 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389164 2565 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 01:20:26.395680 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389167 2565 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 01:20:26.395680 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389169 2565 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 01:20:26.395680 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389172 2565 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 01:20:26.395680 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389175 2565 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 01:20:26.395680 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389178 2565 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 01:20:26.395680 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389181 2565 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 01:20:26.395680 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389190 2565 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 01:20:26.396322 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389192 2565 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 01:20:26.396322 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389195 2565 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 01:20:26.396322 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389198 2565 flags.go:64] FLAG: --pod-cidr="" Jul 06 01:20:26.396322 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389203 2565 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 01:20:26.396322 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389209 2565 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 01:20:26.396322 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389213 2565 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 01:20:26.396322 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389216 2565 flags.go:64] FLAG: --pods-per-core="0" Jul 06 01:20:26.396322 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389219 2565 flags.go:64] FLAG: --port="10250" Jul 06 01:20:26.396322 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389222 2565 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 01:20:26.396322 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389225 2565 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b8ec16a67b267b1d" Jul 06 01:20:26.396322 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389228 2565 flags.go:64] FLAG: --qos-reserved="" Jul 06 01:20:26.396322 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389231 2565 flags.go:64] FLAG: --read-only-port="10255" Jul 06 01:20:26.396322 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389233 2565 flags.go:64] FLAG: --register-node="true" Jul 06 01:20:26.396322 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389237 2565 flags.go:64] FLAG: --register-with-taints="" Jul 06 01:20:26.396322 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389241 2565 flags.go:64] FLAG: --registry-burst="10" Jul 06 01:20:26.396322 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389244 2565 flags.go:64] FLAG: --registry-qps="5" Jul 06 01:20:26.396322 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389246 2565 flags.go:64] FLAG: --reserved-cpus="" Jul 06 01:20:26.396322 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389249 2565 flags.go:64] FLAG: --reserved-memory="" Jul 06 01:20:26.396322 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389253 2565 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 01:20:26.396322 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389256 2565 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 01:20:26.396322 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389259 2565 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 01:20:26.396322 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389262 2565 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 01:20:26.396322 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389264 2565 flags.go:64] FLAG: --runonce="false" Jul 06 01:20:26.396322 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389267 2565 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 01:20:26.396322 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389270 2565 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 01:20:26.397013 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389273 2565 flags.go:64] FLAG: --seccomp-default="false" Jul 06 01:20:26.397013 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389275 2565 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 01:20:26.397013 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389278 2565 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 01:20:26.397013 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389281 2565 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 01:20:26.397013 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389284 2565 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 01:20:26.397013 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389287 2565 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 01:20:26.397013 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389290 2565 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 01:20:26.397013 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389293 2565 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 01:20:26.397013 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389295 2565 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 01:20:26.397013 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389298 2565 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 01:20:26.397013 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389301 2565 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 01:20:26.397013 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389306 2565 flags.go:64] FLAG: --system-cgroups="" Jul 06 01:20:26.397013 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389310 2565 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 01:20:26.397013 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389316 2565 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 01:20:26.397013 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389319 2565 flags.go:64] FLAG: --tls-cert-file="" Jul 06 01:20:26.397013 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389338 2565 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 01:20:26.397013 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389344 2565 flags.go:64] FLAG: --tls-min-version="" Jul 06 01:20:26.397013 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389347 2565 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 01:20:26.397013 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389350 2565 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 01:20:26.397013 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389353 2565 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 01:20:26.397013 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389356 2565 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 01:20:26.397013 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389359 2565 flags.go:64] FLAG: --v="2" Jul 06 01:20:26.397013 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389363 2565 flags.go:64] FLAG: --version="false" Jul 06 01:20:26.397013 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389367 2565 flags.go:64] FLAG: --vmodule="" Jul 06 01:20:26.397013 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389371 2565 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 01:20:26.397680 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.389374 2565 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 01:20:26.397680 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389505 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 01:20:26.397680 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389508 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 01:20:26.397680 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389511 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 01:20:26.397680 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389515 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 01:20:26.397680 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389518 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 01:20:26.397680 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389520 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 01:20:26.397680 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389523 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 01:20:26.397680 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389526 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 01:20:26.397680 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389529 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 01:20:26.397680 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389531 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 01:20:26.397680 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389534 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 01:20:26.397680 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389536 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 01:20:26.397680 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389539 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 01:20:26.397680 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389542 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 01:20:26.397680 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389544 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 01:20:26.397680 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389546 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 01:20:26.397680 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389549 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 01:20:26.397680 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389553 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 01:20:26.397680 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389558 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 01:20:26.398454 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389560 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 01:20:26.398454 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389563 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 01:20:26.398454 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389565 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 01:20:26.398454 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389568 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 01:20:26.398454 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389572 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 01:20:26.398454 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389575 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 01:20:26.398454 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389578 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 01:20:26.398454 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389580 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 01:20:26.398454 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389583 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 01:20:26.398454 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389586 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 01:20:26.398454 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389588 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 01:20:26.398454 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389591 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 01:20:26.398454 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389593 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 01:20:26.398454 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389596 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 01:20:26.398454 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389598 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 01:20:26.398454 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389601 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 01:20:26.398454 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389604 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 01:20:26.398454 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389606 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 01:20:26.398454 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389609 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 01:20:26.399079 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389611 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 01:20:26.399079 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389613 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 01:20:26.399079 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389616 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 01:20:26.399079 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389618 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 01:20:26.399079 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389620 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 01:20:26.399079 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389623 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 01:20:26.399079 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389625 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 01:20:26.399079 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389628 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 01:20:26.399079 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389630 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 01:20:26.399079 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389632 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 01:20:26.399079 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389635 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 01:20:26.399079 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389639 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 01:20:26.399079 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389642 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 01:20:26.399079 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389647 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 01:20:26.399079 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389651 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 01:20:26.399079 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389653 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 01:20:26.399079 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389656 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 01:20:26.399079 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389659 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 01:20:26.399079 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389661 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 01:20:26.399079 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389664 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 01:20:26.399786 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389667 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 01:20:26.399786 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389670 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 01:20:26.399786 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389672 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 01:20:26.399786 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389676 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 01:20:26.399786 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389678 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 01:20:26.399786 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389681 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 01:20:26.399786 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389683 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 01:20:26.399786 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389686 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 01:20:26.399786 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389688 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 06 01:20:26.399786 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389691 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 01:20:26.399786 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389694 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 01:20:26.399786 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389697 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 01:20:26.399786 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389699 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 01:20:26.399786 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389702 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 01:20:26.399786 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389704 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 01:20:26.399786 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389707 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 01:20:26.399786 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389709 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 01:20:26.399786 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389711 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 01:20:26.399786 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389714 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 01:20:26.399786 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389716 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 01:20:26.400294 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389719 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 01:20:26.400294 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389721 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 01:20:26.400294 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389724 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 01:20:26.400294 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389729 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 01:20:26.400294 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389733 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 01:20:26.400294 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389735 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 01:20:26.400294 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389738 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 01:20:26.400294 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389741 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 01:20:26.400294 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389744 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 01:20:26.400294 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389746 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 01:20:26.400294 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389750 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 01:20:26.400294 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389754 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 01:20:26.400294 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389757 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 01:20:26.400294 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389760 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 01:20:26.400294 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389762 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 01:20:26.400294 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389764 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 01:20:26.400294 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389767 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 01:20:26.400294 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389769 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 01:20:26.400294 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389771 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 01:20:26.400294 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389774 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 01:20:26.400814 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.389776 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 01:20:26.400814 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.390364 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 01:20:26.400814 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.398096 2565 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 01:20:26.400814 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.398129 2565 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 01:20:26.400814 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.399054 2565 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 01:20:26.400814 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.399084 2565 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 01:20:26.400814 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399426 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 01:20:26.400814 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399442 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 01:20:26.400814 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399447 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 01:20:26.400814 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399450 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 01:20:26.400814 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399453 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 01:20:26.400814 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399456 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 01:20:26.400814 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399459 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 01:20:26.400814 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399462 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 01:20:26.400814 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399465 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 01:20:26.401197 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399468 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 01:20:26.401197 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399470 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 01:20:26.401197 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399473 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 01:20:26.401197 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399475 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 01:20:26.401197 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399478 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 01:20:26.401197 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399480 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 01:20:26.401197 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399483 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 01:20:26.401197 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399486 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 01:20:26.401197 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399489 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 01:20:26.401197 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399491 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 01:20:26.401197 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399494 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 01:20:26.401197 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399497 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 01:20:26.401197 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399500 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 01:20:26.401197 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399504 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 01:20:26.401197 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399509 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 01:20:26.401197 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399512 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 01:20:26.401197 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399515 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 01:20:26.401197 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399518 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 01:20:26.401197 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399521 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 01:20:26.401684 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399523 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 01:20:26.401684 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399526 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 01:20:26.401684 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399528 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 01:20:26.401684 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399532 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 01:20:26.401684 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399538 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 01:20:26.401684 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399541 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 01:20:26.401684 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399544 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 01:20:26.401684 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399546 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 01:20:26.401684 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399549 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 01:20:26.401684 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399552 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 01:20:26.401684 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399555 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 01:20:26.401684 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399557 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 01:20:26.401684 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399560 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 01:20:26.401684 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399562 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 01:20:26.401684 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399565 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 01:20:26.401684 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399567 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 01:20:26.401684 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399570 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 01:20:26.401684 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399572 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 01:20:26.401684 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399575 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 01:20:26.402153 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399577 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 01:20:26.402153 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399580 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 01:20:26.402153 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399582 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 06 01:20:26.402153 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399585 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 01:20:26.402153 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399587 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 01:20:26.402153 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399591 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 01:20:26.402153 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399593 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 01:20:26.402153 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399596 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 01:20:26.402153 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399598 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 01:20:26.402153 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399601 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 01:20:26.402153 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399603 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 01:20:26.402153 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399606 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 01:20:26.402153 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399608 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 01:20:26.402153 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399611 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 01:20:26.402153 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399613 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 01:20:26.402153 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399616 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 01:20:26.402153 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399618 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 01:20:26.402153 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399622 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 01:20:26.402153 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399625 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 01:20:26.402153 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399628 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 01:20:26.402661 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399630 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 01:20:26.402661 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399633 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 01:20:26.402661 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399635 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 01:20:26.402661 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399637 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 01:20:26.402661 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399640 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 01:20:26.402661 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399642 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 01:20:26.402661 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399645 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 01:20:26.402661 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399647 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 01:20:26.402661 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399649 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 01:20:26.402661 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399652 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 01:20:26.402661 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399655 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 01:20:26.402661 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399657 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 01:20:26.402661 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399660 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 01:20:26.402661 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399662 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 01:20:26.402661 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399665 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 01:20:26.402661 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399668 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 01:20:26.402661 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399670 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 01:20:26.402661 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399673 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 01:20:26.402661 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399676 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 01:20:26.402661 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399678 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 01:20:26.403144 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399680 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 01:20:26.403144 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399683 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 01:20:26.403144 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399685 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 01:20:26.403144 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399687 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 01:20:26.403144 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399690 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 01:20:26.403144 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399693 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 01:20:26.403144 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399695 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 01:20:26.403144 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399699 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 01:20:26.403144 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399703 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 01:20:26.403144 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399707 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 01:20:26.403144 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399711 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 01:20:26.403144 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399714 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 01:20:26.403144 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.399720 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 01:20:26.403144 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399847 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 01:20:26.403594 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399852 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 01:20:26.403594 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399855 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 01:20:26.403594 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399858 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 01:20:26.403594 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399861 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 01:20:26.403594 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399864 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 01:20:26.403594 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399867 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 01:20:26.403594 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399870 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 01:20:26.403594 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399872 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 01:20:26.403594 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399875 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 01:20:26.403594 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399877 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 01:20:26.403594 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399880 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 01:20:26.403594 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399883 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 01:20:26.403594 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399885 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 01:20:26.403594 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399888 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 01:20:26.403594 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399891 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 01:20:26.403594 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399893 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 01:20:26.403594 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399896 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 01:20:26.403594 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399899 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 01:20:26.403594 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399901 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 01:20:26.404095 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399904 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 01:20:26.404095 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399906 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 01:20:26.404095 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399909 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 01:20:26.404095 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399911 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 01:20:26.404095 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399914 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 01:20:26.404095 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399916 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 01:20:26.404095 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399919 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 01:20:26.404095 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399922 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 01:20:26.404095 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399924 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 01:20:26.404095 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399927 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 01:20:26.404095 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399930 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 01:20:26.404095 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399932 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 01:20:26.404095 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399935 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 01:20:26.404095 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399937 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 01:20:26.404095 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399940 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 06 01:20:26.404095 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399942 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 01:20:26.404095 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399945 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 01:20:26.404095 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399947 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 01:20:26.404095 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399949 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 01:20:26.404095 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399952 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 01:20:26.404644 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399954 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 01:20:26.404644 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399957 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 01:20:26.404644 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399959 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 01:20:26.404644 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399961 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 01:20:26.404644 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399964 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 01:20:26.404644 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399966 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 01:20:26.404644 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399969 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 01:20:26.404644 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399972 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 01:20:26.404644 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399975 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 01:20:26.404644 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399977 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 01:20:26.404644 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399979 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 01:20:26.404644 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399982 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 01:20:26.404644 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399984 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 01:20:26.404644 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399987 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 01:20:26.404644 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399989 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 01:20:26.404644 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399991 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 01:20:26.404644 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399994 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 01:20:26.404644 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.399998 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 01:20:26.404644 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.400003 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 01:20:26.404644 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.400005 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 01:20:26.405131 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.400008 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 01:20:26.405131 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.400011 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 01:20:26.405131 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.400014 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 01:20:26.405131 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.400016 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 01:20:26.405131 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.400019 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 01:20:26.405131 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.400022 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 01:20:26.405131 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.400024 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 01:20:26.405131 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.400026 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 01:20:26.405131 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.400029 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 01:20:26.405131 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.400032 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 01:20:26.405131 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.400034 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 01:20:26.405131 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.400037 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 01:20:26.405131 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.400039 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 01:20:26.405131 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.400041 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 01:20:26.405131 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.400044 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 01:20:26.405131 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.400046 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 01:20:26.405131 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.400049 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 01:20:26.405131 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.400051 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 01:20:26.405131 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.400054 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 01:20:26.405131 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.400057 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 01:20:26.405763 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.400059 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 01:20:26.405763 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.400062 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 01:20:26.405763 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.400066 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 01:20:26.405763 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.400069 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 01:20:26.405763 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.400072 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 01:20:26.405763 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.400074 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 01:20:26.405763 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.400077 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 01:20:26.405763 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.400080 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 01:20:26.405763 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.400083 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 01:20:26.405763 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.400087 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 01:20:26.405763 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.400091 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 01:20:26.405763 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.400094 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 01:20:26.405763 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.400097 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 01:20:26.405763 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.400099 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 01:20:26.405763 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.400101 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 01:20:26.405763 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.400104 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 01:20:26.405763 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.400106 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 01:20:26.405763 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.400109 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 01:20:26.405763 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:26.400111 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 01:20:26.406223 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.400127 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 01:20:26.406223 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.400279 2565 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 01:20:26.406223 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.403121 2565 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 01:20:26.406223 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.404351 2565 server.go:1019] "Starting client certificate rotation" Jul 06 01:20:26.406223 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.404452 2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 01:20:26.406223 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.404483 2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 01:20:26.408344 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.408305 2565 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 01:20:26.411661 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.411637 2565 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 01:20:26.425257 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.425226 2565 log.go:25] "Validated CRI v1 runtime API" Jul 06 01:20:26.427549 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.427529 2565 log.go:25] "Validated CRI v1 image API" Jul 06 01:20:26.428446 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.428421 2565 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 01:20:26.430974 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.430950 2565 fs.go:135] Filesystem UUIDs: map[5f83f9c8-211b-4c33-afd0-8435b785f6ae:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 a305183b-12c9-4f3c-9552-3ae6b06cb459:/dev/nvme0n1p3] Jul 06 01:20:26.431060 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.430972 2565 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 01:20:26.434273 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.434252 2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 01:20:26.437014 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.436867 2565 manager.go:216] Machine: {Timestamp:2026-07-06 01:20:26.435576385 +0000 UTC m=+0.318184552 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3092530 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24e94572167236ff609d56ab6bbc70 SystemUUID:ec24e945-7216-7236-ff60-9d56ab6bbc70 BootID:1e4775ca-71a7-4430-b17e-fc908fd91954 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:18:23:0c:e0:b5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:18:23:0c:e0:b5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:8a:fe:8e:4e:c4:63 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 01:20:26.437014 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.437009 2565 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 01:20:26.437160 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.437143 2565 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 01:20:26.440270 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.440236 2565 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 01:20:26.440433 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.440271 2565 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-159.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 01:20:26.441011 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.440997 2565 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 01:20:26.441011 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.441012 2565 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 01:20:26.441081 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.441020 2565 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 01:20:26.442281 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.442263 2565 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 01:20:26.443555 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.443540 2565 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 01:20:26.446854 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.446834 2565 state_mem.go:36] "Initialized new in-memory state store" Jul 06 01:20:26.447017 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.447007 2565 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 01:20:26.447059 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.447049 2565 kubelet.go:486] "Attempting to sync node with API server" Jul 06 01:20:26.447087 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.447078 2565 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 01:20:26.447115 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.447103 2565 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 01:20:26.447146 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.447129 2565 kubelet.go:398] "Adding apiserver pod source" Jul 06 01:20:26.447146 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.447142 2565 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 01:20:26.447810 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.447787 2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 01:20:26.447810 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.447814 2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 01:20:26.453681 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.453647 2565 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 01:20:26.455536 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.455519 2565 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 01:20:26.455631 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.455551 2565 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 01:20:26.456612 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.456597 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 01:20:26.456664 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.456615 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 01:20:26.456664 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.456622 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 01:20:26.456664 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.456627 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 01:20:26.456664 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.456636 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 01:20:26.456664 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.456642 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 01:20:26.456664 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.456650 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 01:20:26.456664 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.456655 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 01:20:26.456664 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.456662 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 01:20:26.456664 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.456668 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 01:20:26.456923 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.456677 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 01:20:26.456923 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.456683 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 01:20:26.457401 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.457388 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 01:20:26.457401 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.457398 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 01:20:26.457586 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:26.457556 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-159.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 01:20:26.457643 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:26.457555 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 01:20:26.460263 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.460236 2565 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-159.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 06 01:20:26.460367 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.460311 2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4xmht" Jul 06 01:20:26.461027 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.461016 2565 server.go:1268] "Started kubelet" Jul 06 01:20:26.461119 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.461094 2565 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 01:20:26.461484 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.461408 2565 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 01:20:26.461579 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.461492 2565 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 01:20:26.462110 ip-10-0-143-159 systemd[1]: Started Kubernetes Kubelet. Jul 06 01:20:26.463145 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.463126 2565 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 01:20:26.463299 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.463275 2565 server.go:310] "Adding debug handlers to kubelet server" Jul 06 01:20:26.466425 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.466396 2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4xmht" Jul 06 01:20:26.467487 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:26.467456 2565 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 06 01:20:26.468821 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.468808 2565 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 01:20:26.468899 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.468829 2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 01:20:26.468961 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:26.468929 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-159.ec2.internal\" not found" Jul 06 01:20:26.469233 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:26.468060 2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-159.ec2.internal.18bf8ea740de140e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-159.ec2.internal,UID:ip-10-0-143-159.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-159.ec2.internal,},FirstTimestamp:2026-07-06 01:20:26.460984334 +0000 UTC m=+0.343592503,LastTimestamp:2026-07-06 01:20:26.460984334 +0000 UTC m=+0.343592503,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-159.ec2.internal,}" Jul 06 01:20:26.469484 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.469466 2565 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 01:20:26.469634 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.469606 2565 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 01:20:26.469634 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.469464 2565 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 01:20:26.469780 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.469686 2565 reconstruct.go:97] "Volume reconstruction finished" Jul 06 01:20:26.469780 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.469696 2565 reconciler.go:29] "Reconciler: start to sync state" Jul 06 01:20:26.470506 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.470488 2565 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 01:20:26.470506 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.470508 2565 factory.go:55] Registering systemd factory Jul 06 01:20:26.470506 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.470522 2565 factory.go:223] Registration of the systemd container factory successfully Jul 06 01:20:26.470808 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.470788 2565 factory.go:153] Registering CRI-O factory Jul 06 01:20:26.470808 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.470810 2565 factory.go:223] Registration of the crio container factory successfully Jul 06 01:20:26.470905 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.470829 2565 factory.go:103] Registering Raw factory Jul 06 01:20:26.470905 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.470840 2565 manager.go:1196] Started watching for new ooms in manager Jul 06 01:20:26.471476 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.471460 2565 manager.go:349] Starting recovery of all containers Jul 06 01:20:26.473572 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.473551 2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 01:20:26.477092 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:26.476941 2565 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-143-159.ec2.internal\" not found" node="ip-10-0-143-159.ec2.internal" Jul 06 01:20:26.478375 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.478306 2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 01:20:26.482751 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.482723 2565 manager.go:354] Recovery completed Jul 06 01:20:26.489196 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.489181 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 01:20:26.491312 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.491289 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeHasSufficientMemory" Jul 06 01:20:26.491426 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.491338 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 01:20:26.491426 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.491351 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeHasSufficientPID" Jul 06 01:20:26.491940 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.491926 2565 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 01:20:26.492009 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.491940 2565 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 01:20:26.492009 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.491946 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 01:20:26.492009 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.491963 2565 state_mem.go:36] "Initialized new in-memory state store" Jul 06 01:20:26.492734 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.492717 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeHasSufficientMemory" Jul 06 01:20:26.492796 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.492748 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 01:20:26.492796 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.492760 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeHasSufficientPID" Jul 06 01:20:26.493548 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.493535 2565 policy_none.go:49] "None policy: Start" Jul 06 01:20:26.493620 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.493556 2565 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 01:20:26.493620 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.493596 2565 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 01:20:26.494808 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.494794 2565 policy_none.go:47] "Start" Jul 06 01:20:26.520125 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.520094 2565 manager.go:337] "Starting Device Plugin manager" Jul 06 01:20:26.520272 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:26.520145 2565 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 01:20:26.520272 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.520157 2565 server.go:87] "Starting device plugin registration server" Jul 06 01:20:26.520501 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.520483 2565 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 01:20:26.520559 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.520503 2565 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 01:20:26.520625 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.520610 2565 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 01:20:26.520704 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.520693 2565 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 01:20:26.520750 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.520706 2565 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 01:20:26.521376 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:26.521351 2565 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 01:20:26.521494 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:26.521413 2565 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-159.ec2.internal\" not found" Jul 06 01:20:26.576400 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.576369 2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 01:20:26.577465 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.576418 2565 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 01:20:26.577465 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.576444 2565 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 01:20:26.577465 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:26.576481 2565 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 06 01:20:26.578757 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.578735 2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 01:20:26.620786 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.620741 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 01:20:26.622002 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.621984 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeHasSufficientMemory" Jul 06 01:20:26.622123 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.622019 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 01:20:26.622123 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.622030 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeHasSufficientPID" Jul 06 01:20:26.622123 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.622055 2565 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-159.ec2.internal" Jul 06 01:20:26.630587 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.630510 2565 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-159.ec2.internal" Jul 06 01:20:26.630587 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:26.630544 2565 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-159.ec2.internal\": node \"ip-10-0-143-159.ec2.internal\" not found" Jul 06 01:20:26.647377 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:26.647349 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-159.ec2.internal\" not found" Jul 06 01:20:26.677305 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.677269 2565 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-159.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-159.ec2.internal"] Jul 06 01:20:26.677488 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.677394 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 01:20:26.678562 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.678546 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeHasSufficientMemory" Jul 06 01:20:26.678628 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.678576 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 01:20:26.678628 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.678586 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeHasSufficientPID" Jul 06 01:20:26.679913 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.679900 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 01:20:26.680029 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.680013 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-159.ec2.internal" Jul 06 01:20:26.680086 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.680076 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 01:20:26.680799 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.680783 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeHasSufficientMemory" Jul 06 01:20:26.680871 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.680816 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 01:20:26.680871 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.680832 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeHasSufficientPID" Jul 06 01:20:26.680871 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.680835 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeHasSufficientMemory" Jul 06 01:20:26.680871 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.680854 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 01:20:26.680871 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.680863 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeHasSufficientPID" Jul 06 01:20:26.682049 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.682034 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-159.ec2.internal" Jul 06 01:20:26.682094 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.682065 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 01:20:26.682954 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.682933 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeHasSufficientMemory" Jul 06 01:20:26.682954 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.682965 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 01:20:26.683123 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.682974 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeHasSufficientPID" Jul 06 01:20:26.710390 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:26.710358 2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-159.ec2.internal\" not found" node="ip-10-0-143-159.ec2.internal" Jul 06 01:20:26.715035 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:26.715014 2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-159.ec2.internal\" not found" node="ip-10-0-143-159.ec2.internal" Jul 06 01:20:26.748420 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:26.748383 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-159.ec2.internal\" not found" Jul 06 01:20:26.771624 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.771590 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/af723d296c5644a068a39b9f14c5579d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-159.ec2.internal\" (UID: \"af723d296c5644a068a39b9f14c5579d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-159.ec2.internal" Jul 06 01:20:26.771773 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.771631 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/af723d296c5644a068a39b9f14c5579d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-159.ec2.internal\" (UID: \"af723d296c5644a068a39b9f14c5579d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-159.ec2.internal" Jul 06 01:20:26.771773 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.771656 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/008efb678643724ebfdbed72c5411e31-config\") pod \"kube-apiserver-proxy-ip-10-0-143-159.ec2.internal\" (UID: \"008efb678643724ebfdbed72c5411e31\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-159.ec2.internal" Jul 06 01:20:26.849493 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:26.849444 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-159.ec2.internal\" not found" Jul 06 01:20:26.871865 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.871843 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/af723d296c5644a068a39b9f14c5579d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-159.ec2.internal\" (UID: \"af723d296c5644a068a39b9f14c5579d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-159.ec2.internal" Jul 06 01:20:26.871949 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.871871 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/af723d296c5644a068a39b9f14c5579d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-159.ec2.internal\" (UID: \"af723d296c5644a068a39b9f14c5579d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-159.ec2.internal" Jul 06 01:20:26.871949 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.871890 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/008efb678643724ebfdbed72c5411e31-config\") pod \"kube-apiserver-proxy-ip-10-0-143-159.ec2.internal\" (UID: \"008efb678643724ebfdbed72c5411e31\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-159.ec2.internal" Jul 06 01:20:26.871949 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.871919 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/008efb678643724ebfdbed72c5411e31-config\") pod \"kube-apiserver-proxy-ip-10-0-143-159.ec2.internal\" (UID: \"008efb678643724ebfdbed72c5411e31\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-159.ec2.internal" Jul 06 01:20:26.872074 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.871949 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/af723d296c5644a068a39b9f14c5579d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-159.ec2.internal\" (UID: \"af723d296c5644a068a39b9f14c5579d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-159.ec2.internal" Jul 06 01:20:26.872074 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:26.871950 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/af723d296c5644a068a39b9f14c5579d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-159.ec2.internal\" (UID: \"af723d296c5644a068a39b9f14c5579d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-159.ec2.internal" Jul 06 01:20:26.950308 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:26.950227 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-159.ec2.internal\" not found" Jul 06 01:20:28.754872 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:28.754825 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-159.ec2.internal" Jul 06 01:20:28.759710 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:28.759689 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-159.ec2.internal" Jul 06 01:20:28.792444 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:28.792406 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-159.ec2.internal\" not found" Jul 06 01:20:28.892976 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:28.892920 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-159.ec2.internal\" not found" Jul 06 01:20:28.993568 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:28.993487 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-159.ec2.internal\" not found" Jul 06 01:20:29.094133 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:29.094094 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-159.ec2.internal\" not found" Jul 06 01:20:29.144619 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:29.144588 2565 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 01:20:29.145350 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:29.144778 2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 01:20:29.145350 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:29.144801 2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 01:20:29.195103 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:29.195067 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-159.ec2.internal\" not found" Jul 06 01:20:29.212712 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:29.212652 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 01:15:28 +0000 UTC" deadline="2027-12-01 06:37:55.841406774 +0000 UTC" Jul 06 01:20:29.212712 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:29.212710 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12317h17m26.628702419s" Jul 06 01:20:29.267127 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:29.267081 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod008efb678643724ebfdbed72c5411e31.slice/crio-c353311a2f26bda49e1e3cfb3e32f8a23ee14d05a5baf5a07d6add28f33d768a WatchSource:0}: Error finding container c353311a2f26bda49e1e3cfb3e32f8a23ee14d05a5baf5a07d6add28f33d768a: Status 404 returned error can't find the container with id c353311a2f26bda49e1e3cfb3e32f8a23ee14d05a5baf5a07d6add28f33d768a Jul 06 01:20:29.268789 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:29.268751 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf723d296c5644a068a39b9f14c5579d.slice/crio-0d825f3082cbebdc0299f6e6d73e72aa7f90b5abcc4d88b44c0850bbacb12494 WatchSource:0}: Error finding container 0d825f3082cbebdc0299f6e6d73e72aa7f90b5abcc4d88b44c0850bbacb12494: Status 404 returned error can't find the container with id 0d825f3082cbebdc0299f6e6d73e72aa7f90b5abcc4d88b44c0850bbacb12494 Jul 06 01:20:29.272672 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:29.272657 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 01:20:29.295862 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:29.295824 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-159.ec2.internal\" not found" Jul 06 01:20:29.321585 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:29.321518 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-159.ec2.internal" event={"ID":"af723d296c5644a068a39b9f14c5579d","Type":"ContainerStarted","Data":"0d825f3082cbebdc0299f6e6d73e72aa7f90b5abcc4d88b44c0850bbacb12494"} Jul 06 01:20:29.322625 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:29.322602 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-159.ec2.internal" event={"ID":"008efb678643724ebfdbed72c5411e31","Type":"ContainerStarted","Data":"c353311a2f26bda49e1e3cfb3e32f8a23ee14d05a5baf5a07d6add28f33d768a"} Jul 06 01:20:29.396848 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:29.396796 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-159.ec2.internal\" not found" Jul 06 01:20:29.492267 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:29.492241 2565 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 01:20:29.510448 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:29.510330 2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-159.ec2.internal" Jul 06 01:20:29.525425 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:29.525394 2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 01:20:29.527227 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:29.527211 2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-159.ec2.internal" Jul 06 01:20:29.528685 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:29.528670 2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 01:20:29.535671 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:29.535650 2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 01:20:30.020001 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.019967 2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 01:20:30.062214 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.062180 2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 01:20:30.190093 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.190047 2565 apiserver.go:52] "Watching apiserver" Jul 06 01:20:30.200082 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.200054 2565 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 01:20:30.200498 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.200471 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-159.ec2.internal","openshift-multus/multus-additional-cni-plugins-5nrjj","openshift-image-registry/node-ca-5nr92","openshift-multus/multus-87qvv","openshift-multus/network-metrics-daemon-drnhm","openshift-network-diagnostics/network-check-target-4ss2v","openshift-network-operator/iptables-alerter-wcl9v","openshift-ovn-kubernetes/ovnkube-node-b9krk","kube-system/global-pull-secret-syncer-pm8wq","kube-system/konnectivity-agent-k46tr","kube-system/kube-apiserver-proxy-ip-10-0-143-159.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8jjq","openshift-cluster-node-tuning-operator/tuned-lzmx9","openshift-dns/node-resolver-2s7xk"] Jul 06 01:20:30.202125 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.202102 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wcl9v" Jul 06 01:20:30.203300 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.203252 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.204500 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.204479 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5nr92" Jul 06 01:20:30.205415 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.205352 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-9mpdd\"" Jul 06 01:20:30.205769 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.205749 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 01:20:30.205864 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.205808 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.205993 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.205977 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 01:20:30.206409 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.206391 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 01:20:30.206710 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.206633 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 01:20:30.206932 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.206913 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-vj8qt\"" Jul 06 01:20:30.207284 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.207180 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 01:20:30.207571 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.207553 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-9c475\"" Jul 06 01:20:30.207808 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.207760 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 01:20:30.208006 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.207992 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 01:20:30.208291 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.208274 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 01:20:30.208588 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.208570 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 01:20:30.209055 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.209036 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-n9zdk\"" Jul 06 01:20:30.209155 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.209068 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-drnhm" Jul 06 01:20:30.209155 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:30.209141 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-drnhm" podUID="fbe62245-715f-4565-bfa8-af7c3ab5d36a" Jul 06 01:20:30.209407 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.209360 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 01:20:30.209494 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.209444 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 01:20:30.209787 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.209758 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ss2v" Jul 06 01:20:30.210033 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:30.209824 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ss2v" podUID="8bcfcc54-ba2d-4049-9077-6cc8ce713838" Jul 06 01:20:30.210033 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.209939 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:20:30.210033 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:30.209995 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pm8wq" podUID="4c0a1cea-8dca-4e9d-afdb-81f874754505" Jul 06 01:20:30.210340 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.210045 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 01:20:30.210522 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.210505 2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 01:20:30.211493 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.211087 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2s7xk" Jul 06 01:20:30.212411 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.212367 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5nrjj" Jul 06 01:20:30.213133 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.213112 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 01:20:30.213528 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.213508 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-lgmlw\"" Jul 06 01:20:30.213810 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.213629 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 01:20:30.214111 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.214033 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-k46tr" Jul 06 01:20:30.214576 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.214552 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 01:20:30.214901 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.214849 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 01:20:30.214981 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.214857 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-4kzbm\"" Jul 06 01:20:30.215966 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.215944 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.216648 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.216631 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-wn6hq\"" Jul 06 01:20:30.217292 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.216780 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 01:20:30.217292 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.217064 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 01:20:30.218308 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.218290 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 01:20:30.218416 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.218307 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 01:20:30.218663 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.218646 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 01:20:30.218717 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.218677 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 01:20:30.218909 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.218892 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dkpt6\"" Jul 06 01:20:30.219192 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.219167 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 01:20:30.219273 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.219243 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 01:20:30.220358 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.220337 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8jjq" Jul 06 01:20:30.222864 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.222842 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/32777e07-d94a-4e69-aad9-68eb154e397b-iptables-alerter-script\") pod \"iptables-alerter-wcl9v\" (UID: \"32777e07-d94a-4e69-aad9-68eb154e397b\") " pod="openshift-network-operator/iptables-alerter-wcl9v" Jul 06 01:20:30.222960 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.222872 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-system-cni-dir\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.222960 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.222933 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-multus-conf-dir\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.222960 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.222950 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4c0a1cea-8dca-4e9d-afdb-81f874754505-dbus\") pod \"global-pull-secret-syncer-pm8wq\" (UID: \"4c0a1cea-8dca-4e9d-afdb-81f874754505\") " pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:20:30.223116 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.222972 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbe62245-715f-4565-bfa8-af7c3ab5d36a-metrics-certs\") pod \"network-metrics-daemon-drnhm\" (UID: \"fbe62245-715f-4565-bfa8-af7c3ab5d36a\") " pod="openshift-multus/network-metrics-daemon-drnhm" Jul 06 01:20:30.223116 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223017 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/32777e07-d94a-4e69-aad9-68eb154e397b-host-slash\") pod \"iptables-alerter-wcl9v\" (UID: \"32777e07-d94a-4e69-aad9-68eb154e397b\") " pod="openshift-network-operator/iptables-alerter-wcl9v" Jul 06 01:20:30.223116 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223048 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d-etc-kubernetes\") pod \"tuned-lzmx9\" (UID: \"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d\") " pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.223116 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223069 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d-lib-modules\") pod \"tuned-lzmx9\" (UID: \"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d\") " pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.223116 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223094 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d-tmp\") pod \"tuned-lzmx9\" (UID: \"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d\") " pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.223116 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223116 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-cni-binary-copy\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.223423 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223139 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-host-run-k8s-cni-cncf-io\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.223423 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223216 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-host-var-lib-cni-bin\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.223423 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223240 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96pht\" (UniqueName: \"kubernetes.io/projected/fbe62245-715f-4565-bfa8-af7c3ab5d36a-kube-api-access-96pht\") pod \"network-metrics-daemon-drnhm\" (UID: \"fbe62245-715f-4565-bfa8-af7c3ab5d36a\") " pod="openshift-multus/network-metrics-daemon-drnhm" Jul 06 01:20:30.223423 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223265 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/28f968d2-c730-436e-8bb9-413d0f88cf10-hosts-file\") pod \"node-resolver-2s7xk\" (UID: \"28f968d2-c730-436e-8bb9-413d0f88cf10\") " pod="openshift-dns/node-resolver-2s7xk" Jul 06 01:20:30.223423 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223290 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d-var-lib-kubelet\") pod \"tuned-lzmx9\" (UID: \"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d\") " pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.223423 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223311 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ec5262e8-19a7-4dce-bf06-4d792eb2b877-host\") pod \"node-ca-5nr92\" (UID: \"ec5262e8-19a7-4dce-bf06-4d792eb2b877\") " pod="openshift-image-registry/node-ca-5nr92" Jul 06 01:20:30.223423 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223341 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ec5262e8-19a7-4dce-bf06-4d792eb2b877-serviceca\") pod \"node-ca-5nr92\" (UID: \"ec5262e8-19a7-4dce-bf06-4d792eb2b877\") " pod="openshift-image-registry/node-ca-5nr92" Jul 06 01:20:30.223423 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223367 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-multus-cni-dir\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.223423 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223401 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-host-run-netns\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.223423 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223423 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-host-var-lib-kubelet\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.223894 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223446 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d-etc-systemd\") pod \"tuned-lzmx9\" (UID: \"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d\") " pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.223894 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223461 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d-run\") pod \"tuned-lzmx9\" (UID: \"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d\") " pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.223894 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223475 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8djjp\" (UniqueName: \"kubernetes.io/projected/ec5262e8-19a7-4dce-bf06-4d792eb2b877-kube-api-access-8djjp\") pod \"node-ca-5nr92\" (UID: \"ec5262e8-19a7-4dce-bf06-4d792eb2b877\") " pod="openshift-image-registry/node-ca-5nr92" Jul 06 01:20:30.223894 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223488 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-etc-kubernetes\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.223894 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223503 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d-etc-modprobe-d\") pod \"tuned-lzmx9\" (UID: \"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d\") " pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.223894 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223516 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d-etc-sysctl-conf\") pod \"tuned-lzmx9\" (UID: \"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d\") " pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.223894 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223520 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 01:20:30.223894 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223529 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d-host\") pod \"tuned-lzmx9\" (UID: \"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d\") " pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.223894 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223556 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d-etc-tuned\") pod \"tuned-lzmx9\" (UID: \"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d\") " pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.223894 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223569 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-os-release\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.223894 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223595 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-hostroot\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.223894 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223614 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-multus-daemon-config\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.223894 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223642 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vblkv\" (UniqueName: \"kubernetes.io/projected/8bcfcc54-ba2d-4049-9077-6cc8ce713838-kube-api-access-vblkv\") pod \"network-check-target-4ss2v\" (UID: \"8bcfcc54-ba2d-4049-9077-6cc8ce713838\") " pod="openshift-network-diagnostics/network-check-target-4ss2v" Jul 06 01:20:30.223894 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223673 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cr2v5\" (UniqueName: \"kubernetes.io/projected/28f968d2-c730-436e-8bb9-413d0f88cf10-kube-api-access-cr2v5\") pod \"node-resolver-2s7xk\" (UID: \"28f968d2-c730-436e-8bb9-413d0f88cf10\") " pod="openshift-dns/node-resolver-2s7xk" Jul 06 01:20:30.223894 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223687 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d-etc-sysctl-d\") pod \"tuned-lzmx9\" (UID: \"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d\") " pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.223894 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223705 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-host-var-lib-cni-multus\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.223894 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223719 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-host-run-multus-certs\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.223894 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223723 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2nxdl\"" Jul 06 01:20:30.224725 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223737 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4c0a1cea-8dca-4e9d-afdb-81f874754505-kubelet-config\") pod \"global-pull-secret-syncer-pm8wq\" (UID: \"4c0a1cea-8dca-4e9d-afdb-81f874754505\") " pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:20:30.224725 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223762 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7h2x\" (UniqueName: \"kubernetes.io/projected/32777e07-d94a-4e69-aad9-68eb154e397b-kube-api-access-v7h2x\") pod \"iptables-alerter-wcl9v\" (UID: \"32777e07-d94a-4e69-aad9-68eb154e397b\") " pod="openshift-network-operator/iptables-alerter-wcl9v" Jul 06 01:20:30.224725 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223786 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d-sys\") pod \"tuned-lzmx9\" (UID: \"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d\") " pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.224725 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223809 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-cnibin\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.224725 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223838 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-multus-socket-dir-parent\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.224725 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223875 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 01:20:30.224725 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223880 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmhjp\" (UniqueName: \"kubernetes.io/projected/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-kube-api-access-rmhjp\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.224725 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223905 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c0a1cea-8dca-4e9d-afdb-81f874754505-original-pull-secret\") pod \"global-pull-secret-syncer-pm8wq\" (UID: \"4c0a1cea-8dca-4e9d-afdb-81f874754505\") " pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:20:30.224725 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223988 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zw4n\" (UniqueName: \"kubernetes.io/projected/1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d-kube-api-access-7zw4n\") pod \"tuned-lzmx9\" (UID: \"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d\") " pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.224725 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.223992 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 01:20:30.224725 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.224018 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d-etc-sysconfig\") pod \"tuned-lzmx9\" (UID: \"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d\") " pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.224725 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.224039 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28f968d2-c730-436e-8bb9-413d0f88cf10-tmp-dir\") pod \"node-resolver-2s7xk\" (UID: \"28f968d2-c730-436e-8bb9-413d0f88cf10\") " pod="openshift-dns/node-resolver-2s7xk" Jul 06 01:20:30.224725 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.224477 2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 01:20:30.244933 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.244899 2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-59z2g" Jul 06 01:20:30.255590 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.255561 2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-59z2g" Jul 06 01:20:30.312440 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.312409 2565 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 01:20:30.325176 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.325138 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d-etc-modprobe-d\") pod \"tuned-lzmx9\" (UID: \"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d\") " pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.325350 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.325184 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d-etc-sysctl-conf\") pod \"tuned-lzmx9\" (UID: \"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d\") " pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.325350 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.325208 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-hostroot\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.325350 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.325264 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-hostroot\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.325350 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.325324 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f248c1a4-cee4-4e0b-90b8-1abec0b46b8f-registration-dir\") pod \"aws-ebs-csi-driver-node-k8jjq\" (UID: \"f248c1a4-cee4-4e0b-90b8-1abec0b46b8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8jjq" Jul 06 01:20:30.325350 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.325339 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d-etc-sysctl-conf\") pod \"tuned-lzmx9\" (UID: \"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d\") " pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.325578 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.325371 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f248c1a4-cee4-4e0b-90b8-1abec0b46b8f-sys-fs\") pod \"aws-ebs-csi-driver-node-k8jjq\" (UID: \"f248c1a4-cee4-4e0b-90b8-1abec0b46b8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8jjq" Jul 06 01:20:30.325578 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.325412 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/68d84c01-13c1-4832-bcbf-41ad082024f6-ovnkube-script-lib\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.325578 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.325329 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d-etc-modprobe-d\") pod \"tuned-lzmx9\" (UID: \"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d\") " pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.325578 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.325437 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f248c1a4-cee4-4e0b-90b8-1abec0b46b8f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k8jjq\" (UID: \"f248c1a4-cee4-4e0b-90b8-1abec0b46b8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8jjq" Jul 06 01:20:30.325578 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.325474 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cr2v5\" (UniqueName: \"kubernetes.io/projected/28f968d2-c730-436e-8bb9-413d0f88cf10-kube-api-access-cr2v5\") pod \"node-resolver-2s7xk\" (UID: \"28f968d2-c730-436e-8bb9-413d0f88cf10\") " pod="openshift-dns/node-resolver-2s7xk" Jul 06 01:20:30.325578 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.325510 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d-etc-sysctl-d\") pod \"tuned-lzmx9\" (UID: \"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d\") " pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.325578 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.325535 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-host-run-multus-certs\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.325578 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.325553 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-host-kubelet\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.325578 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.325568 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d-sys\") pod \"tuned-lzmx9\" (UID: \"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d\") " pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.325578 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.325583 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-multus-socket-dir-parent\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.325936 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.325597 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmhjp\" (UniqueName: \"kubernetes.io/projected/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-kube-api-access-rmhjp\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.325936 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.325611 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d-etc-sysconfig\") pod \"tuned-lzmx9\" (UID: \"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d\") " pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.325936 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.325626 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/32777e07-d94a-4e69-aad9-68eb154e397b-iptables-alerter-script\") pod \"iptables-alerter-wcl9v\" (UID: \"32777e07-d94a-4e69-aad9-68eb154e397b\") " pod="openshift-network-operator/iptables-alerter-wcl9v" Jul 06 01:20:30.325936 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.325640 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-system-cni-dir\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.325936 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.325653 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4c0a1cea-8dca-4e9d-afdb-81f874754505-dbus\") pod \"global-pull-secret-syncer-pm8wq\" (UID: \"4c0a1cea-8dca-4e9d-afdb-81f874754505\") " pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:20:30.325936 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.325674 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9b90df9d-7dc5-41c1-a529-7cfdc8a41d09-agent-certs\") pod \"konnectivity-agent-k46tr\" (UID: \"9b90df9d-7dc5-41c1-a529-7cfdc8a41d09\") " pod="kube-system/konnectivity-agent-k46tr" Jul 06 01:20:30.325936 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.325698 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-log-socket\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.325936 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.325718 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-host-cni-bin\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.325936 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.325733 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b771700e-b5fb-4e58-b526-570e6d21c9d7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5nrjj\" (UID: \"b771700e-b5fb-4e58-b526-570e6d21c9d7\") " pod="openshift-multus/multus-additional-cni-plugins-5nrjj" Jul 06 01:20:30.325936 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.325750 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbe62245-715f-4565-bfa8-af7c3ab5d36a-metrics-certs\") pod \"network-metrics-daemon-drnhm\" (UID: \"fbe62245-715f-4565-bfa8-af7c3ab5d36a\") " pod="openshift-multus/network-metrics-daemon-drnhm" Jul 06 01:20:30.325936 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.325775 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d-lib-modules\") pod \"tuned-lzmx9\" (UID: \"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d\") " pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.325936 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.325790 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-cni-binary-copy\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.325936 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.325806 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-host-var-lib-cni-bin\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.325936 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.325822 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b771700e-b5fb-4e58-b526-570e6d21c9d7-system-cni-dir\") pod \"multus-additional-cni-plugins-5nrjj\" (UID: \"b771700e-b5fb-4e58-b526-570e6d21c9d7\") " pod="openshift-multus/multus-additional-cni-plugins-5nrjj" Jul 06 01:20:30.325936 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.325843 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-var-lib-openvswitch\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.325936 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.325868 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-run-ovn\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.325936 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.325888 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/28f968d2-c730-436e-8bb9-413d0f88cf10-hosts-file\") pod \"node-resolver-2s7xk\" (UID: \"28f968d2-c730-436e-8bb9-413d0f88cf10\") " pod="openshift-dns/node-resolver-2s7xk" Jul 06 01:20:30.326553 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.325907 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d-var-lib-kubelet\") pod \"tuned-lzmx9\" (UID: \"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d\") " pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.326553 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.325929 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ec5262e8-19a7-4dce-bf06-4d792eb2b877-serviceca\") pod \"node-ca-5nr92\" (UID: \"ec5262e8-19a7-4dce-bf06-4d792eb2b877\") " pod="openshift-image-registry/node-ca-5nr92" Jul 06 01:20:30.326553 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.325946 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-host-run-netns\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.326553 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.325963 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9b90df9d-7dc5-41c1-a529-7cfdc8a41d09-konnectivity-ca\") pod \"konnectivity-agent-k46tr\" (UID: \"9b90df9d-7dc5-41c1-a529-7cfdc8a41d09\") " pod="kube-system/konnectivity-agent-k46tr" Jul 06 01:20:30.326553 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.325977 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/68d84c01-13c1-4832-bcbf-41ad082024f6-env-overrides\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.326553 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.325993 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d-etc-systemd\") pod \"tuned-lzmx9\" (UID: \"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d\") " pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.326553 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.326007 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8djjp\" (UniqueName: \"kubernetes.io/projected/ec5262e8-19a7-4dce-bf06-4d792eb2b877-kube-api-access-8djjp\") pod \"node-ca-5nr92\" (UID: \"ec5262e8-19a7-4dce-bf06-4d792eb2b877\") " pod="openshift-image-registry/node-ca-5nr92" Jul 06 01:20:30.326553 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.326022 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-etc-kubernetes\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.326553 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.326036 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b771700e-b5fb-4e58-b526-570e6d21c9d7-cni-binary-copy\") pod \"multus-additional-cni-plugins-5nrjj\" (UID: \"b771700e-b5fb-4e58-b526-570e6d21c9d7\") " pod="openshift-multus/multus-additional-cni-plugins-5nrjj" Jul 06 01:20:30.326553 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.326056 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-host-run-ovn-kubernetes\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.326553 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.326083 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f248c1a4-cee4-4e0b-90b8-1abec0b46b8f-device-dir\") pod \"aws-ebs-csi-driver-node-k8jjq\" (UID: \"f248c1a4-cee4-4e0b-90b8-1abec0b46b8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8jjq" Jul 06 01:20:30.326553 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.326100 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f248c1a4-cee4-4e0b-90b8-1abec0b46b8f-etc-selinux\") pod \"aws-ebs-csi-driver-node-k8jjq\" (UID: \"f248c1a4-cee4-4e0b-90b8-1abec0b46b8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8jjq" Jul 06 01:20:30.326553 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.326120 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b771700e-b5fb-4e58-b526-570e6d21c9d7-os-release\") pod \"multus-additional-cni-plugins-5nrjj\" (UID: \"b771700e-b5fb-4e58-b526-570e6d21c9d7\") " pod="openshift-multus/multus-additional-cni-plugins-5nrjj" Jul 06 01:20:30.326553 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.326147 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d-host\") pod \"tuned-lzmx9\" (UID: \"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d\") " pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.326553 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.326192 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d-etc-tuned\") pod \"tuned-lzmx9\" (UID: \"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d\") " pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.326553 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.326207 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-os-release\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.326553 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.326226 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-systemd-units\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.327348 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.326253 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-host-cni-netd\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.327348 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.326277 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/68d84c01-13c1-4832-bcbf-41ad082024f6-ovn-node-metrics-cert\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.327348 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.326307 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vblkv\" (UniqueName: \"kubernetes.io/projected/8bcfcc54-ba2d-4049-9077-6cc8ce713838-kube-api-access-vblkv\") pod \"network-check-target-4ss2v\" (UID: \"8bcfcc54-ba2d-4049-9077-6cc8ce713838\") " pod="openshift-network-diagnostics/network-check-target-4ss2v" Jul 06 01:20:30.327348 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.326390 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-system-cni-dir\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.327348 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.326425 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-etc-openvswitch\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.327348 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.326431 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d-var-lib-kubelet\") pod \"tuned-lzmx9\" (UID: \"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d\") " pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.327348 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.326439 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d-sys\") pod \"tuned-lzmx9\" (UID: \"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d\") " pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.327348 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.326464 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4c0a1cea-8dca-4e9d-afdb-81f874754505-dbus\") pod \"global-pull-secret-syncer-pm8wq\" (UID: \"4c0a1cea-8dca-4e9d-afdb-81f874754505\") " pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:20:30.327348 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.326534 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-os-release\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.327348 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.326666 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d-etc-sysctl-d\") pod \"tuned-lzmx9\" (UID: \"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d\") " pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.327348 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.326670 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-host-run-netns\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.327348 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.326694 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-host-var-lib-cni-bin\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.327348 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:30.326712 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 01:20:30.327348 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.326765 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/28f968d2-c730-436e-8bb9-413d0f88cf10-hosts-file\") pod \"node-resolver-2s7xk\" (UID: \"28f968d2-c730-436e-8bb9-413d0f88cf10\") " pod="openshift-dns/node-resolver-2s7xk" Jul 06 01:20:30.327348 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.326758 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d-lib-modules\") pod \"tuned-lzmx9\" (UID: \"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d\") " pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.327348 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.326812 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d-host\") pod \"tuned-lzmx9\" (UID: \"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d\") " pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.327348 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:30.326827 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbe62245-715f-4565-bfa8-af7c3ab5d36a-metrics-certs podName:fbe62245-715f-4565-bfa8-af7c3ab5d36a nodeName:}" failed. No retries permitted until 2026-07-06 01:20:30.826802681 +0000 UTC m=+2.968361473 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fbe62245-715f-4565-bfa8-af7c3ab5d36a-metrics-certs") pod "network-metrics-daemon-drnhm" (UID: "fbe62245-715f-4565-bfa8-af7c3ab5d36a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 01:20:30.328331 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.326889 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-multus-socket-dir-parent\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.328331 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.326949 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-host-run-multus-certs\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.328331 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.327034 2565 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 01:20:30.328331 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.327058 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prlgf\" (UniqueName: \"kubernetes.io/projected/68d84c01-13c1-4832-bcbf-41ad082024f6-kube-api-access-prlgf\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.328331 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.327105 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-host-var-lib-cni-multus\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.328331 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.327145 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4c0a1cea-8dca-4e9d-afdb-81f874754505-kubelet-config\") pod \"global-pull-secret-syncer-pm8wq\" (UID: \"4c0a1cea-8dca-4e9d-afdb-81f874754505\") " pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:20:30.328331 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.327174 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-run-systemd\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.328331 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.327202 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hrmc\" (UniqueName: \"kubernetes.io/projected/f248c1a4-cee4-4e0b-90b8-1abec0b46b8f-kube-api-access-6hrmc\") pod \"aws-ebs-csi-driver-node-k8jjq\" (UID: \"f248c1a4-cee4-4e0b-90b8-1abec0b46b8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8jjq" Jul 06 01:20:30.328331 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.327231 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b771700e-b5fb-4e58-b526-570e6d21c9d7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5nrjj\" (UID: \"b771700e-b5fb-4e58-b526-570e6d21c9d7\") " pod="openshift-multus/multus-additional-cni-plugins-5nrjj" Jul 06 01:20:30.328331 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.327291 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7h2x\" (UniqueName: \"kubernetes.io/projected/32777e07-d94a-4e69-aad9-68eb154e397b-kube-api-access-v7h2x\") pod \"iptables-alerter-wcl9v\" (UID: \"32777e07-d94a-4e69-aad9-68eb154e397b\") " pod="openshift-network-operator/iptables-alerter-wcl9v" Jul 06 01:20:30.328331 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.327317 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-cnibin\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.328331 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.327333 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-etc-kubernetes\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.328331 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.327344 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c0a1cea-8dca-4e9d-afdb-81f874754505-original-pull-secret\") pod \"global-pull-secret-syncer-pm8wq\" (UID: \"4c0a1cea-8dca-4e9d-afdb-81f874754505\") " pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:20:30.328331 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.327394 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zw4n\" (UniqueName: \"kubernetes.io/projected/1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d-kube-api-access-7zw4n\") pod \"tuned-lzmx9\" (UID: \"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d\") " pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.328331 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.327409 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d-etc-sysconfig\") pod \"tuned-lzmx9\" (UID: \"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d\") " pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.328331 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.327425 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28f968d2-c730-436e-8bb9-413d0f88cf10-tmp-dir\") pod \"node-resolver-2s7xk\" (UID: \"28f968d2-c730-436e-8bb9-413d0f88cf10\") " pod="openshift-dns/node-resolver-2s7xk" Jul 06 01:20:30.328331 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.327455 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-multus-conf-dir\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.329186 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.327467 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4c0a1cea-8dca-4e9d-afdb-81f874754505-kubelet-config\") pod \"global-pull-secret-syncer-pm8wq\" (UID: \"4c0a1cea-8dca-4e9d-afdb-81f874754505\") " pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:20:30.329186 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.327479 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ec5262e8-19a7-4dce-bf06-4d792eb2b877-serviceca\") pod \"node-ca-5nr92\" (UID: \"ec5262e8-19a7-4dce-bf06-4d792eb2b877\") " pod="openshift-image-registry/node-ca-5nr92" Jul 06 01:20:30.329186 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.327484 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b771700e-b5fb-4e58-b526-570e6d21c9d7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5nrjj\" (UID: \"b771700e-b5fb-4e58-b526-570e6d21c9d7\") " pod="openshift-multus/multus-additional-cni-plugins-5nrjj" Jul 06 01:20:30.329186 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.327480 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/32777e07-d94a-4e69-aad9-68eb154e397b-iptables-alerter-script\") pod \"iptables-alerter-wcl9v\" (UID: \"32777e07-d94a-4e69-aad9-68eb154e397b\") " pod="openshift-network-operator/iptables-alerter-wcl9v" Jul 06 01:20:30.329186 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.327526 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-host-var-lib-cni-multus\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.329186 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.327580 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/32777e07-d94a-4e69-aad9-68eb154e397b-host-slash\") pod \"iptables-alerter-wcl9v\" (UID: \"32777e07-d94a-4e69-aad9-68eb154e397b\") " pod="openshift-network-operator/iptables-alerter-wcl9v" Jul 06 01:20:30.329186 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.327591 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-cnibin\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.329186 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.327616 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d-etc-kubernetes\") pod \"tuned-lzmx9\" (UID: \"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d\") " pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.329186 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.327645 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d-tmp\") pod \"tuned-lzmx9\" (UID: \"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d\") " pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.329186 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.327676 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-host-run-k8s-cni-cncf-io\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.329186 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:30.327727 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 01:20:30.329186 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.327769 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-host-var-lib-kubelet\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.329186 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.327802 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28f968d2-c730-436e-8bb9-413d0f88cf10-tmp-dir\") pod \"node-resolver-2s7xk\" (UID: \"28f968d2-c730-436e-8bb9-413d0f88cf10\") " pod="openshift-dns/node-resolver-2s7xk" Jul 06 01:20:30.329186 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.327804 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-multus-daemon-config\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.329186 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.327838 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-host-run-netns\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.329186 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.327720 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-cni-binary-copy\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.329186 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.327942 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d-etc-systemd\") pod \"tuned-lzmx9\" (UID: \"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d\") " pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.330167 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:30.328026 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c0a1cea-8dca-4e9d-afdb-81f874754505-original-pull-secret podName:4c0a1cea-8dca-4e9d-afdb-81f874754505 nodeName:}" failed. No retries permitted until 2026-07-06 01:20:30.828008704 +0000 UTC m=+2.969567535 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4c0a1cea-8dca-4e9d-afdb-81f874754505-original-pull-secret") pod "global-pull-secret-syncer-pm8wq" (UID: "4c0a1cea-8dca-4e9d-afdb-81f874754505") : object "kube-system"/"original-pull-secret" not registered Jul 06 01:20:30.330167 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.327647 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-multus-conf-dir\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.330167 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.328159 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96pht\" (UniqueName: \"kubernetes.io/projected/fbe62245-715f-4565-bfa8-af7c3ab5d36a-kube-api-access-96pht\") pod \"network-metrics-daemon-drnhm\" (UID: \"fbe62245-715f-4565-bfa8-af7c3ab5d36a\") " pod="openshift-multus/network-metrics-daemon-drnhm" Jul 06 01:20:30.330167 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.328186 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ec5262e8-19a7-4dce-bf06-4d792eb2b877-host\") pod \"node-ca-5nr92\" (UID: \"ec5262e8-19a7-4dce-bf06-4d792eb2b877\") " pod="openshift-image-registry/node-ca-5nr92" Jul 06 01:20:30.330167 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.328225 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/32777e07-d94a-4e69-aad9-68eb154e397b-host-slash\") pod \"iptables-alerter-wcl9v\" (UID: \"32777e07-d94a-4e69-aad9-68eb154e397b\") " pod="openshift-network-operator/iptables-alerter-wcl9v" Jul 06 01:20:30.330167 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.328235 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-multus-cni-dir\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.330167 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.328263 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b771700e-b5fb-4e58-b526-570e6d21c9d7-cnibin\") pod \"multus-additional-cni-plugins-5nrjj\" (UID: \"b771700e-b5fb-4e58-b526-570e6d21c9d7\") " pod="openshift-multus/multus-additional-cni-plugins-5nrjj" Jul 06 01:20:30.330167 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.328271 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-host-var-lib-kubelet\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.330167 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.328307 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ec5262e8-19a7-4dce-bf06-4d792eb2b877-host\") pod \"node-ca-5nr92\" (UID: \"ec5262e8-19a7-4dce-bf06-4d792eb2b877\") " pod="openshift-image-registry/node-ca-5nr92" Jul 06 01:20:30.330167 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.328342 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-host-run-k8s-cni-cncf-io\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.330167 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.328351 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-multus-cni-dir\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.330167 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.328350 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48qkb\" (UniqueName: \"kubernetes.io/projected/b771700e-b5fb-4e58-b526-570e6d21c9d7-kube-api-access-48qkb\") pod \"multus-additional-cni-plugins-5nrjj\" (UID: \"b771700e-b5fb-4e58-b526-570e6d21c9d7\") " pod="openshift-multus/multus-additional-cni-plugins-5nrjj" Jul 06 01:20:30.330167 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.328498 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-run-openvswitch\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.330167 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.328537 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.330167 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.328568 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/68d84c01-13c1-4832-bcbf-41ad082024f6-ovnkube-config\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.330167 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.328571 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d-etc-kubernetes\") pod \"tuned-lzmx9\" (UID: \"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d\") " pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.331068 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.328623 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d-run\") pod \"tuned-lzmx9\" (UID: \"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d\") " pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.331068 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.328656 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-host-slash\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.331068 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.328685 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-node-log\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.331068 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.328710 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f248c1a4-cee4-4e0b-90b8-1abec0b46b8f-socket-dir\") pod \"aws-ebs-csi-driver-node-k8jjq\" (UID: \"f248c1a4-cee4-4e0b-90b8-1abec0b46b8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8jjq" Jul 06 01:20:30.331068 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.328765 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-multus-daemon-config\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.331068 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.328849 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d-run\") pod \"tuned-lzmx9\" (UID: \"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d\") " pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.331068 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.330990 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d-etc-tuned\") pod \"tuned-lzmx9\" (UID: \"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d\") " pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.331396 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.331352 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d-tmp\") pod \"tuned-lzmx9\" (UID: \"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d\") " pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.333999 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:30.333963 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 01:20:30.333999 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:30.333987 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 01:20:30.333999 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:30.334001 2565 projected.go:196] Error preparing data for projected volume kube-api-access-vblkv for pod openshift-network-diagnostics/network-check-target-4ss2v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 01:20:30.334268 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:30.334080 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8bcfcc54-ba2d-4049-9077-6cc8ce713838-kube-api-access-vblkv podName:8bcfcc54-ba2d-4049-9077-6cc8ce713838 nodeName:}" failed. No retries permitted until 2026-07-06 01:20:30.834056185 +0000 UTC m=+2.975614969 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vblkv" (UniqueName: "kubernetes.io/projected/8bcfcc54-ba2d-4049-9077-6cc8ce713838-kube-api-access-vblkv") pod "network-check-target-4ss2v" (UID: "8bcfcc54-ba2d-4049-9077-6cc8ce713838") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 01:20:30.336690 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.336623 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cr2v5\" (UniqueName: \"kubernetes.io/projected/28f968d2-c730-436e-8bb9-413d0f88cf10-kube-api-access-cr2v5\") pod \"node-resolver-2s7xk\" (UID: \"28f968d2-c730-436e-8bb9-413d0f88cf10\") " pod="openshift-dns/node-resolver-2s7xk" Jul 06 01:20:30.336903 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.336857 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8djjp\" (UniqueName: \"kubernetes.io/projected/ec5262e8-19a7-4dce-bf06-4d792eb2b877-kube-api-access-8djjp\") pod \"node-ca-5nr92\" (UID: \"ec5262e8-19a7-4dce-bf06-4d792eb2b877\") " pod="openshift-image-registry/node-ca-5nr92" Jul 06 01:20:30.337990 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.337957 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmhjp\" (UniqueName: \"kubernetes.io/projected/483f8b7d-8c78-4edb-b992-d448f8e1f2ca-kube-api-access-rmhjp\") pod \"multus-87qvv\" (UID: \"483f8b7d-8c78-4edb-b992-d448f8e1f2ca\") " pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.338288 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.338263 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96pht\" (UniqueName: \"kubernetes.io/projected/fbe62245-715f-4565-bfa8-af7c3ab5d36a-kube-api-access-96pht\") pod \"network-metrics-daemon-drnhm\" (UID: \"fbe62245-715f-4565-bfa8-af7c3ab5d36a\") " pod="openshift-multus/network-metrics-daemon-drnhm" Jul 06 01:20:30.338430 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.338396 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7h2x\" (UniqueName: \"kubernetes.io/projected/32777e07-d94a-4e69-aad9-68eb154e397b-kube-api-access-v7h2x\") pod \"iptables-alerter-wcl9v\" (UID: \"32777e07-d94a-4e69-aad9-68eb154e397b\") " pod="openshift-network-operator/iptables-alerter-wcl9v" Jul 06 01:20:30.339804 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.339781 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zw4n\" (UniqueName: \"kubernetes.io/projected/1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d-kube-api-access-7zw4n\") pod \"tuned-lzmx9\" (UID: \"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d\") " pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.431206 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.429271 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9b90df9d-7dc5-41c1-a529-7cfdc8a41d09-agent-certs\") pod \"konnectivity-agent-k46tr\" (UID: \"9b90df9d-7dc5-41c1-a529-7cfdc8a41d09\") " pod="kube-system/konnectivity-agent-k46tr" Jul 06 01:20:30.431206 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.429316 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-log-socket\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.431206 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.429344 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-host-cni-bin\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.431206 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.429368 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b771700e-b5fb-4e58-b526-570e6d21c9d7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5nrjj\" (UID: \"b771700e-b5fb-4e58-b526-570e6d21c9d7\") " pod="openshift-multus/multus-additional-cni-plugins-5nrjj" Jul 06 01:20:30.431206 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.429419 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-log-socket\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.431206 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.429425 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b771700e-b5fb-4e58-b526-570e6d21c9d7-system-cni-dir\") pod \"multus-additional-cni-plugins-5nrjj\" (UID: \"b771700e-b5fb-4e58-b526-570e6d21c9d7\") " pod="openshift-multus/multus-additional-cni-plugins-5nrjj" Jul 06 01:20:30.431206 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.429462 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b771700e-b5fb-4e58-b526-570e6d21c9d7-system-cni-dir\") pod \"multus-additional-cni-plugins-5nrjj\" (UID: \"b771700e-b5fb-4e58-b526-570e6d21c9d7\") " pod="openshift-multus/multus-additional-cni-plugins-5nrjj" Jul 06 01:20:30.431206 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.429469 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-var-lib-openvswitch\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.431206 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.429495 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-run-ovn\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.431206 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.429520 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9b90df9d-7dc5-41c1-a529-7cfdc8a41d09-konnectivity-ca\") pod \"konnectivity-agent-k46tr\" (UID: \"9b90df9d-7dc5-41c1-a529-7cfdc8a41d09\") " pod="kube-system/konnectivity-agent-k46tr" Jul 06 01:20:30.431206 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.429541 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/68d84c01-13c1-4832-bcbf-41ad082024f6-env-overrides\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.431206 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.429566 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b771700e-b5fb-4e58-b526-570e6d21c9d7-cni-binary-copy\") pod \"multus-additional-cni-plugins-5nrjj\" (UID: \"b771700e-b5fb-4e58-b526-570e6d21c9d7\") " pod="openshift-multus/multus-additional-cni-plugins-5nrjj" Jul 06 01:20:30.431206 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.429588 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-host-run-ovn-kubernetes\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.431206 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.429611 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f248c1a4-cee4-4e0b-90b8-1abec0b46b8f-device-dir\") pod \"aws-ebs-csi-driver-node-k8jjq\" (UID: \"f248c1a4-cee4-4e0b-90b8-1abec0b46b8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8jjq" Jul 06 01:20:30.431206 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.429635 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f248c1a4-cee4-4e0b-90b8-1abec0b46b8f-etc-selinux\") pod \"aws-ebs-csi-driver-node-k8jjq\" (UID: \"f248c1a4-cee4-4e0b-90b8-1abec0b46b8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8jjq" Jul 06 01:20:30.431206 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.429642 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b771700e-b5fb-4e58-b526-570e6d21c9d7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5nrjj\" (UID: \"b771700e-b5fb-4e58-b526-570e6d21c9d7\") " pod="openshift-multus/multus-additional-cni-plugins-5nrjj" Jul 06 01:20:30.431206 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.429657 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b771700e-b5fb-4e58-b526-570e6d21c9d7-os-release\") pod \"multus-additional-cni-plugins-5nrjj\" (UID: \"b771700e-b5fb-4e58-b526-570e6d21c9d7\") " pod="openshift-multus/multus-additional-cni-plugins-5nrjj" Jul 06 01:20:30.432114 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.429683 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-systemd-units\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.432114 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.429709 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-host-cni-netd\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.432114 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.429740 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/68d84c01-13c1-4832-bcbf-41ad082024f6-ovn-node-metrics-cert\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.432114 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.429776 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-etc-openvswitch\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.432114 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.429816 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prlgf\" (UniqueName: \"kubernetes.io/projected/68d84c01-13c1-4832-bcbf-41ad082024f6-kube-api-access-prlgf\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.432114 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.429843 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-run-systemd\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.432114 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.429866 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hrmc\" (UniqueName: \"kubernetes.io/projected/f248c1a4-cee4-4e0b-90b8-1abec0b46b8f-kube-api-access-6hrmc\") pod \"aws-ebs-csi-driver-node-k8jjq\" (UID: \"f248c1a4-cee4-4e0b-90b8-1abec0b46b8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8jjq" Jul 06 01:20:30.432114 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.429891 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b771700e-b5fb-4e58-b526-570e6d21c9d7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5nrjj\" (UID: \"b771700e-b5fb-4e58-b526-570e6d21c9d7\") " pod="openshift-multus/multus-additional-cni-plugins-5nrjj" Jul 06 01:20:30.432114 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.429937 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b771700e-b5fb-4e58-b526-570e6d21c9d7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5nrjj\" (UID: \"b771700e-b5fb-4e58-b526-570e6d21c9d7\") " pod="openshift-multus/multus-additional-cni-plugins-5nrjj" Jul 06 01:20:30.432114 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.429970 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-host-run-netns\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.432114 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.430009 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b771700e-b5fb-4e58-b526-570e6d21c9d7-cnibin\") pod \"multus-additional-cni-plugins-5nrjj\" (UID: \"b771700e-b5fb-4e58-b526-570e6d21c9d7\") " pod="openshift-multus/multus-additional-cni-plugins-5nrjj" Jul 06 01:20:30.432114 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.430032 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48qkb\" (UniqueName: \"kubernetes.io/projected/b771700e-b5fb-4e58-b526-570e6d21c9d7-kube-api-access-48qkb\") pod \"multus-additional-cni-plugins-5nrjj\" (UID: \"b771700e-b5fb-4e58-b526-570e6d21c9d7\") " pod="openshift-multus/multus-additional-cni-plugins-5nrjj" Jul 06 01:20:30.432114 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.430055 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-run-openvswitch\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.432114 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.430076 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.432114 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.430097 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/68d84c01-13c1-4832-bcbf-41ad082024f6-ovnkube-config\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.432114 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.430121 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-host-slash\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.432114 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.430145 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-node-log\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.432760 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.430151 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/68d84c01-13c1-4832-bcbf-41ad082024f6-env-overrides\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.432760 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.430169 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f248c1a4-cee4-4e0b-90b8-1abec0b46b8f-socket-dir\") pod \"aws-ebs-csi-driver-node-k8jjq\" (UID: \"f248c1a4-cee4-4e0b-90b8-1abec0b46b8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8jjq" Jul 06 01:20:30.432760 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.430200 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f248c1a4-cee4-4e0b-90b8-1abec0b46b8f-registration-dir\") pod \"aws-ebs-csi-driver-node-k8jjq\" (UID: \"f248c1a4-cee4-4e0b-90b8-1abec0b46b8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8jjq" Jul 06 01:20:30.432760 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.430208 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-run-ovn\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.432760 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.430211 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b771700e-b5fb-4e58-b526-570e6d21c9d7-cni-binary-copy\") pod \"multus-additional-cni-plugins-5nrjj\" (UID: \"b771700e-b5fb-4e58-b526-570e6d21c9d7\") " pod="openshift-multus/multus-additional-cni-plugins-5nrjj" Jul 06 01:20:30.432760 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.430191 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-host-cni-bin\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.432760 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.430230 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f248c1a4-cee4-4e0b-90b8-1abec0b46b8f-sys-fs\") pod \"aws-ebs-csi-driver-node-k8jjq\" (UID: \"f248c1a4-cee4-4e0b-90b8-1abec0b46b8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8jjq" Jul 06 01:20:30.432760 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.430266 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/68d84c01-13c1-4832-bcbf-41ad082024f6-ovnkube-script-lib\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.432760 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.430286 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-var-lib-openvswitch\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.432760 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.430301 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f248c1a4-cee4-4e0b-90b8-1abec0b46b8f-sys-fs\") pod \"aws-ebs-csi-driver-node-k8jjq\" (UID: \"f248c1a4-cee4-4e0b-90b8-1abec0b46b8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8jjq" Jul 06 01:20:30.432760 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.430335 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f248c1a4-cee4-4e0b-90b8-1abec0b46b8f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k8jjq\" (UID: \"f248c1a4-cee4-4e0b-90b8-1abec0b46b8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8jjq" Jul 06 01:20:30.432760 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.430421 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f248c1a4-cee4-4e0b-90b8-1abec0b46b8f-etc-selinux\") pod \"aws-ebs-csi-driver-node-k8jjq\" (UID: \"f248c1a4-cee4-4e0b-90b8-1abec0b46b8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8jjq" Jul 06 01:20:30.432760 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.430566 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f248c1a4-cee4-4e0b-90b8-1abec0b46b8f-socket-dir\") pod \"aws-ebs-csi-driver-node-k8jjq\" (UID: \"f248c1a4-cee4-4e0b-90b8-1abec0b46b8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8jjq" Jul 06 01:20:30.432760 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.430630 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-host-slash\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.432760 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.430679 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-node-log\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.432760 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.430747 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9b90df9d-7dc5-41c1-a529-7cfdc8a41d09-konnectivity-ca\") pod \"konnectivity-agent-k46tr\" (UID: \"9b90df9d-7dc5-41c1-a529-7cfdc8a41d09\") " pod="kube-system/konnectivity-agent-k46tr" Jul 06 01:20:30.432760 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.430796 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-host-run-ovn-kubernetes\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.433406 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.430852 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b771700e-b5fb-4e58-b526-570e6d21c9d7-os-release\") pod \"multus-additional-cni-plugins-5nrjj\" (UID: \"b771700e-b5fb-4e58-b526-570e6d21c9d7\") " pod="openshift-multus/multus-additional-cni-plugins-5nrjj" Jul 06 01:20:30.433406 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.430881 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/68d84c01-13c1-4832-bcbf-41ad082024f6-ovnkube-script-lib\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.433406 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.430910 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b771700e-b5fb-4e58-b526-570e6d21c9d7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5nrjj\" (UID: \"b771700e-b5fb-4e58-b526-570e6d21c9d7\") " pod="openshift-multus/multus-additional-cni-plugins-5nrjj" Jul 06 01:20:30.433406 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.430953 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/68d84c01-13c1-4832-bcbf-41ad082024f6-ovnkube-config\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.433406 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.430993 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f248c1a4-cee4-4e0b-90b8-1abec0b46b8f-device-dir\") pod \"aws-ebs-csi-driver-node-k8jjq\" (UID: \"f248c1a4-cee4-4e0b-90b8-1abec0b46b8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8jjq" Jul 06 01:20:30.433406 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.431010 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-run-openvswitch\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.433406 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.431040 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-etc-openvswitch\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.433406 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.431055 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.433406 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.431093 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-run-systemd\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.433406 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.431111 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f248c1a4-cee4-4e0b-90b8-1abec0b46b8f-registration-dir\") pod \"aws-ebs-csi-driver-node-k8jjq\" (UID: \"f248c1a4-cee4-4e0b-90b8-1abec0b46b8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8jjq" Jul 06 01:20:30.433406 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.431102 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-systemd-units\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.433406 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.431144 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-host-cni-netd\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.433406 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.431158 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b771700e-b5fb-4e58-b526-570e6d21c9d7-cnibin\") pod \"multus-additional-cni-plugins-5nrjj\" (UID: \"b771700e-b5fb-4e58-b526-570e6d21c9d7\") " pod="openshift-multus/multus-additional-cni-plugins-5nrjj" Jul 06 01:20:30.433406 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.431162 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-host-run-netns\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.433406 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.431190 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-host-kubelet\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.433406 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.431196 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f248c1a4-cee4-4e0b-90b8-1abec0b46b8f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k8jjq\" (UID: \"f248c1a4-cee4-4e0b-90b8-1abec0b46b8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8jjq" Jul 06 01:20:30.433406 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.431260 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b771700e-b5fb-4e58-b526-570e6d21c9d7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5nrjj\" (UID: \"b771700e-b5fb-4e58-b526-570e6d21c9d7\") " pod="openshift-multus/multus-additional-cni-plugins-5nrjj" Jul 06 01:20:30.434417 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.431307 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/68d84c01-13c1-4832-bcbf-41ad082024f6-host-kubelet\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.434417 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.432239 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9b90df9d-7dc5-41c1-a529-7cfdc8a41d09-agent-certs\") pod \"konnectivity-agent-k46tr\" (UID: \"9b90df9d-7dc5-41c1-a529-7cfdc8a41d09\") " pod="kube-system/konnectivity-agent-k46tr" Jul 06 01:20:30.434417 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.433935 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/68d84c01-13c1-4832-bcbf-41ad082024f6-ovn-node-metrics-cert\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.439558 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.439529 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-48qkb\" (UniqueName: \"kubernetes.io/projected/b771700e-b5fb-4e58-b526-570e6d21c9d7-kube-api-access-48qkb\") pod \"multus-additional-cni-plugins-5nrjj\" (UID: \"b771700e-b5fb-4e58-b526-570e6d21c9d7\") " pod="openshift-multus/multus-additional-cni-plugins-5nrjj" Jul 06 01:20:30.439689 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.439630 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hrmc\" (UniqueName: \"kubernetes.io/projected/f248c1a4-cee4-4e0b-90b8-1abec0b46b8f-kube-api-access-6hrmc\") pod \"aws-ebs-csi-driver-node-k8jjq\" (UID: \"f248c1a4-cee4-4e0b-90b8-1abec0b46b8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8jjq" Jul 06 01:20:30.439689 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.439632 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-prlgf\" (UniqueName: \"kubernetes.io/projected/68d84c01-13c1-4832-bcbf-41ad082024f6-kube-api-access-prlgf\") pod \"ovnkube-node-b9krk\" (UID: \"68d84c01-13c1-4832-bcbf-41ad082024f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.520890 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.520848 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wcl9v" Jul 06 01:20:30.531982 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.531896 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" Jul 06 01:20:30.541983 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.541947 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5nr92" Jul 06 01:20:30.547166 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.547137 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-87qvv" Jul 06 01:20:30.554222 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.554195 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2s7xk" Jul 06 01:20:30.562988 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.562955 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5nrjj" Jul 06 01:20:30.570903 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.570863 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-k46tr" Jul 06 01:20:30.577886 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.577857 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:30.584733 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.584704 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8jjq" Jul 06 01:20:30.834549 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.834465 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c0a1cea-8dca-4e9d-afdb-81f874754505-original-pull-secret\") pod \"global-pull-secret-syncer-pm8wq\" (UID: \"4c0a1cea-8dca-4e9d-afdb-81f874754505\") " pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:20:30.834549 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.834529 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbe62245-715f-4565-bfa8-af7c3ab5d36a-metrics-certs\") pod \"network-metrics-daemon-drnhm\" (UID: \"fbe62245-715f-4565-bfa8-af7c3ab5d36a\") " pod="openshift-multus/network-metrics-daemon-drnhm" Jul 06 01:20:30.834779 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:30.834570 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vblkv\" (UniqueName: \"kubernetes.io/projected/8bcfcc54-ba2d-4049-9077-6cc8ce713838-kube-api-access-vblkv\") pod \"network-check-target-4ss2v\" (UID: \"8bcfcc54-ba2d-4049-9077-6cc8ce713838\") " pod="openshift-network-diagnostics/network-check-target-4ss2v" Jul 06 01:20:30.834779 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:30.834623 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 01:20:30.834779 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:30.834651 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 01:20:30.834779 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:30.834688 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 01:20:30.834779 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:30.834703 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbe62245-715f-4565-bfa8-af7c3ab5d36a-metrics-certs podName:fbe62245-715f-4565-bfa8-af7c3ab5d36a nodeName:}" failed. No retries permitted until 2026-07-06 01:20:31.834683218 +0000 UTC m=+3.976241991 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fbe62245-715f-4565-bfa8-af7c3ab5d36a-metrics-certs") pod "network-metrics-daemon-drnhm" (UID: "fbe62245-715f-4565-bfa8-af7c3ab5d36a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 01:20:30.834779 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:30.834707 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 01:20:30.834779 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:30.834719 2565 projected.go:196] Error preparing data for projected volume kube-api-access-vblkv for pod openshift-network-diagnostics/network-check-target-4ss2v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 01:20:30.834779 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:30.834720 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c0a1cea-8dca-4e9d-afdb-81f874754505-original-pull-secret podName:4c0a1cea-8dca-4e9d-afdb-81f874754505 nodeName:}" failed. No retries permitted until 2026-07-06 01:20:31.834711717 +0000 UTC m=+3.976270479 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4c0a1cea-8dca-4e9d-afdb-81f874754505-original-pull-secret") pod "global-pull-secret-syncer-pm8wq" (UID: "4c0a1cea-8dca-4e9d-afdb-81f874754505") : object "kube-system"/"original-pull-secret" not registered Jul 06 01:20:30.834779 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:30.834779 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8bcfcc54-ba2d-4049-9077-6cc8ce713838-kube-api-access-vblkv podName:8bcfcc54-ba2d-4049-9077-6cc8ce713838 nodeName:}" failed. No retries permitted until 2026-07-06 01:20:31.834760266 +0000 UTC m=+3.976319048 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vblkv" (UniqueName: "kubernetes.io/projected/8bcfcc54-ba2d-4049-9077-6cc8ce713838-kube-api-access-vblkv") pod "network-check-target-4ss2v" (UID: "8bcfcc54-ba2d-4049-9077-6cc8ce713838") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 01:20:31.204883 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:31.204717 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28f968d2_c730_436e_8bb9_413d0f88cf10.slice/crio-9f840456d97b139981a0c07299557b4caa97a7ed1c24548e9acd3b10b979c48f WatchSource:0}: Error finding container 9f840456d97b139981a0c07299557b4caa97a7ed1c24548e9acd3b10b979c48f: Status 404 returned error can't find the container with id 9f840456d97b139981a0c07299557b4caa97a7ed1c24548e9acd3b10b979c48f Jul 06 01:20:31.206508 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:31.206470 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec5262e8_19a7_4dce_bf06_4d792eb2b877.slice/crio-398aadbb6b5e634a382a11871e79985a47a5756cc70f474a33935e1c239e7c49 WatchSource:0}: Error finding container 398aadbb6b5e634a382a11871e79985a47a5756cc70f474a33935e1c239e7c49: Status 404 returned error can't find the container with id 398aadbb6b5e634a382a11871e79985a47a5756cc70f474a33935e1c239e7c49 Jul 06 01:20:31.208025 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:31.207877 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fc28ea8_a12f_4ec9_bac5_7d60cd5f7c8d.slice/crio-8967c224f25beeac2620d97ba1613f19406bdeddf5e5020f9ba80597cc2ac9f6 WatchSource:0}: Error finding container 8967c224f25beeac2620d97ba1613f19406bdeddf5e5020f9ba80597cc2ac9f6: Status 404 returned error can't find the container with id 8967c224f25beeac2620d97ba1613f19406bdeddf5e5020f9ba80597cc2ac9f6 Jul 06 01:20:31.213422 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:31.213362 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb771700e_b5fb_4e58_b526_570e6d21c9d7.slice/crio-ee9159b3f129614c998de54b337e98faa2436b2c63e229fa2e1d0cb3598fd412 WatchSource:0}: Error finding container ee9159b3f129614c998de54b337e98faa2436b2c63e229fa2e1d0cb3598fd412: Status 404 returned error can't find the container with id ee9159b3f129614c998de54b337e98faa2436b2c63e229fa2e1d0cb3598fd412 Jul 06 01:20:31.214491 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:31.214467 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32777e07_d94a_4e69_aad9_68eb154e397b.slice/crio-349b7ed241613750285e3bab137d8893e0e002a1b39c82316a7e7cfa709cdeb3 WatchSource:0}: Error finding container 349b7ed241613750285e3bab137d8893e0e002a1b39c82316a7e7cfa709cdeb3: Status 404 returned error can't find the container with id 349b7ed241613750285e3bab137d8893e0e002a1b39c82316a7e7cfa709cdeb3 Jul 06 01:20:31.215519 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:31.215441 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68d84c01_13c1_4832_bcbf_41ad082024f6.slice/crio-42dc579ce7332ccd24955d9954f26bb66c553729abc72f173327f90965e470d8 WatchSource:0}: Error finding container 42dc579ce7332ccd24955d9954f26bb66c553729abc72f173327f90965e470d8: Status 404 returned error can't find the container with id 42dc579ce7332ccd24955d9954f26bb66c553729abc72f173327f90965e470d8 Jul 06 01:20:31.216945 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:20:31.216807 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b90df9d_7dc5_41c1_a529_7cfdc8a41d09.slice/crio-05d17fe6dc6e541714b699b87eba442fee5bb528cf7dbfb28dcf5aeac250a22a WatchSource:0}: Error finding container 05d17fe6dc6e541714b699b87eba442fee5bb528cf7dbfb28dcf5aeac250a22a: Status 404 returned error can't find the container with id 05d17fe6dc6e541714b699b87eba442fee5bb528cf7dbfb28dcf5aeac250a22a Jul 06 01:20:31.258138 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:31.258110 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 01:15:30 +0000 UTC" deadline="2028-03-21 21:12:42.4552912 +0000 UTC" Jul 06 01:20:31.258201 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:31.258140 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14995h52m11.197154902s" Jul 06 01:20:31.331587 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:31.331552 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-159.ec2.internal" event={"ID":"008efb678643724ebfdbed72c5411e31","Type":"ContainerStarted","Data":"f0c34c7b4f35ee80a36fbadb4263777fe6308887c1c99c050e9363c9c5ad7dca"} Jul 06 01:20:31.334348 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:31.334317 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wcl9v" event={"ID":"32777e07-d94a-4e69-aad9-68eb154e397b","Type":"ContainerStarted","Data":"349b7ed241613750285e3bab137d8893e0e002a1b39c82316a7e7cfa709cdeb3"} Jul 06 01:20:31.335432 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:31.335406 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-k46tr" event={"ID":"9b90df9d-7dc5-41c1-a529-7cfdc8a41d09","Type":"ContainerStarted","Data":"05d17fe6dc6e541714b699b87eba442fee5bb528cf7dbfb28dcf5aeac250a22a"} Jul 06 01:20:31.336534 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:31.336503 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5nrjj" event={"ID":"b771700e-b5fb-4e58-b526-570e6d21c9d7","Type":"ContainerStarted","Data":"ee9159b3f129614c998de54b337e98faa2436b2c63e229fa2e1d0cb3598fd412"} Jul 06 01:20:31.337657 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:31.337628 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-87qvv" event={"ID":"483f8b7d-8c78-4edb-b992-d448f8e1f2ca","Type":"ContainerStarted","Data":"a06bad290e4d01ffed6a1e7e68abebebe3e8d92cdbcbb646c6a7a6394717b484"} Jul 06 01:20:31.339519 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:31.339489 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8jjq" event={"ID":"f248c1a4-cee4-4e0b-90b8-1abec0b46b8f","Type":"ContainerStarted","Data":"db86bf66c5d73f9e6d76abd5d2fe35af252962bdf92ccb3d7521d31e9a5b901a"} Jul 06 01:20:31.344181 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:31.344148 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" event={"ID":"68d84c01-13c1-4832-bcbf-41ad082024f6","Type":"ContainerStarted","Data":"42dc579ce7332ccd24955d9954f26bb66c553729abc72f173327f90965e470d8"} Jul 06 01:20:31.345343 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:31.345310 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5nr92" event={"ID":"ec5262e8-19a7-4dce-bf06-4d792eb2b877","Type":"ContainerStarted","Data":"398aadbb6b5e634a382a11871e79985a47a5756cc70f474a33935e1c239e7c49"} Jul 06 01:20:31.345985 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:31.345933 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-159.ec2.internal" podStartSLOduration=2.345920386 podStartE2EDuration="2.345920386s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:20:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:20:31.345750192 +0000 UTC m=+3.487308990" watchObservedRunningTime="2026-07-06 01:20:31.345920386 +0000 UTC m=+3.487479170" Jul 06 01:20:31.346451 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:31.346412 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" event={"ID":"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d","Type":"ContainerStarted","Data":"8967c224f25beeac2620d97ba1613f19406bdeddf5e5020f9ba80597cc2ac9f6"} Jul 06 01:20:31.348893 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:31.348866 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2s7xk" event={"ID":"28f968d2-c730-436e-8bb9-413d0f88cf10","Type":"ContainerStarted","Data":"9f840456d97b139981a0c07299557b4caa97a7ed1c24548e9acd3b10b979c48f"} Jul 06 01:20:31.842400 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:31.841696 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbe62245-715f-4565-bfa8-af7c3ab5d36a-metrics-certs\") pod \"network-metrics-daemon-drnhm\" (UID: \"fbe62245-715f-4565-bfa8-af7c3ab5d36a\") " pod="openshift-multus/network-metrics-daemon-drnhm" Jul 06 01:20:31.842400 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:31.841759 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vblkv\" (UniqueName: \"kubernetes.io/projected/8bcfcc54-ba2d-4049-9077-6cc8ce713838-kube-api-access-vblkv\") pod \"network-check-target-4ss2v\" (UID: \"8bcfcc54-ba2d-4049-9077-6cc8ce713838\") " pod="openshift-network-diagnostics/network-check-target-4ss2v" Jul 06 01:20:31.842400 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:31.841791 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c0a1cea-8dca-4e9d-afdb-81f874754505-original-pull-secret\") pod \"global-pull-secret-syncer-pm8wq\" (UID: \"4c0a1cea-8dca-4e9d-afdb-81f874754505\") " pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:20:31.842400 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:31.841865 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 01:20:31.842400 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:31.841909 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 01:20:31.842400 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:31.841948 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbe62245-715f-4565-bfa8-af7c3ab5d36a-metrics-certs podName:fbe62245-715f-4565-bfa8-af7c3ab5d36a nodeName:}" failed. No retries permitted until 2026-07-06 01:20:33.84192672 +0000 UTC m=+5.983485489 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fbe62245-715f-4565-bfa8-af7c3ab5d36a-metrics-certs") pod "network-metrics-daemon-drnhm" (UID: "fbe62245-715f-4565-bfa8-af7c3ab5d36a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 01:20:31.842400 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:31.841962 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 01:20:31.842400 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:31.841971 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c0a1cea-8dca-4e9d-afdb-81f874754505-original-pull-secret podName:4c0a1cea-8dca-4e9d-afdb-81f874754505 nodeName:}" failed. No retries permitted until 2026-07-06 01:20:33.841959008 +0000 UTC m=+5.983517780 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4c0a1cea-8dca-4e9d-afdb-81f874754505-original-pull-secret") pod "global-pull-secret-syncer-pm8wq" (UID: "4c0a1cea-8dca-4e9d-afdb-81f874754505") : object "kube-system"/"original-pull-secret" not registered Jul 06 01:20:31.842400 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:31.841985 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 01:20:31.842400 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:31.842001 2565 projected.go:196] Error preparing data for projected volume kube-api-access-vblkv for pod openshift-network-diagnostics/network-check-target-4ss2v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 01:20:31.842400 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:31.842048 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8bcfcc54-ba2d-4049-9077-6cc8ce713838-kube-api-access-vblkv podName:8bcfcc54-ba2d-4049-9077-6cc8ce713838 nodeName:}" failed. No retries permitted until 2026-07-06 01:20:33.842032878 +0000 UTC m=+5.983591644 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-vblkv" (UniqueName: "kubernetes.io/projected/8bcfcc54-ba2d-4049-9077-6cc8ce713838-kube-api-access-vblkv") pod "network-check-target-4ss2v" (UID: "8bcfcc54-ba2d-4049-9077-6cc8ce713838") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 01:20:32.258927 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:32.258779 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 01:15:30 +0000 UTC" deadline="2027-12-22 05:29:24.658433154 +0000 UTC" Jul 06 01:20:32.258927 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:32.258844 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12820h8m52.399593163s" Jul 06 01:20:32.319553 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:32.319523 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-drnhm" Jul 06 01:20:32.319736 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:32.319523 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ss2v" Jul 06 01:20:32.319736 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:32.319673 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-drnhm" podUID="fbe62245-715f-4565-bfa8-af7c3ab5d36a" Jul 06 01:20:32.319736 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:32.319527 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:20:32.319893 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:32.319823 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pm8wq" podUID="4c0a1cea-8dca-4e9d-afdb-81f874754505" Jul 06 01:20:32.319893 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:32.319702 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ss2v" podUID="8bcfcc54-ba2d-4049-9077-6cc8ce713838" Jul 06 01:20:32.361302 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:32.361260 2565 generic.go:356] "Generic (PLEG): container finished" podID="af723d296c5644a068a39b9f14c5579d" containerID="505394385bc6419880a2c1d8a4e2bad1ffc2ce1f2a46670598a321c5323373c1" exitCode=0 Jul 06 01:20:32.361936 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:32.361908 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-159.ec2.internal" event={"ID":"af723d296c5644a068a39b9f14c5579d","Type":"ContainerDied","Data":"505394385bc6419880a2c1d8a4e2bad1ffc2ce1f2a46670598a321c5323373c1"} Jul 06 01:20:33.385004 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:33.384952 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-159.ec2.internal" event={"ID":"af723d296c5644a068a39b9f14c5579d","Type":"ContainerStarted","Data":"7eb8746436f15400ab251a5f96eee40a4a4004186c8d3d02b469665cbd758801"} Jul 06 01:20:33.400696 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:33.400625 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-159.ec2.internal" podStartSLOduration=4.40060167 podStartE2EDuration="4.40060167s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:20:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:20:33.400065045 +0000 UTC m=+5.541623831" watchObservedRunningTime="2026-07-06 01:20:33.40060167 +0000 UTC m=+5.542160456" Jul 06 01:20:33.860073 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:33.859981 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbe62245-715f-4565-bfa8-af7c3ab5d36a-metrics-certs\") pod \"network-metrics-daemon-drnhm\" (UID: \"fbe62245-715f-4565-bfa8-af7c3ab5d36a\") " pod="openshift-multus/network-metrics-daemon-drnhm" Jul 06 01:20:33.860073 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:33.860060 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vblkv\" (UniqueName: \"kubernetes.io/projected/8bcfcc54-ba2d-4049-9077-6cc8ce713838-kube-api-access-vblkv\") pod \"network-check-target-4ss2v\" (UID: \"8bcfcc54-ba2d-4049-9077-6cc8ce713838\") " pod="openshift-network-diagnostics/network-check-target-4ss2v" Jul 06 01:20:33.860286 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:33.860097 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c0a1cea-8dca-4e9d-afdb-81f874754505-original-pull-secret\") pod \"global-pull-secret-syncer-pm8wq\" (UID: \"4c0a1cea-8dca-4e9d-afdb-81f874754505\") " pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:20:33.860286 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:33.860250 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 01:20:33.860407 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:33.860318 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c0a1cea-8dca-4e9d-afdb-81f874754505-original-pull-secret podName:4c0a1cea-8dca-4e9d-afdb-81f874754505 nodeName:}" failed. No retries permitted until 2026-07-06 01:20:37.860298135 +0000 UTC m=+10.001856901 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4c0a1cea-8dca-4e9d-afdb-81f874754505-original-pull-secret") pod "global-pull-secret-syncer-pm8wq" (UID: "4c0a1cea-8dca-4e9d-afdb-81f874754505") : object "kube-system"/"original-pull-secret" not registered Jul 06 01:20:33.860768 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:33.860749 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 01:20:33.860847 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:33.860811 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbe62245-715f-4565-bfa8-af7c3ab5d36a-metrics-certs podName:fbe62245-715f-4565-bfa8-af7c3ab5d36a nodeName:}" failed. No retries permitted until 2026-07-06 01:20:37.860791423 +0000 UTC m=+10.002350189 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fbe62245-715f-4565-bfa8-af7c3ab5d36a-metrics-certs") pod "network-metrics-daemon-drnhm" (UID: "fbe62245-715f-4565-bfa8-af7c3ab5d36a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 01:20:33.860918 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:33.860892 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 01:20:33.860918 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:33.860908 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 01:20:33.861029 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:33.860921 2565 projected.go:196] Error preparing data for projected volume kube-api-access-vblkv for pod openshift-network-diagnostics/network-check-target-4ss2v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 01:20:33.861029 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:33.860959 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8bcfcc54-ba2d-4049-9077-6cc8ce713838-kube-api-access-vblkv podName:8bcfcc54-ba2d-4049-9077-6cc8ce713838 nodeName:}" failed. No retries permitted until 2026-07-06 01:20:37.860946915 +0000 UTC m=+10.002505683 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-vblkv" (UniqueName: "kubernetes.io/projected/8bcfcc54-ba2d-4049-9077-6cc8ce713838-kube-api-access-vblkv") pod "network-check-target-4ss2v" (UID: "8bcfcc54-ba2d-4049-9077-6cc8ce713838") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 01:20:34.318685 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:34.318599 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ss2v" Jul 06 01:20:34.318841 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:34.318735 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ss2v" podUID="8bcfcc54-ba2d-4049-9077-6cc8ce713838" Jul 06 01:20:34.319165 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:34.319144 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:20:34.319273 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:34.319250 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pm8wq" podUID="4c0a1cea-8dca-4e9d-afdb-81f874754505" Jul 06 01:20:34.319339 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:34.319319 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-drnhm" Jul 06 01:20:34.319459 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:34.319433 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-drnhm" podUID="fbe62245-715f-4565-bfa8-af7c3ab5d36a" Jul 06 01:20:36.318591 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:36.318551 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ss2v" Jul 06 01:20:36.318591 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:36.318584 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:20:36.319123 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:36.318633 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-drnhm" Jul 06 01:20:36.319123 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:36.318719 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ss2v" podUID="8bcfcc54-ba2d-4049-9077-6cc8ce713838" Jul 06 01:20:36.319123 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:36.319107 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-drnhm" podUID="fbe62245-715f-4565-bfa8-af7c3ab5d36a" Jul 06 01:20:36.319289 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:36.319189 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pm8wq" podUID="4c0a1cea-8dca-4e9d-afdb-81f874754505" Jul 06 01:20:37.896876 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:37.896613 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbe62245-715f-4565-bfa8-af7c3ab5d36a-metrics-certs\") pod \"network-metrics-daemon-drnhm\" (UID: \"fbe62245-715f-4565-bfa8-af7c3ab5d36a\") " pod="openshift-multus/network-metrics-daemon-drnhm" Jul 06 01:20:37.896876 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:37.896674 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vblkv\" (UniqueName: \"kubernetes.io/projected/8bcfcc54-ba2d-4049-9077-6cc8ce713838-kube-api-access-vblkv\") pod \"network-check-target-4ss2v\" (UID: \"8bcfcc54-ba2d-4049-9077-6cc8ce713838\") " pod="openshift-network-diagnostics/network-check-target-4ss2v" Jul 06 01:20:37.896876 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:37.896708 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c0a1cea-8dca-4e9d-afdb-81f874754505-original-pull-secret\") pod \"global-pull-secret-syncer-pm8wq\" (UID: \"4c0a1cea-8dca-4e9d-afdb-81f874754505\") " pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:20:37.896876 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:37.896788 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 01:20:37.896876 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:37.896818 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 01:20:37.896876 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:37.896833 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 01:20:37.896876 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:37.896853 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 01:20:37.896876 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:37.896863 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbe62245-715f-4565-bfa8-af7c3ab5d36a-metrics-certs podName:fbe62245-715f-4565-bfa8-af7c3ab5d36a nodeName:}" failed. No retries permitted until 2026-07-06 01:20:45.896841288 +0000 UTC m=+18.038400071 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fbe62245-715f-4565-bfa8-af7c3ab5d36a-metrics-certs") pod "network-metrics-daemon-drnhm" (UID: "fbe62245-715f-4565-bfa8-af7c3ab5d36a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 01:20:37.896876 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:37.896866 2565 projected.go:196] Error preparing data for projected volume kube-api-access-vblkv for pod openshift-network-diagnostics/network-check-target-4ss2v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 01:20:37.896876 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:37.896882 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c0a1cea-8dca-4e9d-afdb-81f874754505-original-pull-secret podName:4c0a1cea-8dca-4e9d-afdb-81f874754505 nodeName:}" failed. No retries permitted until 2026-07-06 01:20:45.896872041 +0000 UTC m=+18.038430810 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4c0a1cea-8dca-4e9d-afdb-81f874754505-original-pull-secret") pod "global-pull-secret-syncer-pm8wq" (UID: "4c0a1cea-8dca-4e9d-afdb-81f874754505") : object "kube-system"/"original-pull-secret" not registered Jul 06 01:20:37.897888 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:37.896904 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8bcfcc54-ba2d-4049-9077-6cc8ce713838-kube-api-access-vblkv podName:8bcfcc54-ba2d-4049-9077-6cc8ce713838 nodeName:}" failed. No retries permitted until 2026-07-06 01:20:45.896895077 +0000 UTC m=+18.038453841 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-vblkv" (UniqueName: "kubernetes.io/projected/8bcfcc54-ba2d-4049-9077-6cc8ce713838-kube-api-access-vblkv") pod "network-check-target-4ss2v" (UID: "8bcfcc54-ba2d-4049-9077-6cc8ce713838") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 01:20:38.319162 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:38.319068 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ss2v" Jul 06 01:20:38.319340 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:38.319180 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ss2v" podUID="8bcfcc54-ba2d-4049-9077-6cc8ce713838" Jul 06 01:20:38.319922 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:38.319618 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:20:38.319922 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:38.319719 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pm8wq" podUID="4c0a1cea-8dca-4e9d-afdb-81f874754505" Jul 06 01:20:38.319922 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:38.319780 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-drnhm" Jul 06 01:20:38.319922 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:38.319851 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-drnhm" podUID="fbe62245-715f-4565-bfa8-af7c3ab5d36a" Jul 06 01:20:40.318822 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:40.318783 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-drnhm" Jul 06 01:20:40.319214 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:40.318933 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-drnhm" podUID="fbe62245-715f-4565-bfa8-af7c3ab5d36a" Jul 06 01:20:40.319288 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:40.319269 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ss2v" Jul 06 01:20:40.319361 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:40.319334 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ss2v" podUID="8bcfcc54-ba2d-4049-9077-6cc8ce713838" Jul 06 01:20:40.319505 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:40.319419 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:20:40.319505 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:40.319496 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pm8wq" podUID="4c0a1cea-8dca-4e9d-afdb-81f874754505" Jul 06 01:20:42.318714 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:42.318674 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ss2v" Jul 06 01:20:42.319144 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:42.318674 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:20:42.319144 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:42.318809 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ss2v" podUID="8bcfcc54-ba2d-4049-9077-6cc8ce713838" Jul 06 01:20:42.319144 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:42.318902 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pm8wq" podUID="4c0a1cea-8dca-4e9d-afdb-81f874754505" Jul 06 01:20:42.319144 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:42.318692 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-drnhm" Jul 06 01:20:42.319144 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:42.319014 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-drnhm" podUID="fbe62245-715f-4565-bfa8-af7c3ab5d36a" Jul 06 01:20:44.318124 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:44.318086 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ss2v" Jul 06 01:20:44.318498 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:44.318140 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:20:44.318498 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:44.318155 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-drnhm" Jul 06 01:20:44.318498 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:44.318243 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pm8wq" podUID="4c0a1cea-8dca-4e9d-afdb-81f874754505" Jul 06 01:20:44.318498 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:44.318287 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-drnhm" podUID="fbe62245-715f-4565-bfa8-af7c3ab5d36a" Jul 06 01:20:44.318498 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:44.318387 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ss2v" podUID="8bcfcc54-ba2d-4049-9077-6cc8ce713838" Jul 06 01:20:45.958823 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:45.958775 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbe62245-715f-4565-bfa8-af7c3ab5d36a-metrics-certs\") pod \"network-metrics-daemon-drnhm\" (UID: \"fbe62245-715f-4565-bfa8-af7c3ab5d36a\") " pod="openshift-multus/network-metrics-daemon-drnhm" Jul 06 01:20:45.959343 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:45.958845 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vblkv\" (UniqueName: \"kubernetes.io/projected/8bcfcc54-ba2d-4049-9077-6cc8ce713838-kube-api-access-vblkv\") pod \"network-check-target-4ss2v\" (UID: \"8bcfcc54-ba2d-4049-9077-6cc8ce713838\") " pod="openshift-network-diagnostics/network-check-target-4ss2v" Jul 06 01:20:45.959343 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:45.958876 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c0a1cea-8dca-4e9d-afdb-81f874754505-original-pull-secret\") pod \"global-pull-secret-syncer-pm8wq\" (UID: \"4c0a1cea-8dca-4e9d-afdb-81f874754505\") " pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:20:45.959343 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:45.958956 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 01:20:45.959343 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:45.958993 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 01:20:45.959343 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:45.959002 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 01:20:45.959343 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:45.959025 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 01:20:45.959343 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:45.959038 2565 projected.go:196] Error preparing data for projected volume kube-api-access-vblkv for pod openshift-network-diagnostics/network-check-target-4ss2v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 01:20:45.959343 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:45.959040 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbe62245-715f-4565-bfa8-af7c3ab5d36a-metrics-certs podName:fbe62245-715f-4565-bfa8-af7c3ab5d36a nodeName:}" failed. No retries permitted until 2026-07-06 01:21:01.95901809 +0000 UTC m=+34.100576878 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fbe62245-715f-4565-bfa8-af7c3ab5d36a-metrics-certs") pod "network-metrics-daemon-drnhm" (UID: "fbe62245-715f-4565-bfa8-af7c3ab5d36a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 01:20:45.959343 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:45.959059 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c0a1cea-8dca-4e9d-afdb-81f874754505-original-pull-secret podName:4c0a1cea-8dca-4e9d-afdb-81f874754505 nodeName:}" failed. No retries permitted until 2026-07-06 01:21:01.959050446 +0000 UTC m=+34.100609216 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4c0a1cea-8dca-4e9d-afdb-81f874754505-original-pull-secret") pod "global-pull-secret-syncer-pm8wq" (UID: "4c0a1cea-8dca-4e9d-afdb-81f874754505") : object "kube-system"/"original-pull-secret" not registered Jul 06 01:20:45.959343 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:45.959074 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8bcfcc54-ba2d-4049-9077-6cc8ce713838-kube-api-access-vblkv podName:8bcfcc54-ba2d-4049-9077-6cc8ce713838 nodeName:}" failed. No retries permitted until 2026-07-06 01:21:01.959065329 +0000 UTC m=+34.100624092 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-vblkv" (UniqueName: "kubernetes.io/projected/8bcfcc54-ba2d-4049-9077-6cc8ce713838-kube-api-access-vblkv") pod "network-check-target-4ss2v" (UID: "8bcfcc54-ba2d-4049-9077-6cc8ce713838") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 01:20:46.318149 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:46.318065 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:20:46.318331 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:46.318068 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-drnhm" Jul 06 01:20:46.318331 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:46.318202 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pm8wq" podUID="4c0a1cea-8dca-4e9d-afdb-81f874754505" Jul 06 01:20:46.318331 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:46.318068 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ss2v" Jul 06 01:20:46.318331 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:46.318273 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-drnhm" podUID="fbe62245-715f-4565-bfa8-af7c3ab5d36a" Jul 06 01:20:46.318543 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:46.318332 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ss2v" podUID="8bcfcc54-ba2d-4049-9077-6cc8ce713838" Jul 06 01:20:48.319014 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:48.318976 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ss2v" Jul 06 01:20:48.319507 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:48.319033 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:20:48.319507 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:48.319107 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ss2v" podUID="8bcfcc54-ba2d-4049-9077-6cc8ce713838" Jul 06 01:20:48.319507 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:48.319141 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-drnhm" Jul 06 01:20:48.319507 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:48.319220 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-drnhm" podUID="fbe62245-715f-4565-bfa8-af7c3ab5d36a" Jul 06 01:20:48.319507 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:48.319215 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pm8wq" podUID="4c0a1cea-8dca-4e9d-afdb-81f874754505" Jul 06 01:20:50.318217 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:50.318180 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ss2v" Jul 06 01:20:50.318858 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:50.318301 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-drnhm" Jul 06 01:20:50.318858 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:50.318306 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ss2v" podUID="8bcfcc54-ba2d-4049-9077-6cc8ce713838" Jul 06 01:20:50.318858 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:50.318187 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:20:50.318858 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:50.318425 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-drnhm" podUID="fbe62245-715f-4565-bfa8-af7c3ab5d36a" Jul 06 01:20:50.318858 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:50.318484 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pm8wq" podUID="4c0a1cea-8dca-4e9d-afdb-81f874754505" Jul 06 01:20:51.440884 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:51.440459 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-k46tr" event={"ID":"9b90df9d-7dc5-41c1-a529-7cfdc8a41d09","Type":"ContainerStarted","Data":"858bc542085a26f627d6460a5dd37669c15f1eafd25f448c3361754b6508102c"} Jul 06 01:20:51.442051 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:51.442028 2565 generic.go:356] "Generic (PLEG): container finished" podID="b771700e-b5fb-4e58-b526-570e6d21c9d7" containerID="ba0091694dbf270c3f342e0bf7d39958f01c578a4d27755da1d11bc86c752fb0" exitCode=0 Jul 06 01:20:51.442133 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:51.442105 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5nrjj" event={"ID":"b771700e-b5fb-4e58-b526-570e6d21c9d7","Type":"ContainerDied","Data":"ba0091694dbf270c3f342e0bf7d39958f01c578a4d27755da1d11bc86c752fb0"} Jul 06 01:20:51.443619 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:51.443586 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-87qvv" event={"ID":"483f8b7d-8c78-4edb-b992-d448f8e1f2ca","Type":"ContainerStarted","Data":"0136e9d6ed06235e096fe3bba0c5fcfd753b203186d261946e82dcf93272b8ef"} Jul 06 01:20:51.445057 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:51.445021 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8jjq" event={"ID":"f248c1a4-cee4-4e0b-90b8-1abec0b46b8f","Type":"ContainerStarted","Data":"568697aadd7101b745dd32d9d6816f2e22f561ed56a049bb6c04f76a75333ca1"} Jul 06 01:20:51.449946 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:51.449915 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" event={"ID":"68d84c01-13c1-4832-bcbf-41ad082024f6","Type":"ContainerStarted","Data":"73f3a4497e0b79aa9f461f7412b04c6857a871e28a16b66837ada6cdeb0c18de"} Jul 06 01:20:51.450060 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:51.450016 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" event={"ID":"68d84c01-13c1-4832-bcbf-41ad082024f6","Type":"ContainerStarted","Data":"c63b1899715c50bfe6c937a5979d0392184697104da6ae699a656ed20c210b34"} Jul 06 01:20:51.450104 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:51.450070 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" event={"ID":"68d84c01-13c1-4832-bcbf-41ad082024f6","Type":"ContainerStarted","Data":"93768df1c7d38804264e4dc674ab40002f7b3cd7fea7d46fb544ba706c819051"} Jul 06 01:20:51.450141 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:51.450119 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" event={"ID":"68d84c01-13c1-4832-bcbf-41ad082024f6","Type":"ContainerStarted","Data":"6918bb29c8cf1d5f0b1b027429fbdd959099a01fd369cdbaca8bf30921ec220f"} Jul 06 01:20:51.450190 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:51.450152 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" event={"ID":"68d84c01-13c1-4832-bcbf-41ad082024f6","Type":"ContainerStarted","Data":"f0bc86ec8aa1dc3701cd81443a9e44ce61bb0847c9eb638efa84be0849e0fa8c"} Jul 06 01:20:51.450190 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:51.450184 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" event={"ID":"68d84c01-13c1-4832-bcbf-41ad082024f6","Type":"ContainerStarted","Data":"7dee367bd7be629699ef8603e5e65549c7aa5a26b17d6a480d80f9df4d49fde0"} Jul 06 01:20:51.451269 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:51.451249 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5nr92" event={"ID":"ec5262e8-19a7-4dce-bf06-4d792eb2b877","Type":"ContainerStarted","Data":"8fd7e9f6e51bf6a38d350f962763efc82a81796a5805e92ef5562e49b8c0dab7"} Jul 06 01:20:51.453083 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:51.453058 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" event={"ID":"1fc28ea8-a12f-4ec9-bac5-7d60cd5f7c8d","Type":"ContainerStarted","Data":"19ba5586c232bd2260e402919397ba4778ad82b3eb1cd3253c10d0cd38c1ef75"} Jul 06 01:20:51.454712 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:51.454688 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2s7xk" event={"ID":"28f968d2-c730-436e-8bb9-413d0f88cf10","Type":"ContainerStarted","Data":"0eaf31c02f0010c7eba1a52c242319d8ccc86aa518e530ec745c678bea7b5811"} Jul 06 01:20:51.455101 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:51.455065 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-k46tr" podStartSLOduration=8.038960553 podStartE2EDuration="23.455048041s" totalImagesPullingTime="15.416087488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:20:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:20:51.454121611 +0000 UTC m=+23.595680397" watchObservedRunningTime="2026-07-06 01:20:51.455048041 +0000 UTC m=+23.596606828" Jul 06 01:20:51.467305 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:51.467249 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5nr92" podStartSLOduration=4.326164607 podStartE2EDuration="23.467234377s" totalImagesPullingTime="19.14106977s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:20:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:20:51.466697084 +0000 UTC m=+23.608255871" watchObservedRunningTime="2026-07-06 01:20:51.467234377 +0000 UTC m=+23.608793161" Jul 06 01:20:51.482052 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:51.482013 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-k46tr" Jul 06 01:20:51.482698 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:51.482677 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-k46tr" Jul 06 01:20:51.505247 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:51.505195 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-lzmx9" podStartSLOduration=4.362357769 podStartE2EDuration="23.505178706s" totalImagesPullingTime="19.142820937s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:20:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:20:51.505029331 +0000 UTC m=+23.646588129" watchObservedRunningTime="2026-07-06 01:20:51.505178706 +0000 UTC m=+23.646737497" Jul 06 01:20:51.522127 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:51.522075 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-87qvv" podStartSLOduration=4.284183615 podStartE2EDuration="23.522058497s" totalImagesPullingTime="19.237874882s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:20:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:20:51.52178701 +0000 UTC m=+23.663345795" watchObservedRunningTime="2026-07-06 01:20:51.522058497 +0000 UTC m=+23.663617281" Jul 06 01:20:51.555740 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:51.555681 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-2s7xk" podStartSLOduration=4.385344007 podStartE2EDuration="23.555662066s" totalImagesPullingTime="19.170318059s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:20:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:20:51.555404648 +0000 UTC m=+23.696963446" watchObservedRunningTime="2026-07-06 01:20:51.555662066 +0000 UTC m=+23.697220850" Jul 06 01:20:51.800801 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:51.800773 2565 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 01:20:52.275571 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:52.275457 2565 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T01:20:51.800793783Z","UUID":"80fac108-3116-41ad-9002-a957b50adf35","Handler":null,"Name":"","Endpoint":""} Jul 06 01:20:52.277690 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:52.277659 2565 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 01:20:52.277836 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:52.277700 2565 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 01:20:52.318230 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:52.318197 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ss2v" Jul 06 01:20:52.318443 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:52.318319 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ss2v" podUID="8bcfcc54-ba2d-4049-9077-6cc8ce713838" Jul 06 01:20:52.318443 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:52.318346 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-drnhm" Jul 06 01:20:52.318564 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:52.318481 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-drnhm" podUID="fbe62245-715f-4565-bfa8-af7c3ab5d36a" Jul 06 01:20:52.318564 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:52.318551 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:20:52.318674 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:52.318621 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pm8wq" podUID="4c0a1cea-8dca-4e9d-afdb-81f874754505" Jul 06 01:20:52.459994 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:52.459959 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8jjq" event={"ID":"f248c1a4-cee4-4e0b-90b8-1abec0b46b8f","Type":"ContainerStarted","Data":"f9fffec1385e421d483a12803bd80470c2e2045df88aa48170ea834c49b4c079"} Jul 06 01:20:52.461837 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:52.461801 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wcl9v" event={"ID":"32777e07-d94a-4e69-aad9-68eb154e397b","Type":"ContainerStarted","Data":"6cd1751b1b87eae822217f11b97c09f03808c0063edccef3995d3ed320236c38"} Jul 06 01:20:52.477529 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:52.477472 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wcl9v" podStartSLOduration=5.340795801 podStartE2EDuration="24.47745206s" totalImagesPullingTime="19.136656259s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:20:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:20:52.477230872 +0000 UTC m=+24.618789657" watchObservedRunningTime="2026-07-06 01:20:52.47745206 +0000 UTC m=+24.619010846" Jul 06 01:20:53.466578 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:53.466293 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8jjq" event={"ID":"f248c1a4-cee4-4e0b-90b8-1abec0b46b8f","Type":"ContainerStarted","Data":"eb17517669ff1d719528b971b3666dea7fe6665136ca6d293e0f9ddcad648362"} Jul 06 01:20:53.470241 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:53.470212 2565 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 06 01:20:53.470241 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:53.470217 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" event={"ID":"68d84c01-13c1-4832-bcbf-41ad082024f6","Type":"ContainerStarted","Data":"dd7631601caf73aafde61ce774343727043af3027a086213587ee0cf60b069bf"} Jul 06 01:20:53.485348 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:53.485253 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8jjq" podStartSLOduration=4.118515363 podStartE2EDuration="25.485235304s" totalImagesPullingTime="21.366719941s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:20:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:20:53.484849123 +0000 UTC m=+25.626407924" watchObservedRunningTime="2026-07-06 01:20:53.485235304 +0000 UTC m=+25.626794129" Jul 06 01:20:54.318219 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:54.318182 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ss2v" Jul 06 01:20:54.318219 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:54.318208 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-drnhm" Jul 06 01:20:54.318551 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:54.318182 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:20:54.318551 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:54.318300 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ss2v" podUID="8bcfcc54-ba2d-4049-9077-6cc8ce713838" Jul 06 01:20:54.318551 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:54.318390 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pm8wq" podUID="4c0a1cea-8dca-4e9d-afdb-81f874754505" Jul 06 01:20:54.318551 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:54.318459 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-drnhm" podUID="fbe62245-715f-4565-bfa8-af7c3ab5d36a" Jul 06 01:20:55.249822 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:55.248824 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-k46tr" Jul 06 01:20:55.249822 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:55.249053 2565 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 06 01:20:55.249822 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:55.249561 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-k46tr" Jul 06 01:20:56.320229 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:56.318494 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ss2v" Jul 06 01:20:56.320229 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:56.318511 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-drnhm" Jul 06 01:20:56.320229 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:56.318856 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ss2v" podUID="8bcfcc54-ba2d-4049-9077-6cc8ce713838" Jul 06 01:20:56.320229 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:56.318952 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-drnhm" podUID="fbe62245-715f-4565-bfa8-af7c3ab5d36a" Jul 06 01:20:56.320229 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:56.318512 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:20:56.320229 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:56.319049 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pm8wq" podUID="4c0a1cea-8dca-4e9d-afdb-81f874754505" Jul 06 01:20:56.478858 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:56.478824 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5nrjj" event={"ID":"b771700e-b5fb-4e58-b526-570e6d21c9d7","Type":"ContainerStarted","Data":"51bdaf8adf384db937236c1e20b96d708d5d8f742221c8a6e1cebd85cdb19e48"} Jul 06 01:20:56.482962 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:56.482932 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" event={"ID":"68d84c01-13c1-4832-bcbf-41ad082024f6","Type":"ContainerStarted","Data":"2b80d9b71d5a3d9eac110902a8fdce62e80e63ae6a2dfe456ba942f3cb4fe581"} Jul 06 01:20:56.483584 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:56.483567 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:56.483742 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:56.483721 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:56.502208 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:56.502160 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:56.506125 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:56.506094 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:20:56.523111 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:56.523050 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" podStartSLOduration=9.329993025 podStartE2EDuration="28.523030377s" totalImagesPullingTime="19.193037352s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:20:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:20:56.522777486 +0000 UTC m=+28.664336272" watchObservedRunningTime="2026-07-06 01:20:56.523030377 +0000 UTC m=+28.664589162" Jul 06 01:20:57.487018 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:57.486980 2565 generic.go:356] "Generic (PLEG): container finished" podID="b771700e-b5fb-4e58-b526-570e6d21c9d7" containerID="51bdaf8adf384db937236c1e20b96d708d5d8f742221c8a6e1cebd85cdb19e48" exitCode=0 Jul 06 01:20:57.487487 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:57.487081 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5nrjj" event={"ID":"b771700e-b5fb-4e58-b526-570e6d21c9d7","Type":"ContainerDied","Data":"51bdaf8adf384db937236c1e20b96d708d5d8f742221c8a6e1cebd85cdb19e48"} Jul 06 01:20:57.487487 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:57.487126 2565 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 06 01:20:58.320120 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:58.320078 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-drnhm" Jul 06 01:20:58.320278 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:58.320128 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ss2v" Jul 06 01:20:58.320278 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:58.320212 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:20:58.320278 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:58.320222 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-drnhm" podUID="fbe62245-715f-4565-bfa8-af7c3ab5d36a" Jul 06 01:20:58.320478 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:58.320292 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pm8wq" podUID="4c0a1cea-8dca-4e9d-afdb-81f874754505" Jul 06 01:20:58.320478 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:20:58.320338 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ss2v" podUID="8bcfcc54-ba2d-4049-9077-6cc8ce713838" Jul 06 01:20:58.488899 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:58.488867 2565 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 06 01:20:59.493554 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:59.493513 2565 generic.go:356] "Generic (PLEG): container finished" podID="b771700e-b5fb-4e58-b526-570e6d21c9d7" containerID="fd18d00e2bbe5e28d0535c3b3d29a6bb73b1f7b2ac016de4cd1636057e9e6911" exitCode=0 Jul 06 01:20:59.493909 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:59.493560 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5nrjj" event={"ID":"b771700e-b5fb-4e58-b526-570e6d21c9d7","Type":"ContainerDied","Data":"fd18d00e2bbe5e28d0535c3b3d29a6bb73b1f7b2ac016de4cd1636057e9e6911"} Jul 06 01:20:59.918990 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:20:59.918890 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:21:00.318704 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:00.318619 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ss2v" Jul 06 01:21:00.318949 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:00.318619 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:21:00.318949 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:00.318750 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ss2v" podUID="8bcfcc54-ba2d-4049-9077-6cc8ce713838" Jul 06 01:21:00.318949 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:00.318627 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-drnhm" Jul 06 01:21:00.318949 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:00.318830 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pm8wq" podUID="4c0a1cea-8dca-4e9d-afdb-81f874754505" Jul 06 01:21:00.318949 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:00.318888 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-drnhm" podUID="fbe62245-715f-4565-bfa8-af7c3ab5d36a" Jul 06 01:21:01.499730 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:01.499696 2565 generic.go:356] "Generic (PLEG): container finished" podID="b771700e-b5fb-4e58-b526-570e6d21c9d7" containerID="d8fa663693b2159b0dca63e3e026701df6f0382fc8be6e81cb38c454ea7177ed" exitCode=0 Jul 06 01:21:01.500116 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:01.499739 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5nrjj" event={"ID":"b771700e-b5fb-4e58-b526-570e6d21c9d7","Type":"ContainerDied","Data":"d8fa663693b2159b0dca63e3e026701df6f0382fc8be6e81cb38c454ea7177ed"} Jul 06 01:21:01.974852 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:01.974751 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbe62245-715f-4565-bfa8-af7c3ab5d36a-metrics-certs\") pod \"network-metrics-daemon-drnhm\" (UID: \"fbe62245-715f-4565-bfa8-af7c3ab5d36a\") " pod="openshift-multus/network-metrics-daemon-drnhm" Jul 06 01:21:01.974852 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:01.974813 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vblkv\" (UniqueName: \"kubernetes.io/projected/8bcfcc54-ba2d-4049-9077-6cc8ce713838-kube-api-access-vblkv\") pod \"network-check-target-4ss2v\" (UID: \"8bcfcc54-ba2d-4049-9077-6cc8ce713838\") " pod="openshift-network-diagnostics/network-check-target-4ss2v" Jul 06 01:21:01.974852 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:01.974833 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c0a1cea-8dca-4e9d-afdb-81f874754505-original-pull-secret\") pod \"global-pull-secret-syncer-pm8wq\" (UID: \"4c0a1cea-8dca-4e9d-afdb-81f874754505\") " pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:21:01.975077 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:01.974906 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 01:21:01.975077 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:01.974908 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 01:21:01.975077 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:01.974923 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 01:21:01.975077 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:01.974942 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 01:21:01.975077 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:01.974951 2565 projected.go:196] Error preparing data for projected volume kube-api-access-vblkv for pod openshift-network-diagnostics/network-check-target-4ss2v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 01:21:01.975077 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:01.974958 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c0a1cea-8dca-4e9d-afdb-81f874754505-original-pull-secret podName:4c0a1cea-8dca-4e9d-afdb-81f874754505 nodeName:}" failed. No retries permitted until 2026-07-06 01:21:33.974945245 +0000 UTC m=+66.116504009 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4c0a1cea-8dca-4e9d-afdb-81f874754505-original-pull-secret") pod "global-pull-secret-syncer-pm8wq" (UID: "4c0a1cea-8dca-4e9d-afdb-81f874754505") : object "kube-system"/"original-pull-secret" not registered Jul 06 01:21:01.975077 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:01.974970 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbe62245-715f-4565-bfa8-af7c3ab5d36a-metrics-certs podName:fbe62245-715f-4565-bfa8-af7c3ab5d36a nodeName:}" failed. No retries permitted until 2026-07-06 01:21:33.974964265 +0000 UTC m=+66.116523029 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fbe62245-715f-4565-bfa8-af7c3ab5d36a-metrics-certs") pod "network-metrics-daemon-drnhm" (UID: "fbe62245-715f-4565-bfa8-af7c3ab5d36a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 01:21:01.975077 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:01.974991 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8bcfcc54-ba2d-4049-9077-6cc8ce713838-kube-api-access-vblkv podName:8bcfcc54-ba2d-4049-9077-6cc8ce713838 nodeName:}" failed. No retries permitted until 2026-07-06 01:21:33.974978797 +0000 UTC m=+66.116537575 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-vblkv" (UniqueName: "kubernetes.io/projected/8bcfcc54-ba2d-4049-9077-6cc8ce713838-kube-api-access-vblkv") pod "network-check-target-4ss2v" (UID: "8bcfcc54-ba2d-4049-9077-6cc8ce713838") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 01:21:02.318227 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:02.318125 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:21:02.318227 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:02.318169 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-drnhm" Jul 06 01:21:02.318227 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:02.318125 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ss2v" Jul 06 01:21:02.318537 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:02.318277 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pm8wq" podUID="4c0a1cea-8dca-4e9d-afdb-81f874754505" Jul 06 01:21:02.318537 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:02.318357 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ss2v" podUID="8bcfcc54-ba2d-4049-9077-6cc8ce713838" Jul 06 01:21:02.318537 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:02.318486 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-drnhm" podUID="fbe62245-715f-4565-bfa8-af7c3ab5d36a" Jul 06 01:21:04.319061 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:04.318793 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ss2v" Jul 06 01:21:04.319602 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:04.319163 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ss2v" podUID="8bcfcc54-ba2d-4049-9077-6cc8ce713838" Jul 06 01:21:04.319602 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:04.318846 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-drnhm" Jul 06 01:21:04.319602 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:04.319273 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-drnhm" podUID="fbe62245-715f-4565-bfa8-af7c3ab5d36a" Jul 06 01:21:04.319602 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:04.318801 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:21:04.319602 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:04.319349 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pm8wq" podUID="4c0a1cea-8dca-4e9d-afdb-81f874754505" Jul 06 01:21:06.318739 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:06.318694 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ss2v" Jul 06 01:21:06.318739 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:06.318719 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-drnhm" Jul 06 01:21:06.319177 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:06.318808 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:21:06.319177 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:06.318826 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ss2v" podUID="8bcfcc54-ba2d-4049-9077-6cc8ce713838" Jul 06 01:21:06.319177 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:06.318892 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pm8wq" podUID="4c0a1cea-8dca-4e9d-afdb-81f874754505" Jul 06 01:21:06.319177 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:06.318944 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-drnhm" podUID="fbe62245-715f-4565-bfa8-af7c3ab5d36a" Jul 06 01:21:06.514639 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:06.514604 2565 generic.go:356] "Generic (PLEG): container finished" podID="b771700e-b5fb-4e58-b526-570e6d21c9d7" containerID="049694771f98f3887efad695d1cc02680c71267b988250013cd8462c57433d15" exitCode=0 Jul 06 01:21:06.514837 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:06.514664 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5nrjj" event={"ID":"b771700e-b5fb-4e58-b526-570e6d21c9d7","Type":"ContainerDied","Data":"049694771f98f3887efad695d1cc02680c71267b988250013cd8462c57433d15"} Jul 06 01:21:07.519814 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:07.519772 2565 generic.go:356] "Generic (PLEG): container finished" podID="b771700e-b5fb-4e58-b526-570e6d21c9d7" containerID="c15e83e5c1f4aaaff60f2898b6d6c666b558f341727f3275742ae36fb4917733" exitCode=0 Jul 06 01:21:07.519814 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:07.519812 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5nrjj" event={"ID":"b771700e-b5fb-4e58-b526-570e6d21c9d7","Type":"ContainerDied","Data":"c15e83e5c1f4aaaff60f2898b6d6c666b558f341727f3275742ae36fb4917733"} Jul 06 01:21:08.318668 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:08.318629 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ss2v" Jul 06 01:21:08.318867 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:08.318717 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-drnhm" Jul 06 01:21:08.318867 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:08.318730 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ss2v" podUID="8bcfcc54-ba2d-4049-9077-6cc8ce713838" Jul 06 01:21:08.318867 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:08.318795 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-drnhm" podUID="fbe62245-715f-4565-bfa8-af7c3ab5d36a" Jul 06 01:21:08.318867 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:08.318800 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:21:08.319011 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:08.318879 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pm8wq" podUID="4c0a1cea-8dca-4e9d-afdb-81f874754505" Jul 06 01:21:08.528119 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:08.528084 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5nrjj" event={"ID":"b771700e-b5fb-4e58-b526-570e6d21c9d7","Type":"ContainerStarted","Data":"adc94108059af7e64c887b7c0e80052554a6e3163bfdb7e2aefd6ebe4f4b3e40"} Jul 06 01:21:08.551609 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:08.551554 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-5nrjj" podStartSLOduration=10.197986684 podStartE2EDuration="40.551536891s" totalImagesPullingTime="30.353550207s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:20:28 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:21:08.549994554 +0000 UTC m=+40.691553339" watchObservedRunningTime="2026-07-06 01:21:08.551536891 +0000 UTC m=+40.693095677" Jul 06 01:21:10.318569 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:10.318537 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:21:10.319096 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:10.318537 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-drnhm" Jul 06 01:21:10.319096 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:10.318647 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pm8wq" podUID="4c0a1cea-8dca-4e9d-afdb-81f874754505" Jul 06 01:21:10.319096 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:10.318537 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ss2v" Jul 06 01:21:10.319096 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:10.318730 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-drnhm" podUID="fbe62245-715f-4565-bfa8-af7c3ab5d36a" Jul 06 01:21:10.319096 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:10.318802 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ss2v" podUID="8bcfcc54-ba2d-4049-9077-6cc8ce713838" Jul 06 01:21:12.318104 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:12.318067 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:21:12.318104 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:12.318096 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-drnhm" Jul 06 01:21:12.318633 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:12.318068 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ss2v" Jul 06 01:21:12.318633 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:12.318172 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pm8wq" podUID="4c0a1cea-8dca-4e9d-afdb-81f874754505" Jul 06 01:21:12.318633 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:12.318268 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ss2v" podUID="8bcfcc54-ba2d-4049-9077-6cc8ce713838" Jul 06 01:21:12.318633 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:12.318349 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-drnhm" podUID="fbe62245-715f-4565-bfa8-af7c3ab5d36a" Jul 06 01:21:14.318783 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:14.318750 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:21:14.319505 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:14.318806 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ss2v" Jul 06 01:21:14.319505 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:14.318825 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-drnhm" Jul 06 01:21:14.319505 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:14.318903 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ss2v" podUID="8bcfcc54-ba2d-4049-9077-6cc8ce713838" Jul 06 01:21:14.319505 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:14.319001 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-drnhm" podUID="fbe62245-715f-4565-bfa8-af7c3ab5d36a" Jul 06 01:21:14.319505 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:14.319089 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pm8wq" podUID="4c0a1cea-8dca-4e9d-afdb-81f874754505" Jul 06 01:21:14.764704 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:14.764662 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4ss2v"] Jul 06 01:21:14.764856 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:14.764797 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ss2v" Jul 06 01:21:14.764941 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:14.764913 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ss2v" podUID="8bcfcc54-ba2d-4049-9077-6cc8ce713838" Jul 06 01:21:14.767295 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:14.767149 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pm8wq"] Jul 06 01:21:14.767295 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:14.767264 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:21:14.767529 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:14.767394 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pm8wq" podUID="4c0a1cea-8dca-4e9d-afdb-81f874754505" Jul 06 01:21:14.767697 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:14.767671 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-drnhm"] Jul 06 01:21:14.767821 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:14.767747 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-drnhm" Jul 06 01:21:14.767881 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:14.767847 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-drnhm" podUID="fbe62245-715f-4565-bfa8-af7c3ab5d36a" Jul 06 01:21:16.318711 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:16.318673 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ss2v" Jul 06 01:21:16.319525 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:16.318673 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:21:16.319525 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:16.318784 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ss2v" podUID="8bcfcc54-ba2d-4049-9077-6cc8ce713838" Jul 06 01:21:16.319525 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:16.318860 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pm8wq" podUID="4c0a1cea-8dca-4e9d-afdb-81f874754505" Jul 06 01:21:16.319525 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:16.318673 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-drnhm" Jul 06 01:21:16.319525 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:16.318947 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-drnhm" podUID="fbe62245-715f-4565-bfa8-af7c3ab5d36a" Jul 06 01:21:18.165752 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.165552 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeReady" Jul 06 01:21:18.166126 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.165863 2565 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 01:21:18.212958 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.212839 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-tfzf9"] Jul 06 01:21:18.228471 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.228434 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5br85"] Jul 06 01:21:18.228656 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.228555 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tfzf9" Jul 06 01:21:18.230861 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.230835 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 01:21:18.231047 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.230924 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 01:21:18.231047 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.230937 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-g4lpx\"" Jul 06 01:21:18.244510 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.244480 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tfzf9"] Jul 06 01:21:18.244510 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.244515 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5br85"] Jul 06 01:21:18.244694 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.244599 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5br85" Jul 06 01:21:18.246949 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.246922 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-b9srw\"" Jul 06 01:21:18.247096 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.246956 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 01:21:18.247096 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.247005 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 01:21:18.247096 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.247072 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 01:21:18.319277 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.319234 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ss2v" Jul 06 01:21:18.319465 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.319334 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-drnhm" Jul 06 01:21:18.319535 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.319467 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:21:18.323052 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.323023 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 01:21:18.323052 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.323049 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 01:21:18.323263 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.323032 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 01:21:18.323263 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.323024 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9bqkt\"" Jul 06 01:21:18.323263 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.323228 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6kdr7\"" Jul 06 01:21:18.323463 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.323445 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 01:21:18.393229 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.393190 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/889f2b80-8fba-40de-a6f6-314a17b310df-cert\") pod \"ingress-canary-5br85\" (UID: \"889f2b80-8fba-40de-a6f6-314a17b310df\") " pod="openshift-ingress-canary/ingress-canary-5br85" Jul 06 01:21:18.393453 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.393258 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpz4c\" (UniqueName: \"kubernetes.io/projected/889f2b80-8fba-40de-a6f6-314a17b310df-kube-api-access-dpz4c\") pod \"ingress-canary-5br85\" (UID: \"889f2b80-8fba-40de-a6f6-314a17b310df\") " pod="openshift-ingress-canary/ingress-canary-5br85" Jul 06 01:21:18.393453 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.393281 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e4384477-41b5-40c5-83e5-23f7c541dfa0-metrics-tls\") pod \"dns-default-tfzf9\" (UID: \"e4384477-41b5-40c5-83e5-23f7c541dfa0\") " pod="openshift-dns/dns-default-tfzf9" Jul 06 01:21:18.393453 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.393309 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4384477-41b5-40c5-83e5-23f7c541dfa0-tmp-dir\") pod \"dns-default-tfzf9\" (UID: \"e4384477-41b5-40c5-83e5-23f7c541dfa0\") " pod="openshift-dns/dns-default-tfzf9" Jul 06 01:21:18.393453 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.393335 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tw4vx\" (UniqueName: \"kubernetes.io/projected/e4384477-41b5-40c5-83e5-23f7c541dfa0-kube-api-access-tw4vx\") pod \"dns-default-tfzf9\" (UID: \"e4384477-41b5-40c5-83e5-23f7c541dfa0\") " pod="openshift-dns/dns-default-tfzf9" Jul 06 01:21:18.393453 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.393396 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e4384477-41b5-40c5-83e5-23f7c541dfa0-config-volume\") pod \"dns-default-tfzf9\" (UID: \"e4384477-41b5-40c5-83e5-23f7c541dfa0\") " pod="openshift-dns/dns-default-tfzf9" Jul 06 01:21:18.494620 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.494568 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpz4c\" (UniqueName: \"kubernetes.io/projected/889f2b80-8fba-40de-a6f6-314a17b310df-kube-api-access-dpz4c\") pod \"ingress-canary-5br85\" (UID: \"889f2b80-8fba-40de-a6f6-314a17b310df\") " pod="openshift-ingress-canary/ingress-canary-5br85" Jul 06 01:21:18.494620 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.494622 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e4384477-41b5-40c5-83e5-23f7c541dfa0-metrics-tls\") pod \"dns-default-tfzf9\" (UID: \"e4384477-41b5-40c5-83e5-23f7c541dfa0\") " pod="openshift-dns/dns-default-tfzf9" Jul 06 01:21:18.494895 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.494758 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4384477-41b5-40c5-83e5-23f7c541dfa0-tmp-dir\") pod \"dns-default-tfzf9\" (UID: \"e4384477-41b5-40c5-83e5-23f7c541dfa0\") " pod="openshift-dns/dns-default-tfzf9" Jul 06 01:21:18.494895 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.494820 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw4vx\" (UniqueName: \"kubernetes.io/projected/e4384477-41b5-40c5-83e5-23f7c541dfa0-kube-api-access-tw4vx\") pod \"dns-default-tfzf9\" (UID: \"e4384477-41b5-40c5-83e5-23f7c541dfa0\") " pod="openshift-dns/dns-default-tfzf9" Jul 06 01:21:18.494997 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.494894 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e4384477-41b5-40c5-83e5-23f7c541dfa0-config-volume\") pod \"dns-default-tfzf9\" (UID: \"e4384477-41b5-40c5-83e5-23f7c541dfa0\") " pod="openshift-dns/dns-default-tfzf9" Jul 06 01:21:18.494997 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.494931 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/889f2b80-8fba-40de-a6f6-314a17b310df-cert\") pod \"ingress-canary-5br85\" (UID: \"889f2b80-8fba-40de-a6f6-314a17b310df\") " pod="openshift-ingress-canary/ingress-canary-5br85" Jul 06 01:21:18.495203 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.495180 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4384477-41b5-40c5-83e5-23f7c541dfa0-tmp-dir\") pod \"dns-default-tfzf9\" (UID: \"e4384477-41b5-40c5-83e5-23f7c541dfa0\") " pod="openshift-dns/dns-default-tfzf9" Jul 06 01:21:18.495549 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.495525 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e4384477-41b5-40c5-83e5-23f7c541dfa0-config-volume\") pod \"dns-default-tfzf9\" (UID: \"e4384477-41b5-40c5-83e5-23f7c541dfa0\") " pod="openshift-dns/dns-default-tfzf9" Jul 06 01:21:18.498988 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.498957 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e4384477-41b5-40c5-83e5-23f7c541dfa0-metrics-tls\") pod \"dns-default-tfzf9\" (UID: \"e4384477-41b5-40c5-83e5-23f7c541dfa0\") " pod="openshift-dns/dns-default-tfzf9" Jul 06 01:21:18.499152 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.499134 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/889f2b80-8fba-40de-a6f6-314a17b310df-cert\") pod \"ingress-canary-5br85\" (UID: \"889f2b80-8fba-40de-a6f6-314a17b310df\") " pod="openshift-ingress-canary/ingress-canary-5br85" Jul 06 01:21:18.502053 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.502030 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tw4vx\" (UniqueName: \"kubernetes.io/projected/e4384477-41b5-40c5-83e5-23f7c541dfa0-kube-api-access-tw4vx\") pod \"dns-default-tfzf9\" (UID: \"e4384477-41b5-40c5-83e5-23f7c541dfa0\") " pod="openshift-dns/dns-default-tfzf9" Jul 06 01:21:18.502176 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.502135 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpz4c\" (UniqueName: \"kubernetes.io/projected/889f2b80-8fba-40de-a6f6-314a17b310df-kube-api-access-dpz4c\") pod \"ingress-canary-5br85\" (UID: \"889f2b80-8fba-40de-a6f6-314a17b310df\") " pod="openshift-ingress-canary/ingress-canary-5br85" Jul 06 01:21:18.538136 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.538088 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tfzf9" Jul 06 01:21:18.552583 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.552556 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5br85" Jul 06 01:21:18.757683 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.757606 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tfzf9"] Jul 06 01:21:18.762196 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:18.762164 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5br85"] Jul 06 01:21:18.766510 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:21:18.766479 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod889f2b80_8fba_40de_a6f6_314a17b310df.slice/crio-ff868ac362b95d0555e2460c3b4baad170dacda3661eccfd8b693ea2dac1b552 WatchSource:0}: Error finding container ff868ac362b95d0555e2460c3b4baad170dacda3661eccfd8b693ea2dac1b552: Status 404 returned error can't find the container with id ff868ac362b95d0555e2460c3b4baad170dacda3661eccfd8b693ea2dac1b552 Jul 06 01:21:19.558368 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:19.558321 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5br85" event={"ID":"889f2b80-8fba-40de-a6f6-314a17b310df","Type":"ContainerStarted","Data":"ff868ac362b95d0555e2460c3b4baad170dacda3661eccfd8b693ea2dac1b552"} Jul 06 01:21:19.559850 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:19.559821 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tfzf9" event={"ID":"e4384477-41b5-40c5-83e5-23f7c541dfa0","Type":"ContainerStarted","Data":"e57532f866a2926525ee1e04009598728caa8a7ced1a884d6a7e4349c93d39ba"} Jul 06 01:21:20.360623 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:20.360580 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-fnb5n"] Jul 06 01:21:20.379125 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:20.379090 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-fnb5n"] Jul 06 01:21:20.379301 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:20.379187 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-fnb5n" Jul 06 01:21:20.381782 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:20.381755 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 06 01:21:20.381927 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:20.381809 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-xt46v\"" Jul 06 01:21:20.512514 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:20.512474 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6d50abeb-044b-4ff4-9f61-408e4363dbe6-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-fnb5n\" (UID: \"6d50abeb-044b-4ff4-9f61-408e4363dbe6\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-fnb5n" Jul 06 01:21:20.613522 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:20.613440 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6d50abeb-044b-4ff4-9f61-408e4363dbe6-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-fnb5n\" (UID: \"6d50abeb-044b-4ff4-9f61-408e4363dbe6\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-fnb5n" Jul 06 01:21:20.613971 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:20.613576 2565 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 06 01:21:20.613971 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:20.613673 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d50abeb-044b-4ff4-9f61-408e4363dbe6-tls-certificates podName:6d50abeb-044b-4ff4-9f61-408e4363dbe6 nodeName:}" failed. No retries permitted until 2026-07-06 01:21:21.113644609 +0000 UTC m=+53.255203389 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/6d50abeb-044b-4ff4-9f61-408e4363dbe6-tls-certificates") pod "prometheus-operator-admission-webhook-d559d66d5-fnb5n" (UID: "6d50abeb-044b-4ff4-9f61-408e4363dbe6") : secret "prometheus-operator-admission-webhook-tls" not found Jul 06 01:21:20.949421 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:20.949326 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-7wptq"] Jul 06 01:21:20.967479 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:20.967413 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-7wptq"] Jul 06 01:21:20.967656 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:20.967507 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-7wptq" Jul 06 01:21:20.970174 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:20.970151 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 06 01:21:20.970337 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:20.970200 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-glnql\"" Jul 06 01:21:20.971579 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:20.971556 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 06 01:21:20.994797 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:20.994765 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-9vxks"] Jul 06 01:21:21.017487 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:21.017457 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9vxks" Jul 06 01:21:21.020050 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:21.019881 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bxc9g\"" Jul 06 01:21:21.020285 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:21.020258 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 01:21:21.020435 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:21.020327 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 01:21:21.020739 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:21.020721 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 01:21:21.021613 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:21.021595 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 01:21:21.023653 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:21.023623 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9vxks"] Jul 06 01:21:21.085670 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:21.085635 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-x9sxx"] Jul 06 01:21:21.100350 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:21.100274 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-x9sxx" Jul 06 01:21:21.102362 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:21.102328 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-x9sxx"] Jul 06 01:21:21.103138 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:21.103113 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 06 01:21:21.103259 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:21.103172 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 06 01:21:21.103485 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:21.103416 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-v9jv9\"" Jul 06 01:21:21.117685 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:21.117650 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/117cb15c-e26a-4b08-9dc2-57f776b22adc-data-volume\") pod \"insights-runtime-extractor-9vxks\" (UID: \"117cb15c-e26a-4b08-9dc2-57f776b22adc\") " pod="openshift-insights/insights-runtime-extractor-9vxks" Jul 06 01:21:21.117850 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:21.117701 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/117cb15c-e26a-4b08-9dc2-57f776b22adc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9vxks\" (UID: \"117cb15c-e26a-4b08-9dc2-57f776b22adc\") " pod="openshift-insights/insights-runtime-extractor-9vxks" Jul 06 01:21:21.117850 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:21.117827 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6d50abeb-044b-4ff4-9f61-408e4363dbe6-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-fnb5n\" (UID: \"6d50abeb-044b-4ff4-9f61-408e4363dbe6\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-fnb5n" Jul 06 01:21:21.117963 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:21.117858 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjvcw\" (UniqueName: \"kubernetes.io/projected/117cb15c-e26a-4b08-9dc2-57f776b22adc-kube-api-access-zjvcw\") pod \"insights-runtime-extractor-9vxks\" (UID: \"117cb15c-e26a-4b08-9dc2-57f776b22adc\") " pod="openshift-insights/insights-runtime-extractor-9vxks" Jul 06 01:21:21.117963 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:21.117879 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/117cb15c-e26a-4b08-9dc2-57f776b22adc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9vxks\" (UID: \"117cb15c-e26a-4b08-9dc2-57f776b22adc\") " pod="openshift-insights/insights-runtime-extractor-9vxks" Jul 06 01:21:21.117963 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:21.117901 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wls5r\" (UniqueName: \"kubernetes.io/projected/0af069fa-e1e2-4bc0-bc70-762b072c0cbb-kube-api-access-wls5r\") pod \"migrator-d57d58cc9-7wptq\" (UID: \"0af069fa-e1e2-4bc0-bc70-762b072c0cbb\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-7wptq" Jul 06 01:21:21.118096 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:21.117977 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/117cb15c-e26a-4b08-9dc2-57f776b22adc-crio-socket\") pod \"insights-runtime-extractor-9vxks\" (UID: \"117cb15c-e26a-4b08-9dc2-57f776b22adc\") " pod="openshift-insights/insights-runtime-extractor-9vxks" Jul 06 01:21:21.122927 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:21.122893 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6d50abeb-044b-4ff4-9f61-408e4363dbe6-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-fnb5n\" (UID: \"6d50abeb-044b-4ff4-9f61-408e4363dbe6\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-fnb5n" Jul 06 01:21:21.218532 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:21.218448 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjvcw\" (UniqueName: \"kubernetes.io/projected/117cb15c-e26a-4b08-9dc2-57f776b22adc-kube-api-access-zjvcw\") pod \"insights-runtime-extractor-9vxks\" (UID: \"117cb15c-e26a-4b08-9dc2-57f776b22adc\") " pod="openshift-insights/insights-runtime-extractor-9vxks" Jul 06 01:21:21.218532 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:21.218489 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/117cb15c-e26a-4b08-9dc2-57f776b22adc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9vxks\" (UID: \"117cb15c-e26a-4b08-9dc2-57f776b22adc\") " pod="openshift-insights/insights-runtime-extractor-9vxks" Jul 06 01:21:21.218532 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:21.218517 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wls5r\" (UniqueName: \"kubernetes.io/projected/0af069fa-e1e2-4bc0-bc70-762b072c0cbb-kube-api-access-wls5r\") pod \"migrator-d57d58cc9-7wptq\" (UID: \"0af069fa-e1e2-4bc0-bc70-762b072c0cbb\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-7wptq" Jul 06 01:21:21.218793 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:21.218561 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/117cb15c-e26a-4b08-9dc2-57f776b22adc-crio-socket\") pod \"insights-runtime-extractor-9vxks\" (UID: \"117cb15c-e26a-4b08-9dc2-57f776b22adc\") " pod="openshift-insights/insights-runtime-extractor-9vxks" Jul 06 01:21:21.218793 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:21.218586 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t56zp\" (UniqueName: \"kubernetes.io/projected/6122d230-b346-4d52-9e50-762d328a2651-kube-api-access-t56zp\") pod \"downloads-685bfb5f64-x9sxx\" (UID: \"6122d230-b346-4d52-9e50-762d328a2651\") " pod="openshift-console/downloads-685bfb5f64-x9sxx" Jul 06 01:21:21.218793 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:21.218622 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/117cb15c-e26a-4b08-9dc2-57f776b22adc-data-volume\") pod \"insights-runtime-extractor-9vxks\" (UID: \"117cb15c-e26a-4b08-9dc2-57f776b22adc\") " pod="openshift-insights/insights-runtime-extractor-9vxks" Jul 06 01:21:21.218793 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:21.218643 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/117cb15c-e26a-4b08-9dc2-57f776b22adc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9vxks\" (UID: \"117cb15c-e26a-4b08-9dc2-57f776b22adc\") " pod="openshift-insights/insights-runtime-extractor-9vxks" Jul 06 01:21:21.218997 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:21.218917 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/117cb15c-e26a-4b08-9dc2-57f776b22adc-crio-socket\") pod \"insights-runtime-extractor-9vxks\" (UID: \"117cb15c-e26a-4b08-9dc2-57f776b22adc\") " pod="openshift-insights/insights-runtime-extractor-9vxks" Jul 06 01:21:21.218997 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:21.218974 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 01:21:21.219101 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:21.219011 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 01:21:21.219101 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:21.219039 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/117cb15c-e26a-4b08-9dc2-57f776b22adc-insights-runtime-extractor-tls podName:117cb15c-e26a-4b08-9dc2-57f776b22adc nodeName:}" failed. No retries permitted until 2026-07-06 01:21:21.719017924 +0000 UTC m=+53.860576707 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/117cb15c-e26a-4b08-9dc2-57f776b22adc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9vxks" (UID: "117cb15c-e26a-4b08-9dc2-57f776b22adc") : secret "insights-runtime-extractor-tls" not found Jul 06 01:21:21.219101 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:21.219085 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/117cb15c-e26a-4b08-9dc2-57f776b22adc-kube-rbac-proxy-cm podName:117cb15c-e26a-4b08-9dc2-57f776b22adc nodeName:}" failed. No retries permitted until 2026-07-06 01:21:21.719069378 +0000 UTC m=+53.860628140 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/117cb15c-e26a-4b08-9dc2-57f776b22adc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9vxks" (UID: "117cb15c-e26a-4b08-9dc2-57f776b22adc") : configmap "kube-rbac-proxy" not found Jul 06 01:21:21.219265 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:21.219122 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/117cb15c-e26a-4b08-9dc2-57f776b22adc-data-volume\") pod \"insights-runtime-extractor-9vxks\" (UID: \"117cb15c-e26a-4b08-9dc2-57f776b22adc\") " pod="openshift-insights/insights-runtime-extractor-9vxks" Jul 06 01:21:21.228020 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:21.227988 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wls5r\" (UniqueName: \"kubernetes.io/projected/0af069fa-e1e2-4bc0-bc70-762b072c0cbb-kube-api-access-wls5r\") pod \"migrator-d57d58cc9-7wptq\" (UID: \"0af069fa-e1e2-4bc0-bc70-762b072c0cbb\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-7wptq" Jul 06 01:21:21.228201 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:21.228067 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjvcw\" (UniqueName: \"kubernetes.io/projected/117cb15c-e26a-4b08-9dc2-57f776b22adc-kube-api-access-zjvcw\") pod \"insights-runtime-extractor-9vxks\" (UID: \"117cb15c-e26a-4b08-9dc2-57f776b22adc\") " pod="openshift-insights/insights-runtime-extractor-9vxks" Jul 06 01:21:21.278297 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:21.278260 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-7wptq" Jul 06 01:21:21.291586 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:21.291547 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-fnb5n" Jul 06 01:21:21.319105 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:21.319069 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t56zp\" (UniqueName: \"kubernetes.io/projected/6122d230-b346-4d52-9e50-762d328a2651-kube-api-access-t56zp\") pod \"downloads-685bfb5f64-x9sxx\" (UID: \"6122d230-b346-4d52-9e50-762d328a2651\") " pod="openshift-console/downloads-685bfb5f64-x9sxx" Jul 06 01:21:21.327674 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:21.327642 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t56zp\" (UniqueName: \"kubernetes.io/projected/6122d230-b346-4d52-9e50-762d328a2651-kube-api-access-t56zp\") pod \"downloads-685bfb5f64-x9sxx\" (UID: \"6122d230-b346-4d52-9e50-762d328a2651\") " pod="openshift-console/downloads-685bfb5f64-x9sxx" Jul 06 01:21:21.416752 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:21.416719 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-x9sxx" Jul 06 01:21:21.722642 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:21.722616 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/117cb15c-e26a-4b08-9dc2-57f776b22adc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9vxks\" (UID: \"117cb15c-e26a-4b08-9dc2-57f776b22adc\") " pod="openshift-insights/insights-runtime-extractor-9vxks" Jul 06 01:21:21.723096 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:21.722712 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/117cb15c-e26a-4b08-9dc2-57f776b22adc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9vxks\" (UID: \"117cb15c-e26a-4b08-9dc2-57f776b22adc\") " pod="openshift-insights/insights-runtime-extractor-9vxks" Jul 06 01:21:21.723096 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:21.722776 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 01:21:21.723096 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:21.722839 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/117cb15c-e26a-4b08-9dc2-57f776b22adc-kube-rbac-proxy-cm podName:117cb15c-e26a-4b08-9dc2-57f776b22adc nodeName:}" failed. No retries permitted until 2026-07-06 01:21:22.722820135 +0000 UTC m=+54.864378902 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/117cb15c-e26a-4b08-9dc2-57f776b22adc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9vxks" (UID: "117cb15c-e26a-4b08-9dc2-57f776b22adc") : configmap "kube-rbac-proxy" not found Jul 06 01:21:21.723096 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:21.722851 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 01:21:21.723096 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:21.722916 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/117cb15c-e26a-4b08-9dc2-57f776b22adc-insights-runtime-extractor-tls podName:117cb15c-e26a-4b08-9dc2-57f776b22adc nodeName:}" failed. No retries permitted until 2026-07-06 01:21:22.722900139 +0000 UTC m=+54.864458923 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/117cb15c-e26a-4b08-9dc2-57f776b22adc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9vxks" (UID: "117cb15c-e26a-4b08-9dc2-57f776b22adc") : secret "insights-runtime-extractor-tls" not found Jul 06 01:21:21.899411 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:21.899314 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-fnb5n"] Jul 06 01:21:21.903155 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:21:21.903129 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d50abeb_044b_4ff4_9f61_408e4363dbe6.slice/crio-517545c9b5059021d8b12f8c4094c4e67235460eb460888726e4bece3a834079 WatchSource:0}: Error finding container 517545c9b5059021d8b12f8c4094c4e67235460eb460888726e4bece3a834079: Status 404 returned error can't find the container with id 517545c9b5059021d8b12f8c4094c4e67235460eb460888726e4bece3a834079 Jul 06 01:21:21.920062 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:21.918993 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-7wptq"] Jul 06 01:21:21.923588 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:21:21.923553 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0af069fa_e1e2_4bc0_bc70_762b072c0cbb.slice/crio-43008cb02e04d4132e781cb7c76bbb28db468ce7d95637ae53bcdfe30eb5f990 WatchSource:0}: Error finding container 43008cb02e04d4132e781cb7c76bbb28db468ce7d95637ae53bcdfe30eb5f990: Status 404 returned error can't find the container with id 43008cb02e04d4132e781cb7c76bbb28db468ce7d95637ae53bcdfe30eb5f990 Jul 06 01:21:21.933358 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:21.933326 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-x9sxx"] Jul 06 01:21:21.936798 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:21:21.936765 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6122d230_b346_4d52_9e50_762d328a2651.slice/crio-a3680b19eefdc863c95fb3fdff160f9286f4d7f6eb9c35129e0cc69c47fb23d9 WatchSource:0}: Error finding container a3680b19eefdc863c95fb3fdff160f9286f4d7f6eb9c35129e0cc69c47fb23d9: Status 404 returned error can't find the container with id a3680b19eefdc863c95fb3fdff160f9286f4d7f6eb9c35129e0cc69c47fb23d9 Jul 06 01:21:22.569916 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:22.569648 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-fnb5n" event={"ID":"6d50abeb-044b-4ff4-9f61-408e4363dbe6","Type":"ContainerStarted","Data":"517545c9b5059021d8b12f8c4094c4e67235460eb460888726e4bece3a834079"} Jul 06 01:21:22.571526 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:22.571492 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-x9sxx" event={"ID":"6122d230-b346-4d52-9e50-762d328a2651","Type":"ContainerStarted","Data":"a3680b19eefdc863c95fb3fdff160f9286f4d7f6eb9c35129e0cc69c47fb23d9"} Jul 06 01:21:22.573844 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:22.573813 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5br85" event={"ID":"889f2b80-8fba-40de-a6f6-314a17b310df","Type":"ContainerStarted","Data":"b1987809e9dd3310b963c6e937719e82cba370885e464b10263fd1cc709a4863"} Jul 06 01:21:22.576047 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:22.576024 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tfzf9" event={"ID":"e4384477-41b5-40c5-83e5-23f7c541dfa0","Type":"ContainerStarted","Data":"0131d5322ca818d859067c1468a1e2b5ae2dd9bb2e8dd764da3e3ae10a94ba44"} Jul 06 01:21:22.576185 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:22.576071 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tfzf9" event={"ID":"e4384477-41b5-40c5-83e5-23f7c541dfa0","Type":"ContainerStarted","Data":"0b1f8f2bd813b27f801723a6302da41724b54b5fa8cc5e657dff1c735398f431"} Jul 06 01:21:22.576185 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:22.576131 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-tfzf9" Jul 06 01:21:22.577775 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:22.577746 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-7wptq" event={"ID":"0af069fa-e1e2-4bc0-bc70-762b072c0cbb","Type":"ContainerStarted","Data":"43008cb02e04d4132e781cb7c76bbb28db468ce7d95637ae53bcdfe30eb5f990"} Jul 06 01:21:22.588468 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:22.588407 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5br85" podStartSLOduration=1.637391069 podStartE2EDuration="4.588388699s" totalImagesPullingTime="2.95099763s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:21:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:21:22.58828896 +0000 UTC m=+54.729847747" watchObservedRunningTime="2026-07-06 01:21:22.588388699 +0000 UTC m=+54.729947475" Jul 06 01:21:22.732154 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:22.730983 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/117cb15c-e26a-4b08-9dc2-57f776b22adc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9vxks\" (UID: \"117cb15c-e26a-4b08-9dc2-57f776b22adc\") " pod="openshift-insights/insights-runtime-extractor-9vxks" Jul 06 01:21:22.732154 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:22.731078 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/117cb15c-e26a-4b08-9dc2-57f776b22adc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9vxks\" (UID: \"117cb15c-e26a-4b08-9dc2-57f776b22adc\") " pod="openshift-insights/insights-runtime-extractor-9vxks" Jul 06 01:21:22.732154 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:22.731496 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 01:21:22.732154 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:22.731563 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/117cb15c-e26a-4b08-9dc2-57f776b22adc-kube-rbac-proxy-cm podName:117cb15c-e26a-4b08-9dc2-57f776b22adc nodeName:}" failed. No retries permitted until 2026-07-06 01:21:24.731544297 +0000 UTC m=+56.873103062 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/117cb15c-e26a-4b08-9dc2-57f776b22adc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9vxks" (UID: "117cb15c-e26a-4b08-9dc2-57f776b22adc") : configmap "kube-rbac-proxy" not found Jul 06 01:21:22.732154 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:22.731599 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 01:21:22.732154 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:22.731654 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/117cb15c-e26a-4b08-9dc2-57f776b22adc-insights-runtime-extractor-tls podName:117cb15c-e26a-4b08-9dc2-57f776b22adc nodeName:}" failed. No retries permitted until 2026-07-06 01:21:24.731638902 +0000 UTC m=+56.873197685 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/117cb15c-e26a-4b08-9dc2-57f776b22adc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9vxks" (UID: "117cb15c-e26a-4b08-9dc2-57f776b22adc") : secret "insights-runtime-extractor-tls" not found Jul 06 01:21:22.767114 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:22.767084 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2s7xk_28f968d2-c730-436e-8bb9-413d0f88cf10/dns-node-resolver/0.log" Jul 06 01:21:23.583034 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:23.582996 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-fnb5n" event={"ID":"6d50abeb-044b-4ff4-9f61-408e4363dbe6","Type":"ContainerStarted","Data":"a6f758af5020fc8d8b3c73bd986e9af1f430369eec63950fd92eb03b8687421a"} Jul 06 01:21:23.583484 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:23.583464 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-fnb5n" Jul 06 01:21:23.585049 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:23.585003 2565 patch_prober.go:28] interesting pod/prometheus-operator-admission-webhook-d559d66d5-fnb5n container/prometheus-operator-admission-webhook namespace/openshift-monitoring: Readiness probe status=failure output="Get \"https://10.134.0.8:8443/healthz\": dial tcp 10.134.0.8:8443: connect: connection refused" start-of-body= Jul 06 01:21:23.585221 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:23.585070 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-fnb5n" podUID="6d50abeb-044b-4ff4-9f61-408e4363dbe6" containerName="prometheus-operator-admission-webhook" probeResult="failure" output="Get \"https://10.134.0.8:8443/healthz\": dial tcp 10.134.0.8:8443: connect: connection refused" Jul 06 01:21:23.585806 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:23.585680 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-7wptq" event={"ID":"0af069fa-e1e2-4bc0-bc70-762b072c0cbb","Type":"ContainerStarted","Data":"699bc2242ff93cc5def7590aaf6e03d3ea25836ea62add7f0190db069bd544ef"} Jul 06 01:21:23.599429 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:23.598704 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-tfzf9" podStartSLOduration=2.647008185 podStartE2EDuration="5.598681259s" totalImagesPullingTime="2.951673074s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:21:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:21:22.605064917 +0000 UTC m=+54.746623702" watchObservedRunningTime="2026-07-06 01:21:23.598681259 +0000 UTC m=+55.740240045" Jul 06 01:21:24.166600 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:24.166570 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5nr92_ec5262e8-19a7-4dce-bf06-4d792eb2b877/node-ca/0.log" Jul 06 01:21:24.592182 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:24.592146 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-7wptq" event={"ID":"0af069fa-e1e2-4bc0-bc70-762b072c0cbb","Type":"ContainerStarted","Data":"2b2cb6097d18dbc68daaa68d07ba440939de85b68010ff0412becd6fbc4e3861"} Jul 06 01:21:24.597187 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:24.597157 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-fnb5n" Jul 06 01:21:24.607889 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:24.607821 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-7wptq" podStartSLOduration=3.037857455 podStartE2EDuration="4.607799607s" totalImagesPullingTime="1.569942152s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:21:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:21:24.607138209 +0000 UTC m=+56.748696996" watchObservedRunningTime="2026-07-06 01:21:24.607799607 +0000 UTC m=+56.749358393" Jul 06 01:21:24.608215 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:24.608181 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-fnb5n" podStartSLOduration=3.017759976 podStartE2EDuration="4.608172782s" totalImagesPullingTime="1.590412806s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:21:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:21:23.600168729 +0000 UTC m=+55.741727534" watchObservedRunningTime="2026-07-06 01:21:24.608172782 +0000 UTC m=+56.749731568" Jul 06 01:21:24.748286 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:24.748242 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/117cb15c-e26a-4b08-9dc2-57f776b22adc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9vxks\" (UID: \"117cb15c-e26a-4b08-9dc2-57f776b22adc\") " pod="openshift-insights/insights-runtime-extractor-9vxks" Jul 06 01:21:24.748513 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:24.748345 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/117cb15c-e26a-4b08-9dc2-57f776b22adc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9vxks\" (UID: \"117cb15c-e26a-4b08-9dc2-57f776b22adc\") " pod="openshift-insights/insights-runtime-extractor-9vxks" Jul 06 01:21:24.748513 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:24.748453 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 01:21:24.748624 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:24.748526 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/117cb15c-e26a-4b08-9dc2-57f776b22adc-insights-runtime-extractor-tls podName:117cb15c-e26a-4b08-9dc2-57f776b22adc nodeName:}" failed. No retries permitted until 2026-07-06 01:21:28.748508365 +0000 UTC m=+60.890067128 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/117cb15c-e26a-4b08-9dc2-57f776b22adc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9vxks" (UID: "117cb15c-e26a-4b08-9dc2-57f776b22adc") : secret "insights-runtime-extractor-tls" not found Jul 06 01:21:24.748713 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:24.748690 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 01:21:24.748766 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:24.748752 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/117cb15c-e26a-4b08-9dc2-57f776b22adc-kube-rbac-proxy-cm podName:117cb15c-e26a-4b08-9dc2-57f776b22adc nodeName:}" failed. No retries permitted until 2026-07-06 01:21:28.748736297 +0000 UTC m=+60.890295061 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/117cb15c-e26a-4b08-9dc2-57f776b22adc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9vxks" (UID: "117cb15c-e26a-4b08-9dc2-57f776b22adc") : configmap "kube-rbac-proxy" not found Jul 06 01:21:24.767735 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:24.767690 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5br85_889f2b80-8fba-40de-a6f6-314a17b310df/serve-healthcheck-canary/0.log" Jul 06 01:21:25.422631 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:25.422592 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-q6scb"] Jul 06 01:21:25.424945 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:25.424918 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q6scb" Jul 06 01:21:25.427250 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:25.427225 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 06 01:21:25.428354 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:25.428121 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 01:21:25.428354 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:25.428232 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 06 01:21:25.428354 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:25.428269 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 01:21:25.428617 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:25.428561 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-bbvtx\"" Jul 06 01:21:25.428791 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:25.428774 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 01:21:25.435626 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:25.435602 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-q6scb"] Jul 06 01:21:25.553808 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:25.553751 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6e241b85-89f3-4b81-9fdf-5af2a97b9033-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-q6scb\" (UID: \"6e241b85-89f3-4b81-9fdf-5af2a97b9033\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q6scb" Jul 06 01:21:25.554010 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:25.553883 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwj7b\" (UniqueName: \"kubernetes.io/projected/6e241b85-89f3-4b81-9fdf-5af2a97b9033-kube-api-access-dwj7b\") pod \"prometheus-operator-567f7b8b7c-q6scb\" (UID: \"6e241b85-89f3-4b81-9fdf-5af2a97b9033\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q6scb" Jul 06 01:21:25.554010 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:25.553910 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6e241b85-89f3-4b81-9fdf-5af2a97b9033-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-q6scb\" (UID: \"6e241b85-89f3-4b81-9fdf-5af2a97b9033\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q6scb" Jul 06 01:21:25.554010 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:25.553935 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/6e241b85-89f3-4b81-9fdf-5af2a97b9033-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-q6scb\" (UID: \"6e241b85-89f3-4b81-9fdf-5af2a97b9033\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q6scb" Jul 06 01:21:25.655388 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:25.655332 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6e241b85-89f3-4b81-9fdf-5af2a97b9033-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-q6scb\" (UID: \"6e241b85-89f3-4b81-9fdf-5af2a97b9033\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q6scb" Jul 06 01:21:25.655572 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:25.655455 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwj7b\" (UniqueName: \"kubernetes.io/projected/6e241b85-89f3-4b81-9fdf-5af2a97b9033-kube-api-access-dwj7b\") pod \"prometheus-operator-567f7b8b7c-q6scb\" (UID: \"6e241b85-89f3-4b81-9fdf-5af2a97b9033\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q6scb" Jul 06 01:21:25.655572 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:25.655478 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6e241b85-89f3-4b81-9fdf-5af2a97b9033-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-q6scb\" (UID: \"6e241b85-89f3-4b81-9fdf-5af2a97b9033\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q6scb" Jul 06 01:21:25.655572 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:25.655497 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/6e241b85-89f3-4b81-9fdf-5af2a97b9033-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-q6scb\" (UID: \"6e241b85-89f3-4b81-9fdf-5af2a97b9033\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q6scb" Jul 06 01:21:25.656297 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:25.656271 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6e241b85-89f3-4b81-9fdf-5af2a97b9033-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-q6scb\" (UID: \"6e241b85-89f3-4b81-9fdf-5af2a97b9033\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q6scb" Jul 06 01:21:25.659817 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:25.659788 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6e241b85-89f3-4b81-9fdf-5af2a97b9033-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-q6scb\" (UID: \"6e241b85-89f3-4b81-9fdf-5af2a97b9033\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q6scb" Jul 06 01:21:25.659947 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:25.659825 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/6e241b85-89f3-4b81-9fdf-5af2a97b9033-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-q6scb\" (UID: \"6e241b85-89f3-4b81-9fdf-5af2a97b9033\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q6scb" Jul 06 01:21:25.667179 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:25.667151 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwj7b\" (UniqueName: \"kubernetes.io/projected/6e241b85-89f3-4b81-9fdf-5af2a97b9033-kube-api-access-dwj7b\") pod \"prometheus-operator-567f7b8b7c-q6scb\" (UID: \"6e241b85-89f3-4b81-9fdf-5af2a97b9033\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q6scb" Jul 06 01:21:25.736089 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:25.736053 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q6scb" Jul 06 01:21:25.889581 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:25.889542 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-q6scb"] Jul 06 01:21:25.893475 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:21:25.893445 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e241b85_89f3_4b81_9fdf_5af2a97b9033.slice/crio-aeddbe9211754b2454070d5c457d65fee20145aa1ae3b8f733b1a0e0d298edec WatchSource:0}: Error finding container aeddbe9211754b2454070d5c457d65fee20145aa1ae3b8f733b1a0e0d298edec: Status 404 returned error can't find the container with id aeddbe9211754b2454070d5c457d65fee20145aa1ae3b8f733b1a0e0d298edec Jul 06 01:21:26.605533 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:26.605414 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q6scb" event={"ID":"6e241b85-89f3-4b81-9fdf-5af2a97b9033","Type":"ContainerStarted","Data":"aeddbe9211754b2454070d5c457d65fee20145aa1ae3b8f733b1a0e0d298edec"} Jul 06 01:21:27.610806 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:27.610707 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q6scb" event={"ID":"6e241b85-89f3-4b81-9fdf-5af2a97b9033","Type":"ContainerStarted","Data":"d195bef27525017bce4cf83b0dbe7bfc443484d4afd8c3ce72fe52037fbc1553"} Jul 06 01:21:27.610806 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:27.610774 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q6scb" event={"ID":"6e241b85-89f3-4b81-9fdf-5af2a97b9033","Type":"ContainerStarted","Data":"e0f77c5a979ec27a52637d90d4dd508a8cebc0cb8d3ba70639f436f168db2d0c"} Jul 06 01:21:27.629674 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:27.629615 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q6scb" podStartSLOduration=1.233561871 podStartE2EDuration="2.629598653s" totalImagesPullingTime="1.396036782s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:21:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:21:27.628641688 +0000 UTC m=+59.770200499" watchObservedRunningTime="2026-07-06 01:21:27.629598653 +0000 UTC m=+59.771157438" Jul 06 01:21:28.784152 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:28.784098 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/117cb15c-e26a-4b08-9dc2-57f776b22adc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9vxks\" (UID: \"117cb15c-e26a-4b08-9dc2-57f776b22adc\") " pod="openshift-insights/insights-runtime-extractor-9vxks" Jul 06 01:21:28.784658 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:28.784327 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/117cb15c-e26a-4b08-9dc2-57f776b22adc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9vxks\" (UID: \"117cb15c-e26a-4b08-9dc2-57f776b22adc\") " pod="openshift-insights/insights-runtime-extractor-9vxks" Jul 06 01:21:28.784881 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:28.784836 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/117cb15c-e26a-4b08-9dc2-57f776b22adc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9vxks\" (UID: \"117cb15c-e26a-4b08-9dc2-57f776b22adc\") " pod="openshift-insights/insights-runtime-extractor-9vxks" Jul 06 01:21:28.786989 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:28.786962 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/117cb15c-e26a-4b08-9dc2-57f776b22adc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9vxks\" (UID: \"117cb15c-e26a-4b08-9dc2-57f776b22adc\") " pod="openshift-insights/insights-runtime-extractor-9vxks" Jul 06 01:21:28.833234 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:28.833204 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bxc9g\"" Jul 06 01:21:28.841330 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:28.841301 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9vxks" Jul 06 01:21:29.027859 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.027826 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9vxks"] Jul 06 01:21:29.032416 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:21:29.032365 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod117cb15c_e26a_4b08_9dc2_57f776b22adc.slice/crio-931656c294a8e840bdd6868c9a1718a4214b443ea2f5496f0c1eb75d4b34f035 WatchSource:0}: Error finding container 931656c294a8e840bdd6868c9a1718a4214b443ea2f5496f0c1eb75d4b34f035: Status 404 returned error can't find the container with id 931656c294a8e840bdd6868c9a1718a4214b443ea2f5496f0c1eb75d4b34f035 Jul 06 01:21:29.622529 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.622437 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9vxks" event={"ID":"117cb15c-e26a-4b08-9dc2-57f776b22adc","Type":"ContainerStarted","Data":"931656c294a8e840bdd6868c9a1718a4214b443ea2f5496f0c1eb75d4b34f035"} Jul 06 01:21:29.773200 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.773143 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-448pq"] Jul 06 01:21:29.775721 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.775662 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-448pq" Jul 06 01:21:29.778978 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.778950 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-2tl9m\"" Jul 06 01:21:29.779436 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.779415 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 06 01:21:29.779550 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.779492 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 06 01:21:29.785770 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.785277 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-448pq"] Jul 06 01:21:29.792774 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.792744 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvzhs\" (UniqueName: \"kubernetes.io/projected/50574349-d208-4de6-aba3-06e535105c6a-kube-api-access-nvzhs\") pod \"openshift-state-metrics-54cf96dfcf-448pq\" (UID: \"50574349-d208-4de6-aba3-06e535105c6a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-448pq" Jul 06 01:21:29.792933 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.792806 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/50574349-d208-4de6-aba3-06e535105c6a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-448pq\" (UID: \"50574349-d208-4de6-aba3-06e535105c6a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-448pq" Jul 06 01:21:29.792933 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.792838 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/50574349-d208-4de6-aba3-06e535105c6a-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-448pq\" (UID: \"50574349-d208-4de6-aba3-06e535105c6a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-448pq" Jul 06 01:21:29.792933 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.792901 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/50574349-d208-4de6-aba3-06e535105c6a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-448pq\" (UID: \"50574349-d208-4de6-aba3-06e535105c6a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-448pq" Jul 06 01:21:29.807412 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.807362 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-cjx7b"] Jul 06 01:21:29.810154 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.810120 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cjx7b" Jul 06 01:21:29.814788 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.813867 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 01:21:29.814788 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.814284 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-q8hgd\"" Jul 06 01:21:29.814788 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.814412 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 01:21:29.814788 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.814577 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 01:21:29.826742 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.824392 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ksnx7"] Jul 06 01:21:29.827684 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.827666 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ksnx7" Jul 06 01:21:29.831445 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.830534 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 06 01:21:29.831445 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.830783 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 06 01:21:29.831445 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.831098 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 06 01:21:29.831445 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.831284 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-8wscz\"" Jul 06 01:21:29.838916 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.837669 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ksnx7"] Jul 06 01:21:29.893791 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.893691 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvzhs\" (UniqueName: \"kubernetes.io/projected/50574349-d208-4de6-aba3-06e535105c6a-kube-api-access-nvzhs\") pod \"openshift-state-metrics-54cf96dfcf-448pq\" (UID: \"50574349-d208-4de6-aba3-06e535105c6a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-448pq" Jul 06 01:21:29.893791 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.893744 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ca215253-2060-46f3-94c8-2b48a3fc81a2-node-exporter-wtmp\") pod \"node-exporter-cjx7b\" (UID: \"ca215253-2060-46f3-94c8-2b48a3fc81a2\") " pod="openshift-monitoring/node-exporter-cjx7b" Jul 06 01:21:29.893791 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.893793 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ca215253-2060-46f3-94c8-2b48a3fc81a2-metrics-client-ca\") pod \"node-exporter-cjx7b\" (UID: \"ca215253-2060-46f3-94c8-2b48a3fc81a2\") " pod="openshift-monitoring/node-exporter-cjx7b" Jul 06 01:21:29.894077 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.893832 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ca215253-2060-46f3-94c8-2b48a3fc81a2-root\") pod \"node-exporter-cjx7b\" (UID: \"ca215253-2060-46f3-94c8-2b48a3fc81a2\") " pod="openshift-monitoring/node-exporter-cjx7b" Jul 06 01:21:29.894077 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.893858 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhgbm\" (UniqueName: \"kubernetes.io/projected/ca215253-2060-46f3-94c8-2b48a3fc81a2-kube-api-access-hhgbm\") pod \"node-exporter-cjx7b\" (UID: \"ca215253-2060-46f3-94c8-2b48a3fc81a2\") " pod="openshift-monitoring/node-exporter-cjx7b" Jul 06 01:21:29.894077 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.893894 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/50574349-d208-4de6-aba3-06e535105c6a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-448pq\" (UID: \"50574349-d208-4de6-aba3-06e535105c6a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-448pq" Jul 06 01:21:29.894077 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.893920 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ca215253-2060-46f3-94c8-2b48a3fc81a2-node-exporter-textfile\") pod \"node-exporter-cjx7b\" (UID: \"ca215253-2060-46f3-94c8-2b48a3fc81a2\") " pod="openshift-monitoring/node-exporter-cjx7b" Jul 06 01:21:29.894077 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.893949 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/50574349-d208-4de6-aba3-06e535105c6a-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-448pq\" (UID: \"50574349-d208-4de6-aba3-06e535105c6a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-448pq" Jul 06 01:21:29.894077 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.893985 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ca215253-2060-46f3-94c8-2b48a3fc81a2-node-exporter-tls\") pod \"node-exporter-cjx7b\" (UID: \"ca215253-2060-46f3-94c8-2b48a3fc81a2\") " pod="openshift-monitoring/node-exporter-cjx7b" Jul 06 01:21:29.894077 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.894025 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/50574349-d208-4de6-aba3-06e535105c6a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-448pq\" (UID: \"50574349-d208-4de6-aba3-06e535105c6a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-448pq" Jul 06 01:21:29.894077 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.894051 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ca215253-2060-46f3-94c8-2b48a3fc81a2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cjx7b\" (UID: \"ca215253-2060-46f3-94c8-2b48a3fc81a2\") " pod="openshift-monitoring/node-exporter-cjx7b" Jul 06 01:21:29.894077 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.894075 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ca215253-2060-46f3-94c8-2b48a3fc81a2-node-exporter-accelerators-collector-config\") pod \"node-exporter-cjx7b\" (UID: \"ca215253-2060-46f3-94c8-2b48a3fc81a2\") " pod="openshift-monitoring/node-exporter-cjx7b" Jul 06 01:21:29.894556 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.894115 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca215253-2060-46f3-94c8-2b48a3fc81a2-sys\") pod \"node-exporter-cjx7b\" (UID: \"ca215253-2060-46f3-94c8-2b48a3fc81a2\") " pod="openshift-monitoring/node-exporter-cjx7b" Jul 06 01:21:29.896101 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.896047 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/50574349-d208-4de6-aba3-06e535105c6a-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-448pq\" (UID: \"50574349-d208-4de6-aba3-06e535105c6a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-448pq" Jul 06 01:21:29.899070 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.899019 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/50574349-d208-4de6-aba3-06e535105c6a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-448pq\" (UID: \"50574349-d208-4de6-aba3-06e535105c6a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-448pq" Jul 06 01:21:29.900408 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.900334 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/50574349-d208-4de6-aba3-06e535105c6a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-448pq\" (UID: \"50574349-d208-4de6-aba3-06e535105c6a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-448pq" Jul 06 01:21:29.912321 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.912287 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvzhs\" (UniqueName: \"kubernetes.io/projected/50574349-d208-4de6-aba3-06e535105c6a-kube-api-access-nvzhs\") pod \"openshift-state-metrics-54cf96dfcf-448pq\" (UID: \"50574349-d208-4de6-aba3-06e535105c6a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-448pq" Jul 06 01:21:29.946065 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.946031 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b9krk" Jul 06 01:21:29.996259 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.994919 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ca215253-2060-46f3-94c8-2b48a3fc81a2-metrics-client-ca\") pod \"node-exporter-cjx7b\" (UID: \"ca215253-2060-46f3-94c8-2b48a3fc81a2\") " pod="openshift-monitoring/node-exporter-cjx7b" Jul 06 01:21:29.996259 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.995001 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ca215253-2060-46f3-94c8-2b48a3fc81a2-root\") pod \"node-exporter-cjx7b\" (UID: \"ca215253-2060-46f3-94c8-2b48a3fc81a2\") " pod="openshift-monitoring/node-exporter-cjx7b" Jul 06 01:21:29.996259 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.995027 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhgbm\" (UniqueName: \"kubernetes.io/projected/ca215253-2060-46f3-94c8-2b48a3fc81a2-kube-api-access-hhgbm\") pod \"node-exporter-cjx7b\" (UID: \"ca215253-2060-46f3-94c8-2b48a3fc81a2\") " pod="openshift-monitoring/node-exporter-cjx7b" Jul 06 01:21:29.996259 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.995067 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a582740b-395e-485b-b214-81c6f58a7747-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-ksnx7\" (UID: \"a582740b-395e-485b-b214-81c6f58a7747\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ksnx7" Jul 06 01:21:29.996259 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.995102 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ca215253-2060-46f3-94c8-2b48a3fc81a2-node-exporter-textfile\") pod \"node-exporter-cjx7b\" (UID: \"ca215253-2060-46f3-94c8-2b48a3fc81a2\") " pod="openshift-monitoring/node-exporter-cjx7b" Jul 06 01:21:29.996259 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.995134 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a582740b-395e-485b-b214-81c6f58a7747-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-ksnx7\" (UID: \"a582740b-395e-485b-b214-81c6f58a7747\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ksnx7" Jul 06 01:21:29.996259 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.995178 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbgcl\" (UniqueName: \"kubernetes.io/projected/a582740b-395e-485b-b214-81c6f58a7747-kube-api-access-rbgcl\") pod \"kube-state-metrics-7c9dbb8b7d-ksnx7\" (UID: \"a582740b-395e-485b-b214-81c6f58a7747\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ksnx7" Jul 06 01:21:29.996259 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.995225 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ca215253-2060-46f3-94c8-2b48a3fc81a2-node-exporter-tls\") pod \"node-exporter-cjx7b\" (UID: \"ca215253-2060-46f3-94c8-2b48a3fc81a2\") " pod="openshift-monitoring/node-exporter-cjx7b" Jul 06 01:21:29.996259 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.995270 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ca215253-2060-46f3-94c8-2b48a3fc81a2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cjx7b\" (UID: \"ca215253-2060-46f3-94c8-2b48a3fc81a2\") " pod="openshift-monitoring/node-exporter-cjx7b" Jul 06 01:21:29.996259 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.995294 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ca215253-2060-46f3-94c8-2b48a3fc81a2-node-exporter-accelerators-collector-config\") pod \"node-exporter-cjx7b\" (UID: \"ca215253-2060-46f3-94c8-2b48a3fc81a2\") " pod="openshift-monitoring/node-exporter-cjx7b" Jul 06 01:21:29.996259 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.995337 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca215253-2060-46f3-94c8-2b48a3fc81a2-sys\") pod \"node-exporter-cjx7b\" (UID: \"ca215253-2060-46f3-94c8-2b48a3fc81a2\") " pod="openshift-monitoring/node-exporter-cjx7b" Jul 06 01:21:29.996259 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.995358 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a582740b-395e-485b-b214-81c6f58a7747-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-ksnx7\" (UID: \"a582740b-395e-485b-b214-81c6f58a7747\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ksnx7" Jul 06 01:21:29.996259 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.995392 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a582740b-395e-485b-b214-81c6f58a7747-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-ksnx7\" (UID: \"a582740b-395e-485b-b214-81c6f58a7747\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ksnx7" Jul 06 01:21:29.996259 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.995423 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ca215253-2060-46f3-94c8-2b48a3fc81a2-node-exporter-wtmp\") pod \"node-exporter-cjx7b\" (UID: \"ca215253-2060-46f3-94c8-2b48a3fc81a2\") " pod="openshift-monitoring/node-exporter-cjx7b" Jul 06 01:21:29.996259 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.995468 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a582740b-395e-485b-b214-81c6f58a7747-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-ksnx7\" (UID: \"a582740b-395e-485b-b214-81c6f58a7747\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ksnx7" Jul 06 01:21:29.996259 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.995569 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ca215253-2060-46f3-94c8-2b48a3fc81a2-node-exporter-wtmp\") pod \"node-exporter-cjx7b\" (UID: \"ca215253-2060-46f3-94c8-2b48a3fc81a2\") " pod="openshift-monitoring/node-exporter-cjx7b" Jul 06 01:21:29.997234 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:29.995812 2565 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 06 01:21:29.997234 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:29.995868 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca215253-2060-46f3-94c8-2b48a3fc81a2-node-exporter-tls podName:ca215253-2060-46f3-94c8-2b48a3fc81a2 nodeName:}" failed. No retries permitted until 2026-07-06 01:21:30.495850891 +0000 UTC m=+62.637409654 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/ca215253-2060-46f3-94c8-2b48a3fc81a2-node-exporter-tls") pod "node-exporter-cjx7b" (UID: "ca215253-2060-46f3-94c8-2b48a3fc81a2") : secret "node-exporter-tls" not found Jul 06 01:21:29.997234 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.997154 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ca215253-2060-46f3-94c8-2b48a3fc81a2-metrics-client-ca\") pod \"node-exporter-cjx7b\" (UID: \"ca215253-2060-46f3-94c8-2b48a3fc81a2\") " pod="openshift-monitoring/node-exporter-cjx7b" Jul 06 01:21:29.999323 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.997524 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ca215253-2060-46f3-94c8-2b48a3fc81a2-root\") pod \"node-exporter-cjx7b\" (UID: \"ca215253-2060-46f3-94c8-2b48a3fc81a2\") " pod="openshift-monitoring/node-exporter-cjx7b" Jul 06 01:21:29.999323 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.998235 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ca215253-2060-46f3-94c8-2b48a3fc81a2-node-exporter-textfile\") pod \"node-exporter-cjx7b\" (UID: \"ca215253-2060-46f3-94c8-2b48a3fc81a2\") " pod="openshift-monitoring/node-exporter-cjx7b" Jul 06 01:21:29.999323 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.998307 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca215253-2060-46f3-94c8-2b48a3fc81a2-sys\") pod \"node-exporter-cjx7b\" (UID: \"ca215253-2060-46f3-94c8-2b48a3fc81a2\") " pod="openshift-monitoring/node-exporter-cjx7b" Jul 06 01:21:29.999323 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:29.998846 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ca215253-2060-46f3-94c8-2b48a3fc81a2-node-exporter-accelerators-collector-config\") pod \"node-exporter-cjx7b\" (UID: \"ca215253-2060-46f3-94c8-2b48a3fc81a2\") " pod="openshift-monitoring/node-exporter-cjx7b" Jul 06 01:21:30.000051 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.000023 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ca215253-2060-46f3-94c8-2b48a3fc81a2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cjx7b\" (UID: \"ca215253-2060-46f3-94c8-2b48a3fc81a2\") " pod="openshift-monitoring/node-exporter-cjx7b" Jul 06 01:21:30.009523 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.009496 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhgbm\" (UniqueName: \"kubernetes.io/projected/ca215253-2060-46f3-94c8-2b48a3fc81a2-kube-api-access-hhgbm\") pod \"node-exporter-cjx7b\" (UID: \"ca215253-2060-46f3-94c8-2b48a3fc81a2\") " pod="openshift-monitoring/node-exporter-cjx7b" Jul 06 01:21:30.090204 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.089914 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-448pq" Jul 06 01:21:30.098844 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.096691 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a582740b-395e-485b-b214-81c6f58a7747-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-ksnx7\" (UID: \"a582740b-395e-485b-b214-81c6f58a7747\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ksnx7" Jul 06 01:21:30.098844 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.096746 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a582740b-395e-485b-b214-81c6f58a7747-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-ksnx7\" (UID: \"a582740b-395e-485b-b214-81c6f58a7747\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ksnx7" Jul 06 01:21:30.098844 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.096790 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbgcl\" (UniqueName: \"kubernetes.io/projected/a582740b-395e-485b-b214-81c6f58a7747-kube-api-access-rbgcl\") pod \"kube-state-metrics-7c9dbb8b7d-ksnx7\" (UID: \"a582740b-395e-485b-b214-81c6f58a7747\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ksnx7" Jul 06 01:21:30.098844 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.096867 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a582740b-395e-485b-b214-81c6f58a7747-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-ksnx7\" (UID: \"a582740b-395e-485b-b214-81c6f58a7747\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ksnx7" Jul 06 01:21:30.098844 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.096894 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a582740b-395e-485b-b214-81c6f58a7747-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-ksnx7\" (UID: \"a582740b-395e-485b-b214-81c6f58a7747\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ksnx7" Jul 06 01:21:30.098844 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.096938 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a582740b-395e-485b-b214-81c6f58a7747-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-ksnx7\" (UID: \"a582740b-395e-485b-b214-81c6f58a7747\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ksnx7" Jul 06 01:21:30.098844 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.097805 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a582740b-395e-485b-b214-81c6f58a7747-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-ksnx7\" (UID: \"a582740b-395e-485b-b214-81c6f58a7747\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ksnx7" Jul 06 01:21:30.100853 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.100045 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a582740b-395e-485b-b214-81c6f58a7747-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-ksnx7\" (UID: \"a582740b-395e-485b-b214-81c6f58a7747\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ksnx7" Jul 06 01:21:30.100853 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.100807 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a582740b-395e-485b-b214-81c6f58a7747-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-ksnx7\" (UID: \"a582740b-395e-485b-b214-81c6f58a7747\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ksnx7" Jul 06 01:21:30.103484 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.103442 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a582740b-395e-485b-b214-81c6f58a7747-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-ksnx7\" (UID: \"a582740b-395e-485b-b214-81c6f58a7747\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ksnx7" Jul 06 01:21:30.104524 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.104481 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a582740b-395e-485b-b214-81c6f58a7747-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-ksnx7\" (UID: \"a582740b-395e-485b-b214-81c6f58a7747\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ksnx7" Jul 06 01:21:30.113678 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.113610 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbgcl\" (UniqueName: \"kubernetes.io/projected/a582740b-395e-485b-b214-81c6f58a7747-kube-api-access-rbgcl\") pod \"kube-state-metrics-7c9dbb8b7d-ksnx7\" (UID: \"a582740b-395e-485b-b214-81c6f58a7747\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ksnx7" Jul 06 01:21:30.142630 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.142309 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ksnx7" Jul 06 01:21:30.286223 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.286183 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-448pq"] Jul 06 01:21:30.291508 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:21:30.291314 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50574349_d208_4de6_aba3_06e535105c6a.slice/crio-475ac34c32766460b591a3566a853b31698c95299e28eff202ddb3905aff8063 WatchSource:0}: Error finding container 475ac34c32766460b591a3566a853b31698c95299e28eff202ddb3905aff8063: Status 404 returned error can't find the container with id 475ac34c32766460b591a3566a853b31698c95299e28eff202ddb3905aff8063 Jul 06 01:21:30.356091 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.354835 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ksnx7"] Jul 06 01:21:30.358631 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:21:30.358597 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda582740b_395e_485b_b214_81c6f58a7747.slice/crio-e0b988b4f2530118e4296fcdaaddef639ba0b0404261903633a099033481075e WatchSource:0}: Error finding container e0b988b4f2530118e4296fcdaaddef639ba0b0404261903633a099033481075e: Status 404 returned error can't find the container with id e0b988b4f2530118e4296fcdaaddef639ba0b0404261903633a099033481075e Jul 06 01:21:30.500935 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.500769 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ca215253-2060-46f3-94c8-2b48a3fc81a2-node-exporter-tls\") pod \"node-exporter-cjx7b\" (UID: \"ca215253-2060-46f3-94c8-2b48a3fc81a2\") " pod="openshift-monitoring/node-exporter-cjx7b" Jul 06 01:21:30.504282 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.504211 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ca215253-2060-46f3-94c8-2b48a3fc81a2-node-exporter-tls\") pod \"node-exporter-cjx7b\" (UID: \"ca215253-2060-46f3-94c8-2b48a3fc81a2\") " pod="openshift-monitoring/node-exporter-cjx7b" Jul 06 01:21:30.549637 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.549452 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-744494649-m22gf"] Jul 06 01:21:30.552844 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.552304 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-744494649-m22gf" Jul 06 01:21:30.555206 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.555179 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-vjkln\"" Jul 06 01:21:30.555357 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.555222 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 06 01:21:30.555677 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.555509 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 06 01:21:30.555677 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.555604 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 06 01:21:30.555677 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.555518 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 06 01:21:30.555879 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.555734 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 06 01:21:30.577855 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.576965 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-744494649-m22gf"] Jul 06 01:21:30.601520 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.601479 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljz6s\" (UniqueName: \"kubernetes.io/projected/b35757f2-50a7-414b-84b0-b560dec30ae1-kube-api-access-ljz6s\") pod \"console-744494649-m22gf\" (UID: \"b35757f2-50a7-414b-84b0-b560dec30ae1\") " pod="openshift-console/console-744494649-m22gf" Jul 06 01:21:30.601715 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.601529 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b35757f2-50a7-414b-84b0-b560dec30ae1-console-config\") pod \"console-744494649-m22gf\" (UID: \"b35757f2-50a7-414b-84b0-b560dec30ae1\") " pod="openshift-console/console-744494649-m22gf" Jul 06 01:21:30.601715 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.601606 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b35757f2-50a7-414b-84b0-b560dec30ae1-oauth-serving-cert\") pod \"console-744494649-m22gf\" (UID: \"b35757f2-50a7-414b-84b0-b560dec30ae1\") " pod="openshift-console/console-744494649-m22gf" Jul 06 01:21:30.601715 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.601687 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b35757f2-50a7-414b-84b0-b560dec30ae1-console-serving-cert\") pod \"console-744494649-m22gf\" (UID: \"b35757f2-50a7-414b-84b0-b560dec30ae1\") " pod="openshift-console/console-744494649-m22gf" Jul 06 01:21:30.601846 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.601732 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b35757f2-50a7-414b-84b0-b560dec30ae1-service-ca\") pod \"console-744494649-m22gf\" (UID: \"b35757f2-50a7-414b-84b0-b560dec30ae1\") " pod="openshift-console/console-744494649-m22gf" Jul 06 01:21:30.601846 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.601804 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b35757f2-50a7-414b-84b0-b560dec30ae1-console-oauth-config\") pod \"console-744494649-m22gf\" (UID: \"b35757f2-50a7-414b-84b0-b560dec30ae1\") " pod="openshift-console/console-744494649-m22gf" Jul 06 01:21:30.630444 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.630368 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ksnx7" event={"ID":"a582740b-395e-485b-b214-81c6f58a7747","Type":"ContainerStarted","Data":"e0b988b4f2530118e4296fcdaaddef639ba0b0404261903633a099033481075e"} Jul 06 01:21:30.633153 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.633063 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-448pq" event={"ID":"50574349-d208-4de6-aba3-06e535105c6a","Type":"ContainerStarted","Data":"80142b6be1a0efc856c4a1efa9e8786c38a411d1abf188290c3cdaf8b344a4d3"} Jul 06 01:21:30.633153 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.633151 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-448pq" event={"ID":"50574349-d208-4de6-aba3-06e535105c6a","Type":"ContainerStarted","Data":"f98729955b8d1f221f58653dd9d97860cff8a68694ba98e974236e92356b71ac"} Jul 06 01:21:30.633364 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.633178 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-448pq" event={"ID":"50574349-d208-4de6-aba3-06e535105c6a","Type":"ContainerStarted","Data":"475ac34c32766460b591a3566a853b31698c95299e28eff202ddb3905aff8063"} Jul 06 01:21:30.703529 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.703434 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b35757f2-50a7-414b-84b0-b560dec30ae1-console-oauth-config\") pod \"console-744494649-m22gf\" (UID: \"b35757f2-50a7-414b-84b0-b560dec30ae1\") " pod="openshift-console/console-744494649-m22gf" Jul 06 01:21:30.703529 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.703510 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljz6s\" (UniqueName: \"kubernetes.io/projected/b35757f2-50a7-414b-84b0-b560dec30ae1-kube-api-access-ljz6s\") pod \"console-744494649-m22gf\" (UID: \"b35757f2-50a7-414b-84b0-b560dec30ae1\") " pod="openshift-console/console-744494649-m22gf" Jul 06 01:21:30.703770 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.703545 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b35757f2-50a7-414b-84b0-b560dec30ae1-console-config\") pod \"console-744494649-m22gf\" (UID: \"b35757f2-50a7-414b-84b0-b560dec30ae1\") " pod="openshift-console/console-744494649-m22gf" Jul 06 01:21:30.703770 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.703605 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b35757f2-50a7-414b-84b0-b560dec30ae1-oauth-serving-cert\") pod \"console-744494649-m22gf\" (UID: \"b35757f2-50a7-414b-84b0-b560dec30ae1\") " pod="openshift-console/console-744494649-m22gf" Jul 06 01:21:30.703770 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.703652 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b35757f2-50a7-414b-84b0-b560dec30ae1-console-serving-cert\") pod \"console-744494649-m22gf\" (UID: \"b35757f2-50a7-414b-84b0-b560dec30ae1\") " pod="openshift-console/console-744494649-m22gf" Jul 06 01:21:30.703770 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.703676 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b35757f2-50a7-414b-84b0-b560dec30ae1-service-ca\") pod \"console-744494649-m22gf\" (UID: \"b35757f2-50a7-414b-84b0-b560dec30ae1\") " pod="openshift-console/console-744494649-m22gf" Jul 06 01:21:30.704649 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.704627 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b35757f2-50a7-414b-84b0-b560dec30ae1-service-ca\") pod \"console-744494649-m22gf\" (UID: \"b35757f2-50a7-414b-84b0-b560dec30ae1\") " pod="openshift-console/console-744494649-m22gf" Jul 06 01:21:30.704953 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.704929 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b35757f2-50a7-414b-84b0-b560dec30ae1-console-config\") pod \"console-744494649-m22gf\" (UID: \"b35757f2-50a7-414b-84b0-b560dec30ae1\") " pod="openshift-console/console-744494649-m22gf" Jul 06 01:21:30.705231 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.705187 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b35757f2-50a7-414b-84b0-b560dec30ae1-oauth-serving-cert\") pod \"console-744494649-m22gf\" (UID: \"b35757f2-50a7-414b-84b0-b560dec30ae1\") " pod="openshift-console/console-744494649-m22gf" Jul 06 01:21:30.707834 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.707783 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b35757f2-50a7-414b-84b0-b560dec30ae1-console-oauth-config\") pod \"console-744494649-m22gf\" (UID: \"b35757f2-50a7-414b-84b0-b560dec30ae1\") " pod="openshift-console/console-744494649-m22gf" Jul 06 01:21:30.707957 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.707832 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b35757f2-50a7-414b-84b0-b560dec30ae1-console-serving-cert\") pod \"console-744494649-m22gf\" (UID: \"b35757f2-50a7-414b-84b0-b560dec30ae1\") " pod="openshift-console/console-744494649-m22gf" Jul 06 01:21:30.721103 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.721068 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljz6s\" (UniqueName: \"kubernetes.io/projected/b35757f2-50a7-414b-84b0-b560dec30ae1-kube-api-access-ljz6s\") pod \"console-744494649-m22gf\" (UID: \"b35757f2-50a7-414b-84b0-b560dec30ae1\") " pod="openshift-console/console-744494649-m22gf" Jul 06 01:21:30.727501 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.727459 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cjx7b" Jul 06 01:21:30.851064 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.851018 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 01:21:30.856521 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.855750 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:30.860665 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.859188 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 06 01:21:30.860665 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.859742 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 06 01:21:30.860665 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.860124 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-4fq5l\"" Jul 06 01:21:30.860665 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.860329 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 06 01:21:30.860665 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.860517 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 06 01:21:30.861050 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.860685 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 06 01:21:30.861050 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.860736 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 06 01:21:30.861050 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.860766 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 06 01:21:30.861050 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.860856 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 06 01:21:30.861050 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.860966 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 06 01:21:30.866141 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.866112 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-744494649-m22gf" Jul 06 01:21:30.883455 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.883312 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 01:21:30.906045 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.906007 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7bf690b-525a-4310-aeb2-206ab91e6346-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:30.906227 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.906061 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wc4b\" (UniqueName: \"kubernetes.io/projected/d7bf690b-525a-4310-aeb2-206ab91e6346-kube-api-access-8wc4b\") pod \"alertmanager-main-0\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:30.906227 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.906103 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:30.906227 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.906157 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d7bf690b-525a-4310-aeb2-206ab91e6346-config-out\") pod \"alertmanager-main-0\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:30.906227 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.906195 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-web-config\") pod \"alertmanager-main-0\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:30.906227 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.906222 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:30.906522 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.906277 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:30.906522 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.906324 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d7bf690b-525a-4310-aeb2-206ab91e6346-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:30.906522 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.906415 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d7bf690b-525a-4310-aeb2-206ab91e6346-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:30.906522 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.906444 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-config-volume\") pod \"alertmanager-main-0\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:30.906522 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.906471 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7bf690b-525a-4310-aeb2-206ab91e6346-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:30.906522 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.906505 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:30.906807 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:30.906544 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:31.008768 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.007666 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d7bf690b-525a-4310-aeb2-206ab91e6346-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:31.008768 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.007718 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-config-volume\") pod \"alertmanager-main-0\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:31.008768 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.007744 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7bf690b-525a-4310-aeb2-206ab91e6346-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:31.008768 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.007774 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:31.008768 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.007809 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:31.008768 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.007862 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7bf690b-525a-4310-aeb2-206ab91e6346-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:31.008768 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.007914 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wc4b\" (UniqueName: \"kubernetes.io/projected/d7bf690b-525a-4310-aeb2-206ab91e6346-kube-api-access-8wc4b\") pod \"alertmanager-main-0\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:31.008768 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.007949 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:31.008768 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.007976 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d7bf690b-525a-4310-aeb2-206ab91e6346-config-out\") pod \"alertmanager-main-0\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:31.008768 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.008000 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-web-config\") pod \"alertmanager-main-0\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:31.008768 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.008037 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:31.008768 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.008107 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:31.008768 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.008144 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d7bf690b-525a-4310-aeb2-206ab91e6346-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:31.010475 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.010118 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d7bf690b-525a-4310-aeb2-206ab91e6346-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:31.010475 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:31.010399 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d7bf690b-525a-4310-aeb2-206ab91e6346-alertmanager-trusted-ca-bundle podName:d7bf690b-525a-4310-aeb2-206ab91e6346 nodeName:}" failed. No retries permitted until 2026-07-06 01:21:31.510350104 +0000 UTC m=+63.651908867 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/d7bf690b-525a-4310-aeb2-206ab91e6346-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "d7bf690b-525a-4310-aeb2-206ab91e6346") : configmap references non-existent config key: ca-bundle.crt Jul 06 01:21:31.010475 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:31.010476 2565 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 06 01:21:31.010747 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:21:31.010552 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-secret-alertmanager-main-tls podName:d7bf690b-525a-4310-aeb2-206ab91e6346 nodeName:}" failed. No retries permitted until 2026-07-06 01:21:31.510533906 +0000 UTC m=+63.652092684 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "d7bf690b-525a-4310-aeb2-206ab91e6346") : secret "alertmanager-main-tls" not found Jul 06 01:21:31.011679 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.011255 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7bf690b-525a-4310-aeb2-206ab91e6346-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:31.020356 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.018871 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:31.020356 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.019493 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-config-volume\") pod \"alertmanager-main-0\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:31.020356 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.019919 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:31.020356 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.020291 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d7bf690b-525a-4310-aeb2-206ab91e6346-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:31.021192 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.020831 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:31.021192 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.021038 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d7bf690b-525a-4310-aeb2-206ab91e6346-config-out\") pod \"alertmanager-main-0\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:31.021424 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.021232 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-web-config\") pod \"alertmanager-main-0\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:31.025468 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.025336 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wc4b\" (UniqueName: \"kubernetes.io/projected/d7bf690b-525a-4310-aeb2-206ab91e6346-kube-api-access-8wc4b\") pod \"alertmanager-main-0\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:31.026855 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.026751 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:31.057292 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.057251 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-744494649-m22gf"] Jul 06 01:21:31.066751 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:21:31.066535 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb35757f2_50a7_414b_84b0_b560dec30ae1.slice/crio-680ab4262beaebb975ab31dfc00704a009adea64b89d3d0f409512985a95c336 WatchSource:0}: Error finding container 680ab4262beaebb975ab31dfc00704a009adea64b89d3d0f409512985a95c336: Status 404 returned error can't find the container with id 680ab4262beaebb975ab31dfc00704a009adea64b89d3d0f409512985a95c336 Jul 06 01:21:31.514245 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.514199 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:31.514436 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.514302 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7bf690b-525a-4310-aeb2-206ab91e6346-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:31.515319 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.515285 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7bf690b-525a-4310-aeb2-206ab91e6346-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:31.524333 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.524297 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:31.640611 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.640552 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cjx7b" event={"ID":"ca215253-2060-46f3-94c8-2b48a3fc81a2","Type":"ContainerStarted","Data":"198b2a3bd149eb18112fb842c40dd303e0d07d8021190aa792786ef5a908b259"} Jul 06 01:21:31.643409 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.643354 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-744494649-m22gf" event={"ID":"b35757f2-50a7-414b-84b0-b560dec30ae1","Type":"ContainerStarted","Data":"680ab4262beaebb975ab31dfc00704a009adea64b89d3d0f409512985a95c336"} Jul 06 01:21:31.771125 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.771034 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:21:31.857009 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.856961 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-857dc8f57d-rtklb"] Jul 06 01:21:31.861327 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.861301 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-857dc8f57d-rtklb" Jul 06 01:21:31.863872 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.863839 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 06 01:21:31.864023 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.863939 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 06 01:21:31.864023 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.863963 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 06 01:21:31.864023 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.863999 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 06 01:21:31.864023 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.864008 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-38d03gjfr6qt1\"" Jul 06 01:21:31.864187 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.863939 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-xc7qt\"" Jul 06 01:21:31.864439 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.864419 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 06 01:21:31.872046 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.871296 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-857dc8f57d-rtklb"] Jul 06 01:21:31.918513 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.918421 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2b5d11a6-bb09-4c32-9e61-db7732341d0c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-857dc8f57d-rtklb\" (UID: \"2b5d11a6-bb09-4c32-9e61-db7732341d0c\") " pod="openshift-monitoring/thanos-querier-857dc8f57d-rtklb" Jul 06 01:21:31.918513 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.918460 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/2b5d11a6-bb09-4c32-9e61-db7732341d0c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-857dc8f57d-rtklb\" (UID: \"2b5d11a6-bb09-4c32-9e61-db7732341d0c\") " pod="openshift-monitoring/thanos-querier-857dc8f57d-rtklb" Jul 06 01:21:31.918726 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.918595 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlgsw\" (UniqueName: \"kubernetes.io/projected/2b5d11a6-bb09-4c32-9e61-db7732341d0c-kube-api-access-tlgsw\") pod \"thanos-querier-857dc8f57d-rtklb\" (UID: \"2b5d11a6-bb09-4c32-9e61-db7732341d0c\") " pod="openshift-monitoring/thanos-querier-857dc8f57d-rtklb" Jul 06 01:21:31.918726 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.918680 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/2b5d11a6-bb09-4c32-9e61-db7732341d0c-secret-thanos-querier-tls\") pod \"thanos-querier-857dc8f57d-rtklb\" (UID: \"2b5d11a6-bb09-4c32-9e61-db7732341d0c\") " pod="openshift-monitoring/thanos-querier-857dc8f57d-rtklb" Jul 06 01:21:31.918810 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.918759 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2b5d11a6-bb09-4c32-9e61-db7732341d0c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-857dc8f57d-rtklb\" (UID: \"2b5d11a6-bb09-4c32-9e61-db7732341d0c\") " pod="openshift-monitoring/thanos-querier-857dc8f57d-rtklb" Jul 06 01:21:31.918810 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.918788 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2b5d11a6-bb09-4c32-9e61-db7732341d0c-secret-grpc-tls\") pod \"thanos-querier-857dc8f57d-rtklb\" (UID: \"2b5d11a6-bb09-4c32-9e61-db7732341d0c\") " pod="openshift-monitoring/thanos-querier-857dc8f57d-rtklb" Jul 06 01:21:31.918909 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.918821 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/2b5d11a6-bb09-4c32-9e61-db7732341d0c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-857dc8f57d-rtklb\" (UID: \"2b5d11a6-bb09-4c32-9e61-db7732341d0c\") " pod="openshift-monitoring/thanos-querier-857dc8f57d-rtklb" Jul 06 01:21:31.918909 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:31.918867 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2b5d11a6-bb09-4c32-9e61-db7732341d0c-metrics-client-ca\") pod \"thanos-querier-857dc8f57d-rtklb\" (UID: \"2b5d11a6-bb09-4c32-9e61-db7732341d0c\") " pod="openshift-monitoring/thanos-querier-857dc8f57d-rtklb" Jul 06 01:21:32.019437 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:32.019398 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/2b5d11a6-bb09-4c32-9e61-db7732341d0c-secret-thanos-querier-tls\") pod \"thanos-querier-857dc8f57d-rtklb\" (UID: \"2b5d11a6-bb09-4c32-9e61-db7732341d0c\") " pod="openshift-monitoring/thanos-querier-857dc8f57d-rtklb" Jul 06 01:21:32.019595 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:32.019504 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2b5d11a6-bb09-4c32-9e61-db7732341d0c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-857dc8f57d-rtklb\" (UID: \"2b5d11a6-bb09-4c32-9e61-db7732341d0c\") " pod="openshift-monitoring/thanos-querier-857dc8f57d-rtklb" Jul 06 01:21:32.019595 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:32.019526 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2b5d11a6-bb09-4c32-9e61-db7732341d0c-secret-grpc-tls\") pod \"thanos-querier-857dc8f57d-rtklb\" (UID: \"2b5d11a6-bb09-4c32-9e61-db7732341d0c\") " pod="openshift-monitoring/thanos-querier-857dc8f57d-rtklb" Jul 06 01:21:32.019595 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:32.019547 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/2b5d11a6-bb09-4c32-9e61-db7732341d0c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-857dc8f57d-rtklb\" (UID: \"2b5d11a6-bb09-4c32-9e61-db7732341d0c\") " pod="openshift-monitoring/thanos-querier-857dc8f57d-rtklb" Jul 06 01:21:32.019595 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:32.019580 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2b5d11a6-bb09-4c32-9e61-db7732341d0c-metrics-client-ca\") pod \"thanos-querier-857dc8f57d-rtklb\" (UID: \"2b5d11a6-bb09-4c32-9e61-db7732341d0c\") " pod="openshift-monitoring/thanos-querier-857dc8f57d-rtklb" Jul 06 01:21:32.019823 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:32.019622 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2b5d11a6-bb09-4c32-9e61-db7732341d0c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-857dc8f57d-rtklb\" (UID: \"2b5d11a6-bb09-4c32-9e61-db7732341d0c\") " pod="openshift-monitoring/thanos-querier-857dc8f57d-rtklb" Jul 06 01:21:32.019823 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:32.019649 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/2b5d11a6-bb09-4c32-9e61-db7732341d0c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-857dc8f57d-rtklb\" (UID: \"2b5d11a6-bb09-4c32-9e61-db7732341d0c\") " pod="openshift-monitoring/thanos-querier-857dc8f57d-rtklb" Jul 06 01:21:32.019823 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:32.019708 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlgsw\" (UniqueName: \"kubernetes.io/projected/2b5d11a6-bb09-4c32-9e61-db7732341d0c-kube-api-access-tlgsw\") pod \"thanos-querier-857dc8f57d-rtklb\" (UID: \"2b5d11a6-bb09-4c32-9e61-db7732341d0c\") " pod="openshift-monitoring/thanos-querier-857dc8f57d-rtklb" Jul 06 01:21:32.020464 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:32.020414 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2b5d11a6-bb09-4c32-9e61-db7732341d0c-metrics-client-ca\") pod \"thanos-querier-857dc8f57d-rtklb\" (UID: \"2b5d11a6-bb09-4c32-9e61-db7732341d0c\") " pod="openshift-monitoring/thanos-querier-857dc8f57d-rtklb" Jul 06 01:21:32.023070 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:32.023000 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2b5d11a6-bb09-4c32-9e61-db7732341d0c-secret-grpc-tls\") pod \"thanos-querier-857dc8f57d-rtklb\" (UID: \"2b5d11a6-bb09-4c32-9e61-db7732341d0c\") " pod="openshift-monitoring/thanos-querier-857dc8f57d-rtklb" Jul 06 01:21:32.023070 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:32.022999 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/2b5d11a6-bb09-4c32-9e61-db7732341d0c-secret-thanos-querier-tls\") pod \"thanos-querier-857dc8f57d-rtklb\" (UID: \"2b5d11a6-bb09-4c32-9e61-db7732341d0c\") " pod="openshift-monitoring/thanos-querier-857dc8f57d-rtklb" Jul 06 01:21:32.023070 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:32.023045 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/2b5d11a6-bb09-4c32-9e61-db7732341d0c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-857dc8f57d-rtklb\" (UID: \"2b5d11a6-bb09-4c32-9e61-db7732341d0c\") " pod="openshift-monitoring/thanos-querier-857dc8f57d-rtklb" Jul 06 01:21:32.023290 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:32.023095 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2b5d11a6-bb09-4c32-9e61-db7732341d0c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-857dc8f57d-rtklb\" (UID: \"2b5d11a6-bb09-4c32-9e61-db7732341d0c\") " pod="openshift-monitoring/thanos-querier-857dc8f57d-rtklb" Jul 06 01:21:32.023447 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:32.023429 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2b5d11a6-bb09-4c32-9e61-db7732341d0c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-857dc8f57d-rtklb\" (UID: \"2b5d11a6-bb09-4c32-9e61-db7732341d0c\") " pod="openshift-monitoring/thanos-querier-857dc8f57d-rtklb" Jul 06 01:21:32.023729 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:32.023700 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/2b5d11a6-bb09-4c32-9e61-db7732341d0c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-857dc8f57d-rtklb\" (UID: \"2b5d11a6-bb09-4c32-9e61-db7732341d0c\") " pod="openshift-monitoring/thanos-querier-857dc8f57d-rtklb" Jul 06 01:21:32.031988 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:32.031927 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlgsw\" (UniqueName: \"kubernetes.io/projected/2b5d11a6-bb09-4c32-9e61-db7732341d0c-kube-api-access-tlgsw\") pod \"thanos-querier-857dc8f57d-rtklb\" (UID: \"2b5d11a6-bb09-4c32-9e61-db7732341d0c\") " pod="openshift-monitoring/thanos-querier-857dc8f57d-rtklb" Jul 06 01:21:32.175116 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:32.175071 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-857dc8f57d-rtklb" Jul 06 01:21:32.588447 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:32.588408 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-tfzf9" Jul 06 01:21:34.040009 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.039970 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vblkv\" (UniqueName: \"kubernetes.io/projected/8bcfcc54-ba2d-4049-9077-6cc8ce713838-kube-api-access-vblkv\") pod \"network-check-target-4ss2v\" (UID: \"8bcfcc54-ba2d-4049-9077-6cc8ce713838\") " pod="openshift-network-diagnostics/network-check-target-4ss2v" Jul 06 01:21:34.040009 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.040024 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c0a1cea-8dca-4e9d-afdb-81f874754505-original-pull-secret\") pod \"global-pull-secret-syncer-pm8wq\" (UID: \"4c0a1cea-8dca-4e9d-afdb-81f874754505\") " pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:21:34.040788 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.040074 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbe62245-715f-4565-bfa8-af7c3ab5d36a-metrics-certs\") pod \"network-metrics-daemon-drnhm\" (UID: \"fbe62245-715f-4565-bfa8-af7c3ab5d36a\") " pod="openshift-multus/network-metrics-daemon-drnhm" Jul 06 01:21:34.042711 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.042682 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 01:21:34.045632 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.045605 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 01:21:34.045632 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.045631 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 01:21:34.052897 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.052866 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 01:21:34.053582 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.053554 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c0a1cea-8dca-4e9d-afdb-81f874754505-original-pull-secret\") pod \"global-pull-secret-syncer-pm8wq\" (UID: \"4c0a1cea-8dca-4e9d-afdb-81f874754505\") " pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:21:34.053718 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.053600 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbe62245-715f-4565-bfa8-af7c3ab5d36a-metrics-certs\") pod \"network-metrics-daemon-drnhm\" (UID: \"fbe62245-715f-4565-bfa8-af7c3ab5d36a\") " pod="openshift-multus/network-metrics-daemon-drnhm" Jul 06 01:21:34.064183 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.064114 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vblkv\" (UniqueName: \"kubernetes.io/projected/8bcfcc54-ba2d-4049-9077-6cc8ce713838-kube-api-access-vblkv\") pod \"network-check-target-4ss2v\" (UID: \"8bcfcc54-ba2d-4049-9077-6cc8ce713838\") " pod="openshift-network-diagnostics/network-check-target-4ss2v" Jul 06 01:21:34.092979 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.092939 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-88b699c9-dc7x2"] Jul 06 01:21:34.097817 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.097782 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-88b699c9-dc7x2" Jul 06 01:21:34.103326 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.103163 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-9h801qj1t6tgo\"" Jul 06 01:21:34.103808 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.103778 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 06 01:21:34.103808 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.103795 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 06 01:21:34.103991 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.103828 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 06 01:21:34.104217 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.104197 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-tkfxs\"" Jul 06 01:21:34.104308 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.104197 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 06 01:21:34.112179 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.112147 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-88b699c9-dc7x2"] Jul 06 01:21:34.141333 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.141289 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e8e13654-1d6d-436b-a582-40074d4bdde2-secret-metrics-server-tls\") pod \"metrics-server-88b699c9-dc7x2\" (UID: \"e8e13654-1d6d-436b-a582-40074d4bdde2\") " pod="openshift-monitoring/metrics-server-88b699c9-dc7x2" Jul 06 01:21:34.141559 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.141359 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e8e13654-1d6d-436b-a582-40074d4bdde2-secret-metrics-server-client-certs\") pod \"metrics-server-88b699c9-dc7x2\" (UID: \"e8e13654-1d6d-436b-a582-40074d4bdde2\") " pod="openshift-monitoring/metrics-server-88b699c9-dc7x2" Jul 06 01:21:34.141559 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.141403 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e8e13654-1d6d-436b-a582-40074d4bdde2-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-88b699c9-dc7x2\" (UID: \"e8e13654-1d6d-436b-a582-40074d4bdde2\") " pod="openshift-monitoring/metrics-server-88b699c9-dc7x2" Jul 06 01:21:34.141559 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.141433 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e8e13654-1d6d-436b-a582-40074d4bdde2-metrics-server-audit-profiles\") pod \"metrics-server-88b699c9-dc7x2\" (UID: \"e8e13654-1d6d-436b-a582-40074d4bdde2\") " pod="openshift-monitoring/metrics-server-88b699c9-dc7x2" Jul 06 01:21:34.141559 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.141510 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2xhz\" (UniqueName: \"kubernetes.io/projected/e8e13654-1d6d-436b-a582-40074d4bdde2-kube-api-access-r2xhz\") pod \"metrics-server-88b699c9-dc7x2\" (UID: \"e8e13654-1d6d-436b-a582-40074d4bdde2\") " pod="openshift-monitoring/metrics-server-88b699c9-dc7x2" Jul 06 01:21:34.141559 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.141542 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e8e13654-1d6d-436b-a582-40074d4bdde2-client-ca-bundle\") pod \"metrics-server-88b699c9-dc7x2\" (UID: \"e8e13654-1d6d-436b-a582-40074d4bdde2\") " pod="openshift-monitoring/metrics-server-88b699c9-dc7x2" Jul 06 01:21:34.141786 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.141574 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e8e13654-1d6d-436b-a582-40074d4bdde2-audit-log\") pod \"metrics-server-88b699c9-dc7x2\" (UID: \"e8e13654-1d6d-436b-a582-40074d4bdde2\") " pod="openshift-monitoring/metrics-server-88b699c9-dc7x2" Jul 06 01:21:34.233469 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.233438 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6kdr7\"" Jul 06 01:21:34.238918 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.238889 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9bqkt\"" Jul 06 01:21:34.242018 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.241989 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ss2v" Jul 06 01:21:34.242321 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.242299 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e8e13654-1d6d-436b-a582-40074d4bdde2-audit-log\") pod \"metrics-server-88b699c9-dc7x2\" (UID: \"e8e13654-1d6d-436b-a582-40074d4bdde2\") " pod="openshift-monitoring/metrics-server-88b699c9-dc7x2" Jul 06 01:21:34.242464 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.242356 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e8e13654-1d6d-436b-a582-40074d4bdde2-secret-metrics-server-tls\") pod \"metrics-server-88b699c9-dc7x2\" (UID: \"e8e13654-1d6d-436b-a582-40074d4bdde2\") " pod="openshift-monitoring/metrics-server-88b699c9-dc7x2" Jul 06 01:21:34.242464 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.242421 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e8e13654-1d6d-436b-a582-40074d4bdde2-secret-metrics-server-client-certs\") pod \"metrics-server-88b699c9-dc7x2\" (UID: \"e8e13654-1d6d-436b-a582-40074d4bdde2\") " pod="openshift-monitoring/metrics-server-88b699c9-dc7x2" Jul 06 01:21:34.242464 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.242449 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e8e13654-1d6d-436b-a582-40074d4bdde2-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-88b699c9-dc7x2\" (UID: \"e8e13654-1d6d-436b-a582-40074d4bdde2\") " pod="openshift-monitoring/metrics-server-88b699c9-dc7x2" Jul 06 01:21:34.242623 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.242474 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e8e13654-1d6d-436b-a582-40074d4bdde2-metrics-server-audit-profiles\") pod \"metrics-server-88b699c9-dc7x2\" (UID: \"e8e13654-1d6d-436b-a582-40074d4bdde2\") " pod="openshift-monitoring/metrics-server-88b699c9-dc7x2" Jul 06 01:21:34.242623 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.242482 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pm8wq" Jul 06 01:21:34.242623 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.242556 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2xhz\" (UniqueName: \"kubernetes.io/projected/e8e13654-1d6d-436b-a582-40074d4bdde2-kube-api-access-r2xhz\") pod \"metrics-server-88b699c9-dc7x2\" (UID: \"e8e13654-1d6d-436b-a582-40074d4bdde2\") " pod="openshift-monitoring/metrics-server-88b699c9-dc7x2" Jul 06 01:21:34.242623 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.242591 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e8e13654-1d6d-436b-a582-40074d4bdde2-client-ca-bundle\") pod \"metrics-server-88b699c9-dc7x2\" (UID: \"e8e13654-1d6d-436b-a582-40074d4bdde2\") " pod="openshift-monitoring/metrics-server-88b699c9-dc7x2" Jul 06 01:21:34.242816 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.242793 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e8e13654-1d6d-436b-a582-40074d4bdde2-audit-log\") pod \"metrics-server-88b699c9-dc7x2\" (UID: \"e8e13654-1d6d-436b-a582-40074d4bdde2\") " pod="openshift-monitoring/metrics-server-88b699c9-dc7x2" Jul 06 01:21:34.243585 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.243532 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e8e13654-1d6d-436b-a582-40074d4bdde2-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-88b699c9-dc7x2\" (UID: \"e8e13654-1d6d-436b-a582-40074d4bdde2\") " pod="openshift-monitoring/metrics-server-88b699c9-dc7x2" Jul 06 01:21:34.243718 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.243702 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e8e13654-1d6d-436b-a582-40074d4bdde2-metrics-server-audit-profiles\") pod \"metrics-server-88b699c9-dc7x2\" (UID: \"e8e13654-1d6d-436b-a582-40074d4bdde2\") " pod="openshift-monitoring/metrics-server-88b699c9-dc7x2" Jul 06 01:21:34.245478 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.245368 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e8e13654-1d6d-436b-a582-40074d4bdde2-secret-metrics-server-tls\") pod \"metrics-server-88b699c9-dc7x2\" (UID: \"e8e13654-1d6d-436b-a582-40074d4bdde2\") " pod="openshift-monitoring/metrics-server-88b699c9-dc7x2" Jul 06 01:21:34.245478 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.245440 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e8e13654-1d6d-436b-a582-40074d4bdde2-secret-metrics-server-client-certs\") pod \"metrics-server-88b699c9-dc7x2\" (UID: \"e8e13654-1d6d-436b-a582-40074d4bdde2\") " pod="openshift-monitoring/metrics-server-88b699c9-dc7x2" Jul 06 01:21:34.246153 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.245982 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e8e13654-1d6d-436b-a582-40074d4bdde2-client-ca-bundle\") pod \"metrics-server-88b699c9-dc7x2\" (UID: \"e8e13654-1d6d-436b-a582-40074d4bdde2\") " pod="openshift-monitoring/metrics-server-88b699c9-dc7x2" Jul 06 01:21:34.247495 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.247474 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-drnhm" Jul 06 01:21:34.254977 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.254941 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2xhz\" (UniqueName: \"kubernetes.io/projected/e8e13654-1d6d-436b-a582-40074d4bdde2-kube-api-access-r2xhz\") pod \"metrics-server-88b699c9-dc7x2\" (UID: \"e8e13654-1d6d-436b-a582-40074d4bdde2\") " pod="openshift-monitoring/metrics-server-88b699c9-dc7x2" Jul 06 01:21:34.411028 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:34.410937 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-88b699c9-dc7x2" Jul 06 01:21:36.047453 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.047419 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 01:21:36.071717 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.071680 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 01:21:36.072684 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.071910 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.075114 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.075087 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 06 01:21:36.075276 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.075248 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 06 01:21:36.075520 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.075502 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-fpc9i5daqe19b\"" Jul 06 01:21:36.075961 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.075942 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 06 01:21:36.076222 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.076195 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 06 01:21:36.076686 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.076667 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 06 01:21:36.076781 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.076671 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 06 01:21:36.076893 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.076874 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 06 01:21:36.076935 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.076920 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 06 01:21:36.077100 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.077082 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 06 01:21:36.077203 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.077117 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 06 01:21:36.077434 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.077272 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 06 01:21:36.078819 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.078277 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-mhhpf\"" Jul 06 01:21:36.078819 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.078433 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 06 01:21:36.078819 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.078683 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 06 01:21:36.080959 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.080939 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 06 01:21:36.161329 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.161277 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f5cf93c2-af80-4384-add5-49cdd601b40c-config-out\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.161329 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.161325 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.161598 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.161351 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.161598 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.161427 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f5cf93c2-af80-4384-add5-49cdd601b40c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.161598 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.161462 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5cf93c2-af80-4384-add5-49cdd601b40c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.161598 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.161492 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.161598 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.161510 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfs5k\" (UniqueName: \"kubernetes.io/projected/f5cf93c2-af80-4384-add5-49cdd601b40c-kube-api-access-dfs5k\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.161598 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.161540 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f5cf93c2-af80-4384-add5-49cdd601b40c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.161598 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.161567 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-web-config\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.161598 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.161590 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.161998 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.161628 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.161998 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.161686 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f5cf93c2-af80-4384-add5-49cdd601b40c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.161998 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.161706 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.161998 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.161752 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-config\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.161998 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.161774 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5cf93c2-af80-4384-add5-49cdd601b40c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.161998 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.161812 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.161998 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.161837 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f5cf93c2-af80-4384-add5-49cdd601b40c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.161998 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.161860 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f5cf93c2-af80-4384-add5-49cdd601b40c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.161998 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.161890 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5cf93c2-af80-4384-add5-49cdd601b40c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.161998 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.161919 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f5cf93c2-af80-4384-add5-49cdd601b40c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.262863 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.262826 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.263076 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.262872 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f5cf93c2-af80-4384-add5-49cdd601b40c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.263076 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.262900 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.263076 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.262943 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-config\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.263076 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.262967 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5cf93c2-af80-4384-add5-49cdd601b40c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.263076 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.263008 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.263076 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.263033 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f5cf93c2-af80-4384-add5-49cdd601b40c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.263076 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.263053 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f5cf93c2-af80-4384-add5-49cdd601b40c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.263457 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.263085 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5cf93c2-af80-4384-add5-49cdd601b40c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.263457 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.263116 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f5cf93c2-af80-4384-add5-49cdd601b40c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.263457 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.263150 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f5cf93c2-af80-4384-add5-49cdd601b40c-config-out\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.263457 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.263176 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.263457 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.263212 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.263457 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.263247 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f5cf93c2-af80-4384-add5-49cdd601b40c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.263457 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.263279 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5cf93c2-af80-4384-add5-49cdd601b40c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.263457 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.263311 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.263457 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.263340 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfs5k\" (UniqueName: \"kubernetes.io/projected/f5cf93c2-af80-4384-add5-49cdd601b40c-kube-api-access-dfs5k\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.263457 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.263395 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f5cf93c2-af80-4384-add5-49cdd601b40c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.263457 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.263420 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-web-config\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.263457 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.263451 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.264169 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.263981 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f5cf93c2-af80-4384-add5-49cdd601b40c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.264564 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.264513 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f5cf93c2-af80-4384-add5-49cdd601b40c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.267689 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.267655 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5cf93c2-af80-4384-add5-49cdd601b40c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.269978 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.269273 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f5cf93c2-af80-4384-add5-49cdd601b40c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.270138 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.270081 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.270241 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.270211 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f5cf93c2-af80-4384-add5-49cdd601b40c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.270341 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.270319 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f5cf93c2-af80-4384-add5-49cdd601b40c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.270457 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.270417 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5cf93c2-af80-4384-add5-49cdd601b40c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.270668 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.270647 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f5cf93c2-af80-4384-add5-49cdd601b40c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.271489 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.270748 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.271489 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.270879 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5cf93c2-af80-4384-add5-49cdd601b40c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.271489 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.271415 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.271489 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.271478 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.272486 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.272433 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-config\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.273479 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.273451 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.274084 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.273689 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f5cf93c2-af80-4384-add5-49cdd601b40c-config-out\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.274084 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.274046 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-web-config\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.274240 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.274081 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.274822 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.274789 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.277740 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.277714 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfs5k\" (UniqueName: \"kubernetes.io/projected/f5cf93c2-af80-4384-add5-49cdd601b40c-kube-api-access-dfs5k\") pod \"prometheus-k8s-0\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:36.390157 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:36.390052 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:21:41.915027 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:41.914987 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-857dc8f57d-rtklb"] Jul 06 01:21:43.178713 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:21:43.178632 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b5d11a6_bb09_4c32_9e61_db7732341d0c.slice/crio-092f6cce4fe05ed4c1c451ba01898ceabfe5bcf89fd1515c77123ce5f1c93a2a WatchSource:0}: Error finding container 092f6cce4fe05ed4c1c451ba01898ceabfe5bcf89fd1515c77123ce5f1c93a2a: Status 404 returned error can't find the container with id 092f6cce4fe05ed4c1c451ba01898ceabfe5bcf89fd1515c77123ce5f1c93a2a Jul 06 01:21:43.522035 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:43.518077 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4ss2v"] Jul 06 01:21:43.580463 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:43.579656 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-88b699c9-dc7x2"] Jul 06 01:21:43.580463 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:43.579876 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 01:21:43.625737 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:43.625663 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-drnhm"] Jul 06 01:21:43.640861 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:43.640658 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pm8wq"] Jul 06 01:21:43.649316 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:43.649271 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 01:21:43.658021 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:21:43.657971 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5cf93c2_af80_4384_add5_49cdd601b40c.slice/crio-c56cd43b0958ada45eb772b9645abeb0f6d95242b801fb1413e379f409b4ff21 WatchSource:0}: Error finding container c56cd43b0958ada45eb772b9645abeb0f6d95242b801fb1413e379f409b4ff21: Status 404 returned error can't find the container with id c56cd43b0958ada45eb772b9645abeb0f6d95242b801fb1413e379f409b4ff21 Jul 06 01:21:43.701581 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:43.701434 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-744494649-m22gf" event={"ID":"b35757f2-50a7-414b-84b0-b560dec30ae1","Type":"ContainerStarted","Data":"825781334e828836c913a67a664d2f506a8732708fb0c2d78a38411467d10793"} Jul 06 01:21:43.703289 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:43.703218 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-drnhm" event={"ID":"fbe62245-715f-4565-bfa8-af7c3ab5d36a","Type":"ContainerStarted","Data":"dd0a424a1f5e3d2df6cdb9e6e096b492b92e571af59b006a76c3f002077b7708"} Jul 06 01:21:43.707436 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:43.707260 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-88b699c9-dc7x2" event={"ID":"e8e13654-1d6d-436b-a582-40074d4bdde2","Type":"ContainerStarted","Data":"fc082a7c13a15eccb963a6249926931fead3000ecaa23faeafbd950803076c07"} Jul 06 01:21:43.709180 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:43.709067 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-857dc8f57d-rtklb" event={"ID":"2b5d11a6-bb09-4c32-9e61-db7732341d0c","Type":"ContainerStarted","Data":"092f6cce4fe05ed4c1c451ba01898ceabfe5bcf89fd1515c77123ce5f1c93a2a"} Jul 06 01:21:43.710968 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:43.710865 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5cf93c2-af80-4384-add5-49cdd601b40c","Type":"ContainerStarted","Data":"c56cd43b0958ada45eb772b9645abeb0f6d95242b801fb1413e379f409b4ff21"} Jul 06 01:21:43.713182 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:43.713128 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4ss2v" event={"ID":"8bcfcc54-ba2d-4049-9077-6cc8ce713838","Type":"ContainerStarted","Data":"0ecea067ccb8fd33850b18e425a50b44ff1fddf589f49339362e99698f7702ce"} Jul 06 01:21:43.721050 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:43.720877 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ksnx7" event={"ID":"a582740b-395e-485b-b214-81c6f58a7747","Type":"ContainerStarted","Data":"b5a4c061a770d7a89ab59d85196b411630024f9bb6f0e71a614f715715b449c2"} Jul 06 01:21:43.721050 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:43.720954 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ksnx7" event={"ID":"a582740b-395e-485b-b214-81c6f58a7747","Type":"ContainerStarted","Data":"805548ce79e75388827f51398b4c5ec8e9930728434b566ca1b7957429d532c5"} Jul 06 01:21:43.724870 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:43.724702 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-448pq" event={"ID":"50574349-d208-4de6-aba3-06e535105c6a","Type":"ContainerStarted","Data":"528e9aa83dc8c80f6614f9997b750fab7fdfbbd231c8aaf24c25a57a14cfea33"} Jul 06 01:21:43.728271 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:43.728188 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pm8wq" event={"ID":"4c0a1cea-8dca-4e9d-afdb-81f874754505","Type":"ContainerStarted","Data":"3386333e153bb3000179f6208126beb2cc21fb5783d03c89235a28cb7e636645"} Jul 06 01:21:43.731105 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:43.730581 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-x9sxx" event={"ID":"6122d230-b346-4d52-9e50-762d328a2651","Type":"ContainerStarted","Data":"a7339400fd22476cc7acbc5e6259009ed3e1431e6d11a2b5aa8f8c6ceb0e449a"} Jul 06 01:21:43.731105 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:43.731063 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-x9sxx" Jul 06 01:21:43.732728 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:43.732369 2565 patch_prober.go:28] interesting pod/downloads-685bfb5f64-x9sxx container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.134.0.11:8080/\": dial tcp 10.134.0.11:8080: connect: connection refused" start-of-body= Jul 06 01:21:43.732728 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:43.732462 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-685bfb5f64-x9sxx" podUID="6122d230-b346-4d52-9e50-762d328a2651" containerName="download-server" probeResult="failure" output="Get \"http://10.134.0.11:8080/\": dial tcp 10.134.0.11:8080: connect: connection refused" Jul 06 01:21:43.733579 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:43.733547 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cjx7b" event={"ID":"ca215253-2060-46f3-94c8-2b48a3fc81a2","Type":"ContainerStarted","Data":"033ad6bcf46b76aec50064f97b4d6eb8663dd881f4cdfbccc2c839e1deb1fe45"} Jul 06 01:21:43.736627 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:43.736592 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9vxks" event={"ID":"117cb15c-e26a-4b08-9dc2-57f776b22adc","Type":"ContainerStarted","Data":"808a91644fbeaa356e5a2f0788b1537147b47a4a45d2fb9534cfbd92b72e4979"} Jul 06 01:21:43.739295 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:43.739185 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7bf690b-525a-4310-aeb2-206ab91e6346","Type":"ContainerStarted","Data":"4ee256ef3cbe024c7281b1e86aae8ca7204b667cba7095d2ea2133dca0c29ba1"} Jul 06 01:21:43.742298 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:43.742229 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-744494649-m22gf" podStartSLOduration=1.5657560670000001 podStartE2EDuration="13.742210502s" totalImagesPullingTime="12.176454435s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:21:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:21:43.719912744 +0000 UTC m=+75.861471530" watchObservedRunningTime="2026-07-06 01:21:43.742210502 +0000 UTC m=+75.883769285" Jul 06 01:21:43.742876 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:43.742836 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-448pq" podStartSLOduration=3.438466848 podStartE2EDuration="14.742809938s" totalImagesPullingTime="11.30434309s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:21:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:21:43.741282255 +0000 UTC m=+75.882841035" watchObservedRunningTime="2026-07-06 01:21:43.742809938 +0000 UTC m=+75.884368723" Jul 06 01:21:43.763884 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:43.763135 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-x9sxx" podStartSLOduration=1.474553484 podStartE2EDuration="22.763111155s" totalImagesPullingTime="21.288557671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:21:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:21:43.761407206 +0000 UTC m=+75.902965991" watchObservedRunningTime="2026-07-06 01:21:43.763111155 +0000 UTC m=+75.904669941" Jul 06 01:21:44.773411 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:44.772817 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ksnx7" event={"ID":"a582740b-395e-485b-b214-81c6f58a7747","Type":"ContainerStarted","Data":"2b35e417916c272ed646ea58b10bc9417161873b3c02f6671fabf22c1ed34fad"} Jul 06 01:21:44.784655 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:44.784493 2565 generic.go:356] "Generic (PLEG): container finished" podID="ca215253-2060-46f3-94c8-2b48a3fc81a2" containerID="033ad6bcf46b76aec50064f97b4d6eb8663dd881f4cdfbccc2c839e1deb1fe45" exitCode=0 Jul 06 01:21:44.784655 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:44.784623 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cjx7b" event={"ID":"ca215253-2060-46f3-94c8-2b48a3fc81a2","Type":"ContainerDied","Data":"033ad6bcf46b76aec50064f97b4d6eb8663dd881f4cdfbccc2c839e1deb1fe45"} Jul 06 01:21:44.784798 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:44.784682 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cjx7b" event={"ID":"ca215253-2060-46f3-94c8-2b48a3fc81a2","Type":"ContainerStarted","Data":"86861c067a7f173291c2801e5660aacd20edfb2434eeb38946a9b5bf3780a13e"} Jul 06 01:21:44.784798 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:44.784708 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cjx7b" event={"ID":"ca215253-2060-46f3-94c8-2b48a3fc81a2","Type":"ContainerStarted","Data":"4a392a44752ab9f2b0848fba0855829813e90745a3893a9c6d4b089006d18d92"} Jul 06 01:21:44.808535 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:44.808466 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-x9sxx" Jul 06 01:21:44.817415 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:44.815157 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ksnx7" podStartSLOduration=4.415405792 podStartE2EDuration="15.815133911s" totalImagesPullingTime="11.399728119s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:21:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:21:44.794978388 +0000 UTC m=+76.936537175" watchObservedRunningTime="2026-07-06 01:21:44.815133911 +0000 UTC m=+76.956692700" Jul 06 01:21:44.837450 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:44.837218 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-cjx7b" podStartSLOduration=3.179143067 podStartE2EDuration="15.837188647s" totalImagesPullingTime="12.482840829s" totalInitContainerRuntime="175.204751ms" isStatefulPod=true podCreationTimestamp="2026-07-06 01:21:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:21:44.817938259 +0000 UTC m=+76.959497064" watchObservedRunningTime="2026-07-06 01:21:44.837188647 +0000 UTC m=+76.978747441" Jul 06 01:21:45.827054 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:45.825025 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9vxks" event={"ID":"117cb15c-e26a-4b08-9dc2-57f776b22adc","Type":"ContainerStarted","Data":"9fcfb571d2e09b8639b12ea0b2341f114b08e39dc4c33f8819ab05522ae513f2"} Jul 06 01:21:50.856317 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:50.856281 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-744494649-m22gf"] Jul 06 01:21:50.866773 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:50.866738 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-744494649-m22gf" Jul 06 01:21:55.882865 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:55.882833 2565 generic.go:356] "Generic (PLEG): container finished" podID="d7bf690b-525a-4310-aeb2-206ab91e6346" containerID="df0c5cfdff96164c5c36911483f222439d9a4871dfaed6aba6e25483e8736cb3" exitCode=0 Jul 06 01:21:55.883221 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:55.882952 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7bf690b-525a-4310-aeb2-206ab91e6346","Type":"ContainerDied","Data":"df0c5cfdff96164c5c36911483f222439d9a4871dfaed6aba6e25483e8736cb3"} Jul 06 01:21:55.886017 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:55.885986 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-drnhm" event={"ID":"fbe62245-715f-4565-bfa8-af7c3ab5d36a","Type":"ContainerStarted","Data":"5bfee39676413a14817edd6fd36494f07e0231a9a94f4193f29b30e7ba916595"} Jul 06 01:21:55.888118 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:55.888087 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-88b699c9-dc7x2" event={"ID":"e8e13654-1d6d-436b-a582-40074d4bdde2","Type":"ContainerStarted","Data":"fa441bb2ab8c30498f7fad53a08c2825167dbae453e7b9a92c3eb4c781811e13"} Jul 06 01:21:55.890289 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:55.889810 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-857dc8f57d-rtklb" event={"ID":"2b5d11a6-bb09-4c32-9e61-db7732341d0c","Type":"ContainerStarted","Data":"47735abd2b22fa6e0fa646fbf8ce0380cd30b63e8c5d2174b01f65e684ec549e"} Jul 06 01:21:55.892113 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:55.891625 2565 generic.go:356] "Generic (PLEG): container finished" podID="f5cf93c2-af80-4384-add5-49cdd601b40c" containerID="f3aaad56290da5b1fb8a6fe3ade74d8c27450ed46845facd8b053c768be8de89" exitCode=0 Jul 06 01:21:55.892113 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:55.891715 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5cf93c2-af80-4384-add5-49cdd601b40c","Type":"ContainerDied","Data":"f3aaad56290da5b1fb8a6fe3ade74d8c27450ed46845facd8b053c768be8de89"} Jul 06 01:21:55.893835 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:55.893782 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4ss2v" event={"ID":"8bcfcc54-ba2d-4049-9077-6cc8ce713838","Type":"ContainerStarted","Data":"9adb480b91dabbffa732a142071192be88cebf57b77e3bb11597a29eae02744e"} Jul 06 01:21:55.894211 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:55.894191 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-4ss2v" Jul 06 01:21:55.897601 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:55.897579 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9vxks" event={"ID":"117cb15c-e26a-4b08-9dc2-57f776b22adc","Type":"ContainerStarted","Data":"c2e452a70fdc98c56263c6a3e2267b66d389a9e0554e4748445a861073f8b55e"} Jul 06 01:21:55.927717 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:55.927595 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-9vxks" podStartSLOduration=10.327314771 podStartE2EDuration="35.927573708s" totalImagesPullingTime="25.600258937s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:21:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:21:55.926695162 +0000 UTC m=+88.068253947" watchObservedRunningTime="2026-07-06 01:21:55.927573708 +0000 UTC m=+88.069132494" Jul 06 01:21:55.966332 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:55.966274 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-4ss2v" podStartSLOduration=76.367189901 podStartE2EDuration="1m27.966255944s" totalImagesPullingTime="11.599066043s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:20:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:21:55.964764974 +0000 UTC m=+88.106323760" watchObservedRunningTime="2026-07-06 01:21:55.966255944 +0000 UTC m=+88.107814707" Jul 06 01:21:55.983366 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:55.983303 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-88b699c9-dc7x2" podStartSLOduration=10.456694034 podStartE2EDuration="21.983280001s" totalImagesPullingTime="11.526585967s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:21:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:21:55.982009478 +0000 UTC m=+88.123568279" watchObservedRunningTime="2026-07-06 01:21:55.983280001 +0000 UTC m=+88.124838791" Jul 06 01:21:56.905428 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:56.905367 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-drnhm" event={"ID":"fbe62245-715f-4565-bfa8-af7c3ab5d36a","Type":"ContainerStarted","Data":"9f7899114ae785035007e049a93b4a1b45d1dc770f7653eb201a89ff3f5e1322"} Jul 06 01:21:56.909254 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:56.909199 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-857dc8f57d-rtklb" event={"ID":"2b5d11a6-bb09-4c32-9e61-db7732341d0c","Type":"ContainerStarted","Data":"bfd8962683fa6f0226dead158ce6ec94f1df8bf162e644ffdb6ea0d272ba119a"} Jul 06 01:21:56.909521 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:56.909306 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-857dc8f57d-rtklb" event={"ID":"2b5d11a6-bb09-4c32-9e61-db7732341d0c","Type":"ContainerStarted","Data":"28b9525c15a0853872346462708d23cb1abe6d8288a82348e67df1f9ff7be9c6"} Jul 06 01:21:56.911662 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:56.911625 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pm8wq" event={"ID":"4c0a1cea-8dca-4e9d-afdb-81f874754505","Type":"ContainerStarted","Data":"1e37e30eabed612d5b3bddf8701eae6537083741461a9a3acf6e72da102b872a"} Jul 06 01:21:56.921902 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:56.921779 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-drnhm" podStartSLOduration=77.708362477 podStartE2EDuration="1m28.921757978s" totalImagesPullingTime="11.213395501s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:20:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:21:56.921741796 +0000 UTC m=+89.063300581" watchObservedRunningTime="2026-07-06 01:21:56.921757978 +0000 UTC m=+89.063316763" Jul 06 01:21:56.937154 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:56.937096 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-pm8wq" podStartSLOduration=76.889908547 podStartE2EDuration="1m28.937077446s" totalImagesPullingTime="12.047168899s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:20:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:21:56.935141925 +0000 UTC m=+89.076700716" watchObservedRunningTime="2026-07-06 01:21:56.937077446 +0000 UTC m=+89.078636209" Jul 06 01:21:58.923296 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:58.923258 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7bf690b-525a-4310-aeb2-206ab91e6346","Type":"ContainerStarted","Data":"11b4d907f54a644f6b4be39d72e68ef2bc73ed54d7e86b5d6ebef95f0cab4c02"} Jul 06 01:21:58.926042 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:58.926005 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-857dc8f57d-rtklb" event={"ID":"2b5d11a6-bb09-4c32-9e61-db7732341d0c","Type":"ContainerStarted","Data":"855f27ea3e479f09d29e53bac3a9b13657f9437099bef2309c3df6d72bba9c60"} Jul 06 01:21:59.934559 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:59.934512 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-857dc8f57d-rtklb" event={"ID":"2b5d11a6-bb09-4c32-9e61-db7732341d0c","Type":"ContainerStarted","Data":"a18f12684f9fe526d69af098577f4104c3d4ec7f10a50b64031606aee147b54a"} Jul 06 01:21:59.935014 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:59.934667 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-857dc8f57d-rtklb" Jul 06 01:21:59.935014 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:59.934713 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-857dc8f57d-rtklb" event={"ID":"2b5d11a6-bb09-4c32-9e61-db7732341d0c","Type":"ContainerStarted","Data":"5623c9c925e0a5b0446c892584ef72f13b4e49f4f58634af93a874c7adb611c6"} Jul 06 01:21:59.937439 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:59.937412 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7bf690b-525a-4310-aeb2-206ab91e6346","Type":"ContainerStarted","Data":"b55986b74386a55b6e2359ae8c958f54643ad061aa3ba1a210e37f1c477eb5b0"} Jul 06 01:21:59.937570 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:59.937470 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7bf690b-525a-4310-aeb2-206ab91e6346","Type":"ContainerStarted","Data":"632e123e80c73cb4a646b924d0ffca9cd732b062c7a74a93cdf273c904b360b3"} Jul 06 01:21:59.954536 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:21:59.954457 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-857dc8f57d-rtklb" podStartSLOduration=15.120860466 podStartE2EDuration="28.954437669s" totalImagesPullingTime="13.833577203s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:21:31 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:21:59.953789953 +0000 UTC m=+92.095348786" watchObservedRunningTime="2026-07-06 01:21:59.954437669 +0000 UTC m=+92.095996451" Jul 06 01:22:00.946179 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:00.946074 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7bf690b-525a-4310-aeb2-206ab91e6346","Type":"ContainerStarted","Data":"9bd3ff1f390c148533a50efed04c42f4a03b3e82c17dce643ca11361c84f5ef2"} Jul 06 01:22:00.946179 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:00.946160 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7bf690b-525a-4310-aeb2-206ab91e6346","Type":"ContainerStarted","Data":"656869cd1adb0d77c4bf1c056c15aaa8fcd8c848924801c130543c1d2d1e9746"} Jul 06 01:22:00.946708 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:00.946203 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7bf690b-525a-4310-aeb2-206ab91e6346","Type":"ContainerStarted","Data":"46a7cd47439380223d0e448b76e87c63f9e17bc8dbfa32cbf19474b1237432c3"} Jul 06 01:22:00.949755 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:00.949716 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5cf93c2-af80-4384-add5-49cdd601b40c","Type":"ContainerStarted","Data":"7481585220cf802cc416c54b694536f1550363ba218d891257702cb267073a14"} Jul 06 01:22:00.949937 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:00.949798 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5cf93c2-af80-4384-add5-49cdd601b40c","Type":"ContainerStarted","Data":"5d6d090979962c7cbdfea6b12f745ae0d4bc743eb7222b0ea4faf3fced052c80"} Jul 06 01:22:00.949937 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:00.949827 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5cf93c2-af80-4384-add5-49cdd601b40c","Type":"ContainerStarted","Data":"64f901021a44f7d5134297d7e2968ad0cc63ff814ae2c097948bd81392bbd868"} Jul 06 01:22:00.949937 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:00.949852 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5cf93c2-af80-4384-add5-49cdd601b40c","Type":"ContainerStarted","Data":"608cde94a0548d7f5e181750409054de510aeb53254655cae7e7173048956e18"} Jul 06 01:22:00.949937 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:00.949874 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5cf93c2-af80-4384-add5-49cdd601b40c","Type":"ContainerStarted","Data":"e69034755d0292d6c7808ef2126f934888ec8cf408afe632dd385d711a8042a4"} Jul 06 01:22:00.949937 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:00.949909 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5cf93c2-af80-4384-add5-49cdd601b40c","Type":"ContainerStarted","Data":"8e35dea0111da30f93c5572e670a19b04a7796de3fe1a390659ebd87a47b4c44"} Jul 06 01:22:00.955694 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:00.955665 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-857dc8f57d-rtklb" Jul 06 01:22:00.970697 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:00.970631 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=17.140803926 podStartE2EDuration="30.970609272s" totalImagesPullingTime="13.791467412s" totalInitContainerRuntime="38.337934ms" isStatefulPod=false podCreationTimestamp="2026-07-06 01:21:30 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:22:00.96887879 +0000 UTC m=+93.110437585" watchObservedRunningTime="2026-07-06 01:22:00.970609272 +0000 UTC m=+93.112168058" Jul 06 01:22:00.995224 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:00.994691 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=9.528653283 podStartE2EDuration="24.994669558s" totalImagesPullingTime="15.430975333s" totalInitContainerRuntime="35.040942ms" isStatefulPod=false podCreationTimestamp="2026-07-06 01:21:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:22:00.993150172 +0000 UTC m=+93.134708985" watchObservedRunningTime="2026-07-06 01:22:00.994669558 +0000 UTC m=+93.136228343" Jul 06 01:22:01.391032 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:01.390984 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:14.411895 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:14.411857 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-88b699c9-dc7x2" Jul 06 01:22:14.412476 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:14.411928 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-88b699c9-dc7x2" Jul 06 01:22:15.885805 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:15.885730 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-744494649-m22gf" podUID="b35757f2-50a7-414b-84b0-b560dec30ae1" containerName="console" containerID="cri-o://825781334e828836c913a67a664d2f506a8732708fb0c2d78a38411467d10793" gracePeriod=15 Jul 06 01:22:16.017730 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:16.017698 2565 generic.go:356] "Generic (PLEG): container finished" podID="b35757f2-50a7-414b-84b0-b560dec30ae1" containerID="825781334e828836c913a67a664d2f506a8732708fb0c2d78a38411467d10793" exitCode=0 Jul 06 01:22:16.017838 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:16.017781 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-744494649-m22gf" event={"ID":"b35757f2-50a7-414b-84b0-b560dec30ae1","Type":"ContainerDied","Data":"825781334e828836c913a67a664d2f506a8732708fb0c2d78a38411467d10793"} Jul 06 01:22:16.175366 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:16.175330 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-744494649-m22gf" Jul 06 01:22:16.286450 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:16.286406 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b35757f2-50a7-414b-84b0-b560dec30ae1-console-serving-cert\") pod \"b35757f2-50a7-414b-84b0-b560dec30ae1\" (UID: \"b35757f2-50a7-414b-84b0-b560dec30ae1\") " Jul 06 01:22:16.286634 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:16.286464 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b35757f2-50a7-414b-84b0-b560dec30ae1-console-oauth-config\") pod \"b35757f2-50a7-414b-84b0-b560dec30ae1\" (UID: \"b35757f2-50a7-414b-84b0-b560dec30ae1\") " Jul 06 01:22:16.286634 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:16.286500 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b35757f2-50a7-414b-84b0-b560dec30ae1-oauth-serving-cert\") pod \"b35757f2-50a7-414b-84b0-b560dec30ae1\" (UID: \"b35757f2-50a7-414b-84b0-b560dec30ae1\") " Jul 06 01:22:16.286634 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:16.286543 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ljz6s\" (UniqueName: \"kubernetes.io/projected/b35757f2-50a7-414b-84b0-b560dec30ae1-kube-api-access-ljz6s\") pod \"b35757f2-50a7-414b-84b0-b560dec30ae1\" (UID: \"b35757f2-50a7-414b-84b0-b560dec30ae1\") " Jul 06 01:22:16.286634 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:16.286591 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b35757f2-50a7-414b-84b0-b560dec30ae1-console-config\") pod \"b35757f2-50a7-414b-84b0-b560dec30ae1\" (UID: \"b35757f2-50a7-414b-84b0-b560dec30ae1\") " Jul 06 01:22:16.286852 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:16.286667 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b35757f2-50a7-414b-84b0-b560dec30ae1-service-ca\") pod \"b35757f2-50a7-414b-84b0-b560dec30ae1\" (UID: \"b35757f2-50a7-414b-84b0-b560dec30ae1\") " Jul 06 01:22:16.286930 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:16.286899 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b35757f2-50a7-414b-84b0-b560dec30ae1-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b35757f2-50a7-414b-84b0-b560dec30ae1" (UID: "b35757f2-50a7-414b-84b0-b560dec30ae1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 01:22:16.287066 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:16.287039 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b35757f2-50a7-414b-84b0-b560dec30ae1-console-config" (OuterVolumeSpecName: "console-config") pod "b35757f2-50a7-414b-84b0-b560dec30ae1" (UID: "b35757f2-50a7-414b-84b0-b560dec30ae1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 01:22:16.287116 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:16.287096 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b35757f2-50a7-414b-84b0-b560dec30ae1-service-ca" (OuterVolumeSpecName: "service-ca") pod "b35757f2-50a7-414b-84b0-b560dec30ae1" (UID: "b35757f2-50a7-414b-84b0-b560dec30ae1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 01:22:16.289069 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:16.289021 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b35757f2-50a7-414b-84b0-b560dec30ae1-kube-api-access-ljz6s" (OuterVolumeSpecName: "kube-api-access-ljz6s") pod "b35757f2-50a7-414b-84b0-b560dec30ae1" (UID: "b35757f2-50a7-414b-84b0-b560dec30ae1"). InnerVolumeSpecName "kube-api-access-ljz6s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:22:16.289178 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:16.289076 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b35757f2-50a7-414b-84b0-b560dec30ae1-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b35757f2-50a7-414b-84b0-b560dec30ae1" (UID: "b35757f2-50a7-414b-84b0-b560dec30ae1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 01:22:16.289178 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:16.289132 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b35757f2-50a7-414b-84b0-b560dec30ae1-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b35757f2-50a7-414b-84b0-b560dec30ae1" (UID: "b35757f2-50a7-414b-84b0-b560dec30ae1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 01:22:16.388365 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:16.388321 2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b35757f2-50a7-414b-84b0-b560dec30ae1-service-ca\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:22:16.388365 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:16.388358 2565 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b35757f2-50a7-414b-84b0-b560dec30ae1-console-serving-cert\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:22:16.388644 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:16.388369 2565 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b35757f2-50a7-414b-84b0-b560dec30ae1-console-oauth-config\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:22:16.388644 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:16.388433 2565 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b35757f2-50a7-414b-84b0-b560dec30ae1-oauth-serving-cert\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:22:16.388644 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:16.388443 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ljz6s\" (UniqueName: \"kubernetes.io/projected/b35757f2-50a7-414b-84b0-b560dec30ae1-kube-api-access-ljz6s\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:22:16.388644 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:16.388452 2565 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b35757f2-50a7-414b-84b0-b560dec30ae1-console-config\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:22:17.023993 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:17.023950 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-744494649-m22gf" event={"ID":"b35757f2-50a7-414b-84b0-b560dec30ae1","Type":"ContainerDied","Data":"680ab4262beaebb975ab31dfc00704a009adea64b89d3d0f409512985a95c336"} Jul 06 01:22:17.023993 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:17.023987 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-744494649-m22gf" Jul 06 01:22:17.024602 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:17.024008 2565 scope.go:117] "RemoveContainer" containerID="825781334e828836c913a67a664d2f506a8732708fb0c2d78a38411467d10793" Jul 06 01:22:17.044040 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:17.044003 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-744494649-m22gf"] Jul 06 01:22:17.047544 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:17.047516 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-744494649-m22gf"] Jul 06 01:22:18.324253 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:18.324221 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b35757f2-50a7-414b-84b0-b560dec30ae1" path="/var/lib/kubelet/pods/b35757f2-50a7-414b-84b0-b560dec30ae1/volumes" Jul 06 01:22:27.919471 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:27.919438 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-4ss2v" Jul 06 01:22:34.416951 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:34.416920 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-88b699c9-dc7x2" Jul 06 01:22:34.420899 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:34.420867 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-88b699c9-dc7x2" Jul 06 01:22:36.390906 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:36.390863 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:36.458142 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:36.458107 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:37.124122 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:37.124097 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:50.137149 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:50.137114 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 01:22:50.137674 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:50.137469 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d7bf690b-525a-4310-aeb2-206ab91e6346" containerName="alertmanager" containerID="cri-o://11b4d907f54a644f6b4be39d72e68ef2bc73ed54d7e86b5d6ebef95f0cab4c02" gracePeriod=120 Jul 06 01:22:50.137674 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:50.137525 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d7bf690b-525a-4310-aeb2-206ab91e6346" containerName="kube-rbac-proxy-metric" containerID="cri-o://656869cd1adb0d77c4bf1c056c15aaa8fcd8c848924801c130543c1d2d1e9746" gracePeriod=120 Jul 06 01:22:50.137674 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:50.137568 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d7bf690b-525a-4310-aeb2-206ab91e6346" containerName="kube-rbac-proxy-web" containerID="cri-o://b55986b74386a55b6e2359ae8c958f54643ad061aa3ba1a210e37f1c477eb5b0" gracePeriod=120 Jul 06 01:22:50.137674 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:50.137616 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d7bf690b-525a-4310-aeb2-206ab91e6346" containerName="config-reloader" containerID="cri-o://632e123e80c73cb4a646b924d0ffca9cd732b062c7a74a93cdf273c904b360b3" gracePeriod=120 Jul 06 01:22:50.137674 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:50.137634 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d7bf690b-525a-4310-aeb2-206ab91e6346" containerName="prom-label-proxy" containerID="cri-o://9bd3ff1f390c148533a50efed04c42f4a03b3e82c17dce643ca11361c84f5ef2" gracePeriod=120 Jul 06 01:22:50.137674 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:50.137561 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d7bf690b-525a-4310-aeb2-206ab91e6346" containerName="kube-rbac-proxy" containerID="cri-o://46a7cd47439380223d0e448b76e87c63f9e17bc8dbfa32cbf19474b1237432c3" gracePeriod=120 Jul 06 01:22:51.170520 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.170472 2565 generic.go:356] "Generic (PLEG): container finished" podID="d7bf690b-525a-4310-aeb2-206ab91e6346" containerID="9bd3ff1f390c148533a50efed04c42f4a03b3e82c17dce643ca11361c84f5ef2" exitCode=0 Jul 06 01:22:51.170520 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.170507 2565 generic.go:356] "Generic (PLEG): container finished" podID="d7bf690b-525a-4310-aeb2-206ab91e6346" containerID="656869cd1adb0d77c4bf1c056c15aaa8fcd8c848924801c130543c1d2d1e9746" exitCode=0 Jul 06 01:22:51.170520 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.170517 2565 generic.go:356] "Generic (PLEG): container finished" podID="d7bf690b-525a-4310-aeb2-206ab91e6346" containerID="46a7cd47439380223d0e448b76e87c63f9e17bc8dbfa32cbf19474b1237432c3" exitCode=0 Jul 06 01:22:51.170520 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.170527 2565 generic.go:356] "Generic (PLEG): container finished" podID="d7bf690b-525a-4310-aeb2-206ab91e6346" containerID="632e123e80c73cb4a646b924d0ffca9cd732b062c7a74a93cdf273c904b360b3" exitCode=0 Jul 06 01:22:51.171005 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.170538 2565 generic.go:356] "Generic (PLEG): container finished" podID="d7bf690b-525a-4310-aeb2-206ab91e6346" containerID="11b4d907f54a644f6b4be39d72e68ef2bc73ed54d7e86b5d6ebef95f0cab4c02" exitCode=0 Jul 06 01:22:51.171005 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.170541 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7bf690b-525a-4310-aeb2-206ab91e6346","Type":"ContainerDied","Data":"9bd3ff1f390c148533a50efed04c42f4a03b3e82c17dce643ca11361c84f5ef2"} Jul 06 01:22:51.171005 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.170635 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7bf690b-525a-4310-aeb2-206ab91e6346","Type":"ContainerDied","Data":"656869cd1adb0d77c4bf1c056c15aaa8fcd8c848924801c130543c1d2d1e9746"} Jul 06 01:22:51.171005 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.170664 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7bf690b-525a-4310-aeb2-206ab91e6346","Type":"ContainerDied","Data":"46a7cd47439380223d0e448b76e87c63f9e17bc8dbfa32cbf19474b1237432c3"} Jul 06 01:22:51.171005 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.170688 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7bf690b-525a-4310-aeb2-206ab91e6346","Type":"ContainerDied","Data":"632e123e80c73cb4a646b924d0ffca9cd732b062c7a74a93cdf273c904b360b3"} Jul 06 01:22:51.171005 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.170713 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7bf690b-525a-4310-aeb2-206ab91e6346","Type":"ContainerDied","Data":"11b4d907f54a644f6b4be39d72e68ef2bc73ed54d7e86b5d6ebef95f0cab4c02"} Jul 06 01:22:51.390988 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.390963 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:51.394325 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.394303 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-secret-alertmanager-kube-rbac-proxy\") pod \"d7bf690b-525a-4310-aeb2-206ab91e6346\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " Jul 06 01:22:51.394459 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.394333 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-cluster-tls-config\") pod \"d7bf690b-525a-4310-aeb2-206ab91e6346\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " Jul 06 01:22:51.394459 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.394356 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-secret-alertmanager-kube-rbac-proxy-web\") pod \"d7bf690b-525a-4310-aeb2-206ab91e6346\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " Jul 06 01:22:51.394459 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.394400 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d7bf690b-525a-4310-aeb2-206ab91e6346-alertmanager-main-db\") pod \"d7bf690b-525a-4310-aeb2-206ab91e6346\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " Jul 06 01:22:51.394459 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.394433 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-secret-alertmanager-kube-rbac-proxy-metric\") pod \"d7bf690b-525a-4310-aeb2-206ab91e6346\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " Jul 06 01:22:51.394459 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.394456 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d7bf690b-525a-4310-aeb2-206ab91e6346-tls-assets\") pod \"d7bf690b-525a-4310-aeb2-206ab91e6346\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " Jul 06 01:22:51.394711 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.394482 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7bf690b-525a-4310-aeb2-206ab91e6346-alertmanager-trusted-ca-bundle\") pod \"d7bf690b-525a-4310-aeb2-206ab91e6346\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " Jul 06 01:22:51.394711 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.394516 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-config-volume\") pod \"d7bf690b-525a-4310-aeb2-206ab91e6346\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " Jul 06 01:22:51.394711 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.394577 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-web-config\") pod \"d7bf690b-525a-4310-aeb2-206ab91e6346\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " Jul 06 01:22:51.394711 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.394606 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d7bf690b-525a-4310-aeb2-206ab91e6346-config-out\") pod \"d7bf690b-525a-4310-aeb2-206ab91e6346\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " Jul 06 01:22:51.394711 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.394641 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7bf690b-525a-4310-aeb2-206ab91e6346-metrics-client-ca\") pod \"d7bf690b-525a-4310-aeb2-206ab91e6346\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " Jul 06 01:22:51.394711 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.394667 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8wc4b\" (UniqueName: \"kubernetes.io/projected/d7bf690b-525a-4310-aeb2-206ab91e6346-kube-api-access-8wc4b\") pod \"d7bf690b-525a-4310-aeb2-206ab91e6346\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " Jul 06 01:22:51.394711 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.394690 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-secret-alertmanager-main-tls\") pod \"d7bf690b-525a-4310-aeb2-206ab91e6346\" (UID: \"d7bf690b-525a-4310-aeb2-206ab91e6346\") " Jul 06 01:22:51.395075 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.395046 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7bf690b-525a-4310-aeb2-206ab91e6346-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "d7bf690b-525a-4310-aeb2-206ab91e6346" (UID: "d7bf690b-525a-4310-aeb2-206ab91e6346"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 01:22:51.395665 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.395596 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7bf690b-525a-4310-aeb2-206ab91e6346-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "d7bf690b-525a-4310-aeb2-206ab91e6346" (UID: "d7bf690b-525a-4310-aeb2-206ab91e6346"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:22:51.395665 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.395654 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7bf690b-525a-4310-aeb2-206ab91e6346-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "d7bf690b-525a-4310-aeb2-206ab91e6346" (UID: "d7bf690b-525a-4310-aeb2-206ab91e6346"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 01:22:51.398341 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.398292 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "d7bf690b-525a-4310-aeb2-206ab91e6346" (UID: "d7bf690b-525a-4310-aeb2-206ab91e6346"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 01:22:51.400115 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.400053 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "d7bf690b-525a-4310-aeb2-206ab91e6346" (UID: "d7bf690b-525a-4310-aeb2-206ab91e6346"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 01:22:51.400115 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.400053 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7bf690b-525a-4310-aeb2-206ab91e6346-config-out" (OuterVolumeSpecName: "config-out") pod "d7bf690b-525a-4310-aeb2-206ab91e6346" (UID: "d7bf690b-525a-4310-aeb2-206ab91e6346"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:22:51.400479 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.400452 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "d7bf690b-525a-4310-aeb2-206ab91e6346" (UID: "d7bf690b-525a-4310-aeb2-206ab91e6346"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 01:22:51.400632 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.400603 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "d7bf690b-525a-4310-aeb2-206ab91e6346" (UID: "d7bf690b-525a-4310-aeb2-206ab91e6346"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 01:22:51.400632 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.400614 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7bf690b-525a-4310-aeb2-206ab91e6346-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "d7bf690b-525a-4310-aeb2-206ab91e6346" (UID: "d7bf690b-525a-4310-aeb2-206ab91e6346"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:22:51.400753 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.400709 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-config-volume" (OuterVolumeSpecName: "config-volume") pod "d7bf690b-525a-4310-aeb2-206ab91e6346" (UID: "d7bf690b-525a-4310-aeb2-206ab91e6346"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 01:22:51.400753 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.400724 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7bf690b-525a-4310-aeb2-206ab91e6346-kube-api-access-8wc4b" (OuterVolumeSpecName: "kube-api-access-8wc4b") pod "d7bf690b-525a-4310-aeb2-206ab91e6346" (UID: "d7bf690b-525a-4310-aeb2-206ab91e6346"). InnerVolumeSpecName "kube-api-access-8wc4b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:22:51.407641 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.407598 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "d7bf690b-525a-4310-aeb2-206ab91e6346" (UID: "d7bf690b-525a-4310-aeb2-206ab91e6346"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 01:22:51.414196 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.414157 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-web-config" (OuterVolumeSpecName: "web-config") pod "d7bf690b-525a-4310-aeb2-206ab91e6346" (UID: "d7bf690b-525a-4310-aeb2-206ab91e6346"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 01:22:51.495895 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.495849 2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:22:51.495895 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.495886 2565 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-cluster-tls-config\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:22:51.495895 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.495897 2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:22:51.495895 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.495906 2565 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d7bf690b-525a-4310-aeb2-206ab91e6346-alertmanager-main-db\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:22:51.496181 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.495919 2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:22:51.496181 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.495952 2565 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d7bf690b-525a-4310-aeb2-206ab91e6346-tls-assets\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:22:51.496181 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.495961 2565 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7bf690b-525a-4310-aeb2-206ab91e6346-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:22:51.496181 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.495969 2565 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-config-volume\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:22:51.496181 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.495978 2565 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-web-config\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:22:51.496181 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.495985 2565 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d7bf690b-525a-4310-aeb2-206ab91e6346-config-out\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:22:51.496181 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.495992 2565 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7bf690b-525a-4310-aeb2-206ab91e6346-metrics-client-ca\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:22:51.496181 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.495999 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wc4b\" (UniqueName: \"kubernetes.io/projected/d7bf690b-525a-4310-aeb2-206ab91e6346-kube-api-access-8wc4b\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:22:51.496181 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:51.496009 2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d7bf690b-525a-4310-aeb2-206ab91e6346-secret-alertmanager-main-tls\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:22:52.177887 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.177842 2565 generic.go:356] "Generic (PLEG): container finished" podID="d7bf690b-525a-4310-aeb2-206ab91e6346" containerID="b55986b74386a55b6e2359ae8c958f54643ad061aa3ba1a210e37f1c477eb5b0" exitCode=0 Jul 06 01:22:52.178301 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.177888 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7bf690b-525a-4310-aeb2-206ab91e6346","Type":"ContainerDied","Data":"b55986b74386a55b6e2359ae8c958f54643ad061aa3ba1a210e37f1c477eb5b0"} Jul 06 01:22:52.178301 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.177942 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:52.178301 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.177967 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7bf690b-525a-4310-aeb2-206ab91e6346","Type":"ContainerDied","Data":"4ee256ef3cbe024c7281b1e86aae8ca7204b667cba7095d2ea2133dca0c29ba1"} Jul 06 01:22:52.178301 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.178004 2565 scope.go:117] "RemoveContainer" containerID="9bd3ff1f390c148533a50efed04c42f4a03b3e82c17dce643ca11361c84f5ef2" Jul 06 01:22:52.186488 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.186463 2565 scope.go:117] "RemoveContainer" containerID="656869cd1adb0d77c4bf1c056c15aaa8fcd8c848924801c130543c1d2d1e9746" Jul 06 01:22:52.194731 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.194707 2565 scope.go:117] "RemoveContainer" containerID="46a7cd47439380223d0e448b76e87c63f9e17bc8dbfa32cbf19474b1237432c3" Jul 06 01:22:52.202833 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.202800 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 01:22:52.204423 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.204397 2565 scope.go:117] "RemoveContainer" containerID="b55986b74386a55b6e2359ae8c958f54643ad061aa3ba1a210e37f1c477eb5b0" Jul 06 01:22:52.206107 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.206087 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 01:22:52.212662 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.212638 2565 scope.go:117] "RemoveContainer" containerID="632e123e80c73cb4a646b924d0ffca9cd732b062c7a74a93cdf273c904b360b3" Jul 06 01:22:52.220461 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.220441 2565 scope.go:117] "RemoveContainer" containerID="11b4d907f54a644f6b4be39d72e68ef2bc73ed54d7e86b5d6ebef95f0cab4c02" Jul 06 01:22:52.232321 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.232289 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 01:22:52.232650 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.232635 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7bf690b-525a-4310-aeb2-206ab91e6346" containerName="alertmanager" Jul 06 01:22:52.232650 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.232650 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7bf690b-525a-4310-aeb2-206ab91e6346" containerName="alertmanager" Jul 06 01:22:52.232758 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.232660 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7bf690b-525a-4310-aeb2-206ab91e6346" containerName="kube-rbac-proxy-metric" Jul 06 01:22:52.232758 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.232666 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7bf690b-525a-4310-aeb2-206ab91e6346" containerName="kube-rbac-proxy-metric" Jul 06 01:22:52.232758 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.232674 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7bf690b-525a-4310-aeb2-206ab91e6346" containerName="prom-label-proxy" Jul 06 01:22:52.232758 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.232681 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7bf690b-525a-4310-aeb2-206ab91e6346" containerName="prom-label-proxy" Jul 06 01:22:52.232758 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.232690 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b35757f2-50a7-414b-84b0-b560dec30ae1" containerName="console" Jul 06 01:22:52.232758 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.232695 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b35757f2-50a7-414b-84b0-b560dec30ae1" containerName="console" Jul 06 01:22:52.232758 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.232702 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7bf690b-525a-4310-aeb2-206ab91e6346" containerName="config-reloader" Jul 06 01:22:52.232758 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.232708 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7bf690b-525a-4310-aeb2-206ab91e6346" containerName="config-reloader" Jul 06 01:22:52.232758 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.232719 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7bf690b-525a-4310-aeb2-206ab91e6346" containerName="init-config-reloader" Jul 06 01:22:52.232758 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.232726 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7bf690b-525a-4310-aeb2-206ab91e6346" containerName="init-config-reloader" Jul 06 01:22:52.232758 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.232733 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7bf690b-525a-4310-aeb2-206ab91e6346" containerName="kube-rbac-proxy-web" Jul 06 01:22:52.232758 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.232740 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7bf690b-525a-4310-aeb2-206ab91e6346" containerName="kube-rbac-proxy-web" Jul 06 01:22:52.232758 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.232751 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7bf690b-525a-4310-aeb2-206ab91e6346" containerName="kube-rbac-proxy" Jul 06 01:22:52.232758 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.232756 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7bf690b-525a-4310-aeb2-206ab91e6346" containerName="kube-rbac-proxy" Jul 06 01:22:52.233141 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.232810 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7bf690b-525a-4310-aeb2-206ab91e6346" containerName="kube-rbac-proxy" Jul 06 01:22:52.233141 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.232816 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7bf690b-525a-4310-aeb2-206ab91e6346" containerName="kube-rbac-proxy-web" Jul 06 01:22:52.233141 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.232823 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7bf690b-525a-4310-aeb2-206ab91e6346" containerName="prom-label-proxy" Jul 06 01:22:52.233141 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.232828 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7bf690b-525a-4310-aeb2-206ab91e6346" containerName="alertmanager" Jul 06 01:22:52.233141 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.232833 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7bf690b-525a-4310-aeb2-206ab91e6346" containerName="config-reloader" Jul 06 01:22:52.233141 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.232840 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7bf690b-525a-4310-aeb2-206ab91e6346" containerName="kube-rbac-proxy-metric" Jul 06 01:22:52.233141 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.232845 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b35757f2-50a7-414b-84b0-b560dec30ae1" containerName="console" Jul 06 01:22:52.234584 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.234558 2565 scope.go:117] "RemoveContainer" containerID="df0c5cfdff96164c5c36911483f222439d9a4871dfaed6aba6e25483e8736cb3" Jul 06 01:22:52.236435 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.236413 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:52.238932 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.238907 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 06 01:22:52.239046 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.238907 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 06 01:22:52.239046 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.238979 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 06 01:22:52.239305 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.239288 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 06 01:22:52.239475 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.239293 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 06 01:22:52.239593 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.239523 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 06 01:22:52.239593 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.239542 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-4fq5l\"" Jul 06 01:22:52.239593 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.239577 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 06 01:22:52.240061 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.240043 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 06 01:22:52.244976 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.244952 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 06 01:22:52.245508 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.245489 2565 scope.go:117] "RemoveContainer" containerID="9bd3ff1f390c148533a50efed04c42f4a03b3e82c17dce643ca11361c84f5ef2" Jul 06 01:22:52.245850 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:22:52.245828 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bd3ff1f390c148533a50efed04c42f4a03b3e82c17dce643ca11361c84f5ef2\": container with ID starting with 9bd3ff1f390c148533a50efed04c42f4a03b3e82c17dce643ca11361c84f5ef2 not found: ID does not exist" containerID="9bd3ff1f390c148533a50efed04c42f4a03b3e82c17dce643ca11361c84f5ef2" Jul 06 01:22:52.245897 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.245859 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bd3ff1f390c148533a50efed04c42f4a03b3e82c17dce643ca11361c84f5ef2"} err="failed to get container status \"9bd3ff1f390c148533a50efed04c42f4a03b3e82c17dce643ca11361c84f5ef2\": rpc error: code = NotFound desc = could not find container \"9bd3ff1f390c148533a50efed04c42f4a03b3e82c17dce643ca11361c84f5ef2\": container with ID starting with 9bd3ff1f390c148533a50efed04c42f4a03b3e82c17dce643ca11361c84f5ef2 not found: ID does not exist" Jul 06 01:22:52.245897 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.245894 2565 scope.go:117] "RemoveContainer" containerID="656869cd1adb0d77c4bf1c056c15aaa8fcd8c848924801c130543c1d2d1e9746" Jul 06 01:22:52.246186 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:22:52.246166 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"656869cd1adb0d77c4bf1c056c15aaa8fcd8c848924801c130543c1d2d1e9746\": container with ID starting with 656869cd1adb0d77c4bf1c056c15aaa8fcd8c848924801c130543c1d2d1e9746 not found: ID does not exist" containerID="656869cd1adb0d77c4bf1c056c15aaa8fcd8c848924801c130543c1d2d1e9746" Jul 06 01:22:52.246256 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.246192 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"656869cd1adb0d77c4bf1c056c15aaa8fcd8c848924801c130543c1d2d1e9746"} err="failed to get container status \"656869cd1adb0d77c4bf1c056c15aaa8fcd8c848924801c130543c1d2d1e9746\": rpc error: code = NotFound desc = could not find container \"656869cd1adb0d77c4bf1c056c15aaa8fcd8c848924801c130543c1d2d1e9746\": container with ID starting with 656869cd1adb0d77c4bf1c056c15aaa8fcd8c848924801c130543c1d2d1e9746 not found: ID does not exist" Jul 06 01:22:52.246256 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.246208 2565 scope.go:117] "RemoveContainer" containerID="46a7cd47439380223d0e448b76e87c63f9e17bc8dbfa32cbf19474b1237432c3" Jul 06 01:22:52.246523 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:22:52.246499 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46a7cd47439380223d0e448b76e87c63f9e17bc8dbfa32cbf19474b1237432c3\": container with ID starting with 46a7cd47439380223d0e448b76e87c63f9e17bc8dbfa32cbf19474b1237432c3 not found: ID does not exist" containerID="46a7cd47439380223d0e448b76e87c63f9e17bc8dbfa32cbf19474b1237432c3" Jul 06 01:22:52.246578 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.246555 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46a7cd47439380223d0e448b76e87c63f9e17bc8dbfa32cbf19474b1237432c3"} err="failed to get container status \"46a7cd47439380223d0e448b76e87c63f9e17bc8dbfa32cbf19474b1237432c3\": rpc error: code = NotFound desc = could not find container \"46a7cd47439380223d0e448b76e87c63f9e17bc8dbfa32cbf19474b1237432c3\": container with ID starting with 46a7cd47439380223d0e448b76e87c63f9e17bc8dbfa32cbf19474b1237432c3 not found: ID does not exist" Jul 06 01:22:52.246578 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.246574 2565 scope.go:117] "RemoveContainer" containerID="b55986b74386a55b6e2359ae8c958f54643ad061aa3ba1a210e37f1c477eb5b0" Jul 06 01:22:52.246821 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:22:52.246803 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b55986b74386a55b6e2359ae8c958f54643ad061aa3ba1a210e37f1c477eb5b0\": container with ID starting with b55986b74386a55b6e2359ae8c958f54643ad061aa3ba1a210e37f1c477eb5b0 not found: ID does not exist" containerID="b55986b74386a55b6e2359ae8c958f54643ad061aa3ba1a210e37f1c477eb5b0" Jul 06 01:22:52.246876 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.246826 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b55986b74386a55b6e2359ae8c958f54643ad061aa3ba1a210e37f1c477eb5b0"} err="failed to get container status \"b55986b74386a55b6e2359ae8c958f54643ad061aa3ba1a210e37f1c477eb5b0\": rpc error: code = NotFound desc = could not find container \"b55986b74386a55b6e2359ae8c958f54643ad061aa3ba1a210e37f1c477eb5b0\": container with ID starting with b55986b74386a55b6e2359ae8c958f54643ad061aa3ba1a210e37f1c477eb5b0 not found: ID does not exist" Jul 06 01:22:52.246876 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.246841 2565 scope.go:117] "RemoveContainer" containerID="632e123e80c73cb4a646b924d0ffca9cd732b062c7a74a93cdf273c904b360b3" Jul 06 01:22:52.247075 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:22:52.247062 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"632e123e80c73cb4a646b924d0ffca9cd732b062c7a74a93cdf273c904b360b3\": container with ID starting with 632e123e80c73cb4a646b924d0ffca9cd732b062c7a74a93cdf273c904b360b3 not found: ID does not exist" containerID="632e123e80c73cb4a646b924d0ffca9cd732b062c7a74a93cdf273c904b360b3" Jul 06 01:22:52.247118 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.247079 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"632e123e80c73cb4a646b924d0ffca9cd732b062c7a74a93cdf273c904b360b3"} err="failed to get container status \"632e123e80c73cb4a646b924d0ffca9cd732b062c7a74a93cdf273c904b360b3\": rpc error: code = NotFound desc = could not find container \"632e123e80c73cb4a646b924d0ffca9cd732b062c7a74a93cdf273c904b360b3\": container with ID starting with 632e123e80c73cb4a646b924d0ffca9cd732b062c7a74a93cdf273c904b360b3 not found: ID does not exist" Jul 06 01:22:52.247118 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.247091 2565 scope.go:117] "RemoveContainer" containerID="11b4d907f54a644f6b4be39d72e68ef2bc73ed54d7e86b5d6ebef95f0cab4c02" Jul 06 01:22:52.247308 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:22:52.247290 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11b4d907f54a644f6b4be39d72e68ef2bc73ed54d7e86b5d6ebef95f0cab4c02\": container with ID starting with 11b4d907f54a644f6b4be39d72e68ef2bc73ed54d7e86b5d6ebef95f0cab4c02 not found: ID does not exist" containerID="11b4d907f54a644f6b4be39d72e68ef2bc73ed54d7e86b5d6ebef95f0cab4c02" Jul 06 01:22:52.247347 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.247313 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11b4d907f54a644f6b4be39d72e68ef2bc73ed54d7e86b5d6ebef95f0cab4c02"} err="failed to get container status \"11b4d907f54a644f6b4be39d72e68ef2bc73ed54d7e86b5d6ebef95f0cab4c02\": rpc error: code = NotFound desc = could not find container \"11b4d907f54a644f6b4be39d72e68ef2bc73ed54d7e86b5d6ebef95f0cab4c02\": container with ID starting with 11b4d907f54a644f6b4be39d72e68ef2bc73ed54d7e86b5d6ebef95f0cab4c02 not found: ID does not exist" Jul 06 01:22:52.247347 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.247327 2565 scope.go:117] "RemoveContainer" containerID="df0c5cfdff96164c5c36911483f222439d9a4871dfaed6aba6e25483e8736cb3" Jul 06 01:22:52.247585 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:22:52.247567 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df0c5cfdff96164c5c36911483f222439d9a4871dfaed6aba6e25483e8736cb3\": container with ID starting with df0c5cfdff96164c5c36911483f222439d9a4871dfaed6aba6e25483e8736cb3 not found: ID does not exist" containerID="df0c5cfdff96164c5c36911483f222439d9a4871dfaed6aba6e25483e8736cb3" Jul 06 01:22:52.247629 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.247589 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df0c5cfdff96164c5c36911483f222439d9a4871dfaed6aba6e25483e8736cb3"} err="failed to get container status \"df0c5cfdff96164c5c36911483f222439d9a4871dfaed6aba6e25483e8736cb3\": rpc error: code = NotFound desc = could not find container \"df0c5cfdff96164c5c36911483f222439d9a4871dfaed6aba6e25483e8736cb3\": container with ID starting with df0c5cfdff96164c5c36911483f222439d9a4871dfaed6aba6e25483e8736cb3 not found: ID does not exist" Jul 06 01:22:52.250628 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.250603 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 01:22:52.303062 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.303020 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/14ae4c05-d30f-413c-88b5-bd77d3fbeda6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"14ae4c05-d30f-413c-88b5-bd77d3fbeda6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:52.303062 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.303060 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/14ae4c05-d30f-413c-88b5-bd77d3fbeda6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"14ae4c05-d30f-413c-88b5-bd77d3fbeda6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:52.303347 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.303136 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/14ae4c05-d30f-413c-88b5-bd77d3fbeda6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"14ae4c05-d30f-413c-88b5-bd77d3fbeda6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:52.303347 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.303188 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/14ae4c05-d30f-413c-88b5-bd77d3fbeda6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"14ae4c05-d30f-413c-88b5-bd77d3fbeda6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:52.303347 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.303267 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14ae4c05-d30f-413c-88b5-bd77d3fbeda6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"14ae4c05-d30f-413c-88b5-bd77d3fbeda6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:52.303347 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.303287 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldmnl\" (UniqueName: \"kubernetes.io/projected/14ae4c05-d30f-413c-88b5-bd77d3fbeda6-kube-api-access-ldmnl\") pod \"alertmanager-main-0\" (UID: \"14ae4c05-d30f-413c-88b5-bd77d3fbeda6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:52.303347 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.303302 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/14ae4c05-d30f-413c-88b5-bd77d3fbeda6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"14ae4c05-d30f-413c-88b5-bd77d3fbeda6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:52.303588 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.303354 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/14ae4c05-d30f-413c-88b5-bd77d3fbeda6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"14ae4c05-d30f-413c-88b5-bd77d3fbeda6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:52.303588 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.303401 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/14ae4c05-d30f-413c-88b5-bd77d3fbeda6-config-volume\") pod \"alertmanager-main-0\" (UID: \"14ae4c05-d30f-413c-88b5-bd77d3fbeda6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:52.303588 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.303424 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/14ae4c05-d30f-413c-88b5-bd77d3fbeda6-config-out\") pod \"alertmanager-main-0\" (UID: \"14ae4c05-d30f-413c-88b5-bd77d3fbeda6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:52.303588 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.303448 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/14ae4c05-d30f-413c-88b5-bd77d3fbeda6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"14ae4c05-d30f-413c-88b5-bd77d3fbeda6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:52.303588 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.303467 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/14ae4c05-d30f-413c-88b5-bd77d3fbeda6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"14ae4c05-d30f-413c-88b5-bd77d3fbeda6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:52.303588 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.303482 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/14ae4c05-d30f-413c-88b5-bd77d3fbeda6-web-config\") pod \"alertmanager-main-0\" (UID: \"14ae4c05-d30f-413c-88b5-bd77d3fbeda6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:52.323265 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.323228 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7bf690b-525a-4310-aeb2-206ab91e6346" path="/var/lib/kubelet/pods/d7bf690b-525a-4310-aeb2-206ab91e6346/volumes" Jul 06 01:22:52.404950 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.404908 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/14ae4c05-d30f-413c-88b5-bd77d3fbeda6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"14ae4c05-d30f-413c-88b5-bd77d3fbeda6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:52.404950 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.404950 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/14ae4c05-d30f-413c-88b5-bd77d3fbeda6-config-volume\") pod \"alertmanager-main-0\" (UID: \"14ae4c05-d30f-413c-88b5-bd77d3fbeda6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:52.405221 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.404975 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/14ae4c05-d30f-413c-88b5-bd77d3fbeda6-config-out\") pod \"alertmanager-main-0\" (UID: \"14ae4c05-d30f-413c-88b5-bd77d3fbeda6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:52.405221 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.404998 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/14ae4c05-d30f-413c-88b5-bd77d3fbeda6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"14ae4c05-d30f-413c-88b5-bd77d3fbeda6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:52.405221 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.405021 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/14ae4c05-d30f-413c-88b5-bd77d3fbeda6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"14ae4c05-d30f-413c-88b5-bd77d3fbeda6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:52.405221 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.405037 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/14ae4c05-d30f-413c-88b5-bd77d3fbeda6-web-config\") pod \"alertmanager-main-0\" (UID: \"14ae4c05-d30f-413c-88b5-bd77d3fbeda6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:52.405221 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.405072 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/14ae4c05-d30f-413c-88b5-bd77d3fbeda6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"14ae4c05-d30f-413c-88b5-bd77d3fbeda6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:52.405221 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.405100 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/14ae4c05-d30f-413c-88b5-bd77d3fbeda6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"14ae4c05-d30f-413c-88b5-bd77d3fbeda6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:52.405221 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.405140 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/14ae4c05-d30f-413c-88b5-bd77d3fbeda6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"14ae4c05-d30f-413c-88b5-bd77d3fbeda6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:52.405221 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.405166 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/14ae4c05-d30f-413c-88b5-bd77d3fbeda6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"14ae4c05-d30f-413c-88b5-bd77d3fbeda6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:52.405221 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.405218 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14ae4c05-d30f-413c-88b5-bd77d3fbeda6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"14ae4c05-d30f-413c-88b5-bd77d3fbeda6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:52.405685 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.405242 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldmnl\" (UniqueName: \"kubernetes.io/projected/14ae4c05-d30f-413c-88b5-bd77d3fbeda6-kube-api-access-ldmnl\") pod \"alertmanager-main-0\" (UID: \"14ae4c05-d30f-413c-88b5-bd77d3fbeda6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:52.405685 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.405271 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/14ae4c05-d30f-413c-88b5-bd77d3fbeda6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"14ae4c05-d30f-413c-88b5-bd77d3fbeda6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:52.406010 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.405978 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/14ae4c05-d30f-413c-88b5-bd77d3fbeda6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"14ae4c05-d30f-413c-88b5-bd77d3fbeda6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:52.408451 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.408422 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/14ae4c05-d30f-413c-88b5-bd77d3fbeda6-config-volume\") pod \"alertmanager-main-0\" (UID: \"14ae4c05-d30f-413c-88b5-bd77d3fbeda6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:52.408451 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.408445 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/14ae4c05-d30f-413c-88b5-bd77d3fbeda6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"14ae4c05-d30f-413c-88b5-bd77d3fbeda6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:52.408619 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.408463 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/14ae4c05-d30f-413c-88b5-bd77d3fbeda6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"14ae4c05-d30f-413c-88b5-bd77d3fbeda6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:52.408757 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.408735 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/14ae4c05-d30f-413c-88b5-bd77d3fbeda6-config-out\") pod \"alertmanager-main-0\" (UID: \"14ae4c05-d30f-413c-88b5-bd77d3fbeda6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:52.408816 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.408799 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/14ae4c05-d30f-413c-88b5-bd77d3fbeda6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"14ae4c05-d30f-413c-88b5-bd77d3fbeda6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:52.408982 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.408963 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/14ae4c05-d30f-413c-88b5-bd77d3fbeda6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"14ae4c05-d30f-413c-88b5-bd77d3fbeda6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:52.409069 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.409051 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/14ae4c05-d30f-413c-88b5-bd77d3fbeda6-web-config\") pod \"alertmanager-main-0\" (UID: \"14ae4c05-d30f-413c-88b5-bd77d3fbeda6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:52.409104 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.409074 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/14ae4c05-d30f-413c-88b5-bd77d3fbeda6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"14ae4c05-d30f-413c-88b5-bd77d3fbeda6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:52.409225 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.409204 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/14ae4c05-d30f-413c-88b5-bd77d3fbeda6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"14ae4c05-d30f-413c-88b5-bd77d3fbeda6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:52.409793 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.409775 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14ae4c05-d30f-413c-88b5-bd77d3fbeda6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"14ae4c05-d30f-413c-88b5-bd77d3fbeda6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:52.410813 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.410796 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/14ae4c05-d30f-413c-88b5-bd77d3fbeda6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"14ae4c05-d30f-413c-88b5-bd77d3fbeda6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:52.416845 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.416819 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldmnl\" (UniqueName: \"kubernetes.io/projected/14ae4c05-d30f-413c-88b5-bd77d3fbeda6-kube-api-access-ldmnl\") pod \"alertmanager-main-0\" (UID: \"14ae4c05-d30f-413c-88b5-bd77d3fbeda6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:52.547535 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.547496 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:22:52.720777 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:52.720746 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 01:22:52.723439 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:22:52.723403 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14ae4c05_d30f_413c_88b5_bd77d3fbeda6.slice/crio-b3c57f6fdd0e4879d113e98147f771bcb0920789a18f83931f9536528483e3cf WatchSource:0}: Error finding container b3c57f6fdd0e4879d113e98147f771bcb0920789a18f83931f9536528483e3cf: Status 404 returned error can't find the container with id b3c57f6fdd0e4879d113e98147f771bcb0920789a18f83931f9536528483e3cf Jul 06 01:22:53.184718 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:53.184623 2565 generic.go:356] "Generic (PLEG): container finished" podID="14ae4c05-d30f-413c-88b5-bd77d3fbeda6" containerID="dcf7d005b506a134c99b34aee7e16e27e558d6759c4811d015ede88ca74e48dc" exitCode=0 Jul 06 01:22:53.184718 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:53.184688 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"14ae4c05-d30f-413c-88b5-bd77d3fbeda6","Type":"ContainerDied","Data":"dcf7d005b506a134c99b34aee7e16e27e558d6759c4811d015ede88ca74e48dc"} Jul 06 01:22:53.185131 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:53.184787 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"14ae4c05-d30f-413c-88b5-bd77d3fbeda6","Type":"ContainerStarted","Data":"b3c57f6fdd0e4879d113e98147f771bcb0920789a18f83931f9536528483e3cf"} Jul 06 01:22:54.194263 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.194217 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"14ae4c05-d30f-413c-88b5-bd77d3fbeda6","Type":"ContainerStarted","Data":"f4872f619b4b2143183394f96c020a1616dcef8b0668b16f469079ea796eaad5"} Jul 06 01:22:54.194784 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.194413 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"14ae4c05-d30f-413c-88b5-bd77d3fbeda6","Type":"ContainerStarted","Data":"5858ff9982a50149c62ad0967b33f542a4ad40602c9061edd9fc425c99c7ed24"} Jul 06 01:22:54.194784 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.194499 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"14ae4c05-d30f-413c-88b5-bd77d3fbeda6","Type":"ContainerStarted","Data":"10436b5e565f4e2bd1dcd99c57a152f260045913672e97e2e03d8eed0edd1564"} Jul 06 01:22:54.194784 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.194564 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"14ae4c05-d30f-413c-88b5-bd77d3fbeda6","Type":"ContainerStarted","Data":"7fe0a0ffb8e68068645e6124babd4315aad873250323f61944ad408f3cd07572"} Jul 06 01:22:54.194784 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.194648 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"14ae4c05-d30f-413c-88b5-bd77d3fbeda6","Type":"ContainerStarted","Data":"93bd6e0f85459e9c89f3977fc6c3002da209ab867f1f2599ca5ed64f1c2d7948"} Jul 06 01:22:54.194784 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.194684 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"14ae4c05-d30f-413c-88b5-bd77d3fbeda6","Type":"ContainerStarted","Data":"fc035c15d5b792c79be567e2bf3da46ac4ebfc129364eaf69e921bfb0cd6b964"} Jul 06 01:22:54.223199 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.223134 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.206366625 podStartE2EDuration="2.22311272s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.746095ms" isStatefulPod=false podCreationTimestamp="2026-07-06 01:22:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:22:54.22148903 +0000 UTC m=+146.363047815" watchObservedRunningTime="2026-07-06 01:22:54.22311272 +0000 UTC m=+146.364671506" Jul 06 01:22:54.463930 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.463842 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 01:22:54.464360 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.464152 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f5cf93c2-af80-4384-add5-49cdd601b40c" containerName="prometheus" containerID="cri-o://8e35dea0111da30f93c5572e670a19b04a7796de3fe1a390659ebd87a47b4c44" gracePeriod=600 Jul 06 01:22:54.464360 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.464176 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f5cf93c2-af80-4384-add5-49cdd601b40c" containerName="kube-rbac-proxy" containerID="cri-o://5d6d090979962c7cbdfea6b12f745ae0d4bc743eb7222b0ea4faf3fced052c80" gracePeriod=600 Jul 06 01:22:54.464360 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.464190 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f5cf93c2-af80-4384-add5-49cdd601b40c" containerName="kube-rbac-proxy-web" containerID="cri-o://64f901021a44f7d5134297d7e2968ad0cc63ff814ae2c097948bd81392bbd868" gracePeriod=600 Jul 06 01:22:54.464360 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.464192 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f5cf93c2-af80-4384-add5-49cdd601b40c" containerName="thanos-sidecar" containerID="cri-o://608cde94a0548d7f5e181750409054de510aeb53254655cae7e7173048956e18" gracePeriod=600 Jul 06 01:22:54.464360 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.464281 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f5cf93c2-af80-4384-add5-49cdd601b40c" containerName="kube-rbac-proxy-thanos" containerID="cri-o://7481585220cf802cc416c54b694536f1550363ba218d891257702cb267073a14" gracePeriod=600 Jul 06 01:22:54.464360 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.464266 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f5cf93c2-af80-4384-add5-49cdd601b40c" containerName="config-reloader" containerID="cri-o://e69034755d0292d6c7808ef2126f934888ec8cf408afe632dd385d711a8042a4" gracePeriod=600 Jul 06 01:22:54.728856 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.728817 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:54.828283 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.828248 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-secret-grpc-tls\") pod \"f5cf93c2-af80-4384-add5-49cdd601b40c\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " Jul 06 01:22:54.828283 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.828288 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"f5cf93c2-af80-4384-add5-49cdd601b40c\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " Jul 06 01:22:54.828559 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.828308 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-config\") pod \"f5cf93c2-af80-4384-add5-49cdd601b40c\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " Jul 06 01:22:54.828559 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.828334 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-secret-prometheus-k8s-tls\") pod \"f5cf93c2-af80-4384-add5-49cdd601b40c\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " Jul 06 01:22:54.828559 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.828361 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f5cf93c2-af80-4384-add5-49cdd601b40c-prometheus-k8s-rulefiles-1\") pod \"f5cf93c2-af80-4384-add5-49cdd601b40c\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " Jul 06 01:22:54.828559 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.828534 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f5cf93c2-af80-4384-add5-49cdd601b40c-prometheus-k8s-rulefiles-2\") pod \"f5cf93c2-af80-4384-add5-49cdd601b40c\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " Jul 06 01:22:54.828701 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.828576 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f5cf93c2-af80-4384-add5-49cdd601b40c-configmap-metrics-client-ca\") pod \"f5cf93c2-af80-4384-add5-49cdd601b40c\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " Jul 06 01:22:54.828753 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.828722 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-secret-kube-rbac-proxy\") pod \"f5cf93c2-af80-4384-add5-49cdd601b40c\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " Jul 06 01:22:54.828806 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.828776 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"f5cf93c2-af80-4384-add5-49cdd601b40c\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " Jul 06 01:22:54.828862 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.828807 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dfs5k\" (UniqueName: \"kubernetes.io/projected/f5cf93c2-af80-4384-add5-49cdd601b40c-kube-api-access-dfs5k\") pod \"f5cf93c2-af80-4384-add5-49cdd601b40c\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " Jul 06 01:22:54.828862 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.828831 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f5cf93c2-af80-4384-add5-49cdd601b40c-prometheus-k8s-rulefiles-0\") pod \"f5cf93c2-af80-4384-add5-49cdd601b40c\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " Jul 06 01:22:54.828862 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.828851 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5cf93c2-af80-4384-add5-49cdd601b40c-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "f5cf93c2-af80-4384-add5-49cdd601b40c" (UID: "f5cf93c2-af80-4384-add5-49cdd601b40c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 01:22:54.829024 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.828864 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-secret-metrics-client-certs\") pod \"f5cf93c2-af80-4384-add5-49cdd601b40c\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " Jul 06 01:22:54.829024 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.828882 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5cf93c2-af80-4384-add5-49cdd601b40c-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "f5cf93c2-af80-4384-add5-49cdd601b40c" (UID: "f5cf93c2-af80-4384-add5-49cdd601b40c"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 01:22:54.829024 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.828891 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f5cf93c2-af80-4384-add5-49cdd601b40c-tls-assets\") pod \"f5cf93c2-af80-4384-add5-49cdd601b40c\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " Jul 06 01:22:54.829024 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.828936 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5cf93c2-af80-4384-add5-49cdd601b40c-configmap-kubelet-serving-ca-bundle\") pod \"f5cf93c2-af80-4384-add5-49cdd601b40c\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " Jul 06 01:22:54.829024 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.828975 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-web-config\") pod \"f5cf93c2-af80-4384-add5-49cdd601b40c\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " Jul 06 01:22:54.829024 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.829007 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-thanos-prometheus-http-client-file\") pod \"f5cf93c2-af80-4384-add5-49cdd601b40c\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " Jul 06 01:22:54.829490 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.829040 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5cf93c2-af80-4384-add5-49cdd601b40c-prometheus-trusted-ca-bundle\") pod \"f5cf93c2-af80-4384-add5-49cdd601b40c\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " Jul 06 01:22:54.829490 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.829109 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f5cf93c2-af80-4384-add5-49cdd601b40c-prometheus-k8s-db\") pod \"f5cf93c2-af80-4384-add5-49cdd601b40c\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " Jul 06 01:22:54.829490 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.829238 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5cf93c2-af80-4384-add5-49cdd601b40c-configmap-serving-certs-ca-bundle\") pod \"f5cf93c2-af80-4384-add5-49cdd601b40c\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " Jul 06 01:22:54.829490 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.829275 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f5cf93c2-af80-4384-add5-49cdd601b40c-config-out\") pod \"f5cf93c2-af80-4384-add5-49cdd601b40c\" (UID: \"f5cf93c2-af80-4384-add5-49cdd601b40c\") " Jul 06 01:22:54.829490 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.829414 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5cf93c2-af80-4384-add5-49cdd601b40c-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "f5cf93c2-af80-4384-add5-49cdd601b40c" (UID: "f5cf93c2-af80-4384-add5-49cdd601b40c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 01:22:54.829733 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.829616 2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f5cf93c2-af80-4384-add5-49cdd601b40c-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:22:54.829733 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.829637 2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f5cf93c2-af80-4384-add5-49cdd601b40c-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:22:54.829733 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.829652 2565 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f5cf93c2-af80-4384-add5-49cdd601b40c-configmap-metrics-client-ca\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:22:54.829929 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.829905 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5cf93c2-af80-4384-add5-49cdd601b40c-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "f5cf93c2-af80-4384-add5-49cdd601b40c" (UID: "f5cf93c2-af80-4384-add5-49cdd601b40c"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 01:22:54.830508 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.830365 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5cf93c2-af80-4384-add5-49cdd601b40c-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "f5cf93c2-af80-4384-add5-49cdd601b40c" (UID: "f5cf93c2-af80-4384-add5-49cdd601b40c"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 01:22:54.831182 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.831154 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5cf93c2-af80-4384-add5-49cdd601b40c-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "f5cf93c2-af80-4384-add5-49cdd601b40c" (UID: "f5cf93c2-af80-4384-add5-49cdd601b40c"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:22:54.831862 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.831832 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5cf93c2-af80-4384-add5-49cdd601b40c-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "f5cf93c2-af80-4384-add5-49cdd601b40c" (UID: "f5cf93c2-af80-4384-add5-49cdd601b40c"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 01:22:54.832323 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.832296 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "f5cf93c2-af80-4384-add5-49cdd601b40c" (UID: "f5cf93c2-af80-4384-add5-49cdd601b40c"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 01:22:54.832433 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.832417 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "f5cf93c2-af80-4384-add5-49cdd601b40c" (UID: "f5cf93c2-af80-4384-add5-49cdd601b40c"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 01:22:54.832854 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.832724 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-config" (OuterVolumeSpecName: "config") pod "f5cf93c2-af80-4384-add5-49cdd601b40c" (UID: "f5cf93c2-af80-4384-add5-49cdd601b40c"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 01:22:54.833027 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.832993 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "f5cf93c2-af80-4384-add5-49cdd601b40c" (UID: "f5cf93c2-af80-4384-add5-49cdd601b40c"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 01:22:54.833027 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.833002 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5cf93c2-af80-4384-add5-49cdd601b40c-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "f5cf93c2-af80-4384-add5-49cdd601b40c" (UID: "f5cf93c2-af80-4384-add5-49cdd601b40c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 01:22:54.833455 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.833425 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "f5cf93c2-af80-4384-add5-49cdd601b40c" (UID: "f5cf93c2-af80-4384-add5-49cdd601b40c"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 01:22:54.833744 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.833719 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "f5cf93c2-af80-4384-add5-49cdd601b40c" (UID: "f5cf93c2-af80-4384-add5-49cdd601b40c"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 01:22:54.833811 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.833735 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5cf93c2-af80-4384-add5-49cdd601b40c-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "f5cf93c2-af80-4384-add5-49cdd601b40c" (UID: "f5cf93c2-af80-4384-add5-49cdd601b40c"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:22:54.833924 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.833901 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5cf93c2-af80-4384-add5-49cdd601b40c-kube-api-access-dfs5k" (OuterVolumeSpecName: "kube-api-access-dfs5k") pod "f5cf93c2-af80-4384-add5-49cdd601b40c" (UID: "f5cf93c2-af80-4384-add5-49cdd601b40c"). InnerVolumeSpecName "kube-api-access-dfs5k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:22:54.834013 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.833998 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5cf93c2-af80-4384-add5-49cdd601b40c-config-out" (OuterVolumeSpecName: "config-out") pod "f5cf93c2-af80-4384-add5-49cdd601b40c" (UID: "f5cf93c2-af80-4384-add5-49cdd601b40c"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:22:54.834248 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.834228 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "f5cf93c2-af80-4384-add5-49cdd601b40c" (UID: "f5cf93c2-af80-4384-add5-49cdd601b40c"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 01:22:54.835071 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.835047 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "f5cf93c2-af80-4384-add5-49cdd601b40c" (UID: "f5cf93c2-af80-4384-add5-49cdd601b40c"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 01:22:54.846017 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.845976 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-web-config" (OuterVolumeSpecName: "web-config") pod "f5cf93c2-af80-4384-add5-49cdd601b40c" (UID: "f5cf93c2-af80-4384-add5-49cdd601b40c"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 01:22:54.930225 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.930182 2565 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5cf93c2-af80-4384-add5-49cdd601b40c-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:22:54.930225 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.930218 2565 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f5cf93c2-af80-4384-add5-49cdd601b40c-config-out\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:22:54.930225 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.930228 2565 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-secret-grpc-tls\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:22:54.930500 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.930237 2565 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:22:54.930500 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.930247 2565 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-config\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:22:54.930500 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.930257 2565 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-secret-prometheus-k8s-tls\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:22:54.930500 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.930266 2565 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-secret-kube-rbac-proxy\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:22:54.930500 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.930275 2565 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:22:54.930500 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.930284 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dfs5k\" (UniqueName: \"kubernetes.io/projected/f5cf93c2-af80-4384-add5-49cdd601b40c-kube-api-access-dfs5k\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:22:54.930500 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.930294 2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f5cf93c2-af80-4384-add5-49cdd601b40c-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:22:54.930500 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.930303 2565 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-secret-metrics-client-certs\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:22:54.930500 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.930311 2565 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f5cf93c2-af80-4384-add5-49cdd601b40c-tls-assets\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:22:54.930500 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.930320 2565 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5cf93c2-af80-4384-add5-49cdd601b40c-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:22:54.930500 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.930328 2565 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-web-config\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:22:54.930500 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.930352 2565 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f5cf93c2-af80-4384-add5-49cdd601b40c-thanos-prometheus-http-client-file\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:22:54.930500 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.930362 2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5cf93c2-af80-4384-add5-49cdd601b40c-prometheus-trusted-ca-bundle\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:22:54.930500 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:54.930370 2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f5cf93c2-af80-4384-add5-49cdd601b40c-prometheus-k8s-db\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:22:55.205326 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.205287 2565 generic.go:356] "Generic (PLEG): container finished" podID="f5cf93c2-af80-4384-add5-49cdd601b40c" containerID="7481585220cf802cc416c54b694536f1550363ba218d891257702cb267073a14" exitCode=0 Jul 06 01:22:55.205326 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.205315 2565 generic.go:356] "Generic (PLEG): container finished" podID="f5cf93c2-af80-4384-add5-49cdd601b40c" containerID="5d6d090979962c7cbdfea6b12f745ae0d4bc743eb7222b0ea4faf3fced052c80" exitCode=0 Jul 06 01:22:55.205326 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.205325 2565 generic.go:356] "Generic (PLEG): container finished" podID="f5cf93c2-af80-4384-add5-49cdd601b40c" containerID="64f901021a44f7d5134297d7e2968ad0cc63ff814ae2c097948bd81392bbd868" exitCode=0 Jul 06 01:22:55.205326 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.205332 2565 generic.go:356] "Generic (PLEG): container finished" podID="f5cf93c2-af80-4384-add5-49cdd601b40c" containerID="608cde94a0548d7f5e181750409054de510aeb53254655cae7e7173048956e18" exitCode=0 Jul 06 01:22:55.205936 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.205339 2565 generic.go:356] "Generic (PLEG): container finished" podID="f5cf93c2-af80-4384-add5-49cdd601b40c" containerID="e69034755d0292d6c7808ef2126f934888ec8cf408afe632dd385d711a8042a4" exitCode=0 Jul 06 01:22:55.205936 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.205346 2565 generic.go:356] "Generic (PLEG): container finished" podID="f5cf93c2-af80-4384-add5-49cdd601b40c" containerID="8e35dea0111da30f93c5572e670a19b04a7796de3fe1a390659ebd87a47b4c44" exitCode=0 Jul 06 01:22:55.205936 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.205367 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5cf93c2-af80-4384-add5-49cdd601b40c","Type":"ContainerDied","Data":"7481585220cf802cc416c54b694536f1550363ba218d891257702cb267073a14"} Jul 06 01:22:55.205936 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.205395 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.205936 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.205472 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5cf93c2-af80-4384-add5-49cdd601b40c","Type":"ContainerDied","Data":"5d6d090979962c7cbdfea6b12f745ae0d4bc743eb7222b0ea4faf3fced052c80"} Jul 06 01:22:55.205936 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.205500 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5cf93c2-af80-4384-add5-49cdd601b40c","Type":"ContainerDied","Data":"64f901021a44f7d5134297d7e2968ad0cc63ff814ae2c097948bd81392bbd868"} Jul 06 01:22:55.205936 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.205526 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5cf93c2-af80-4384-add5-49cdd601b40c","Type":"ContainerDied","Data":"608cde94a0548d7f5e181750409054de510aeb53254655cae7e7173048956e18"} Jul 06 01:22:55.205936 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.205536 2565 scope.go:117] "RemoveContainer" containerID="7481585220cf802cc416c54b694536f1550363ba218d891257702cb267073a14" Jul 06 01:22:55.205936 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.205551 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5cf93c2-af80-4384-add5-49cdd601b40c","Type":"ContainerDied","Data":"e69034755d0292d6c7808ef2126f934888ec8cf408afe632dd385d711a8042a4"} Jul 06 01:22:55.205936 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.205576 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5cf93c2-af80-4384-add5-49cdd601b40c","Type":"ContainerDied","Data":"8e35dea0111da30f93c5572e670a19b04a7796de3fe1a390659ebd87a47b4c44"} Jul 06 01:22:55.205936 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.205601 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5cf93c2-af80-4384-add5-49cdd601b40c","Type":"ContainerDied","Data":"c56cd43b0958ada45eb772b9645abeb0f6d95242b801fb1413e379f409b4ff21"} Jul 06 01:22:55.214066 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.214006 2565 scope.go:117] "RemoveContainer" containerID="5d6d090979962c7cbdfea6b12f745ae0d4bc743eb7222b0ea4faf3fced052c80" Jul 06 01:22:55.222364 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.222342 2565 scope.go:117] "RemoveContainer" containerID="64f901021a44f7d5134297d7e2968ad0cc63ff814ae2c097948bd81392bbd868" Jul 06 01:22:55.231062 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.231033 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 01:22:55.235125 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.235078 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 01:22:55.254252 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.254221 2565 scope.go:117] "RemoveContainer" containerID="608cde94a0548d7f5e181750409054de510aeb53254655cae7e7173048956e18" Jul 06 01:22:55.261064 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.261037 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 01:22:55.261812 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.261790 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5cf93c2-af80-4384-add5-49cdd601b40c" containerName="kube-rbac-proxy-web" Jul 06 01:22:55.261964 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.261953 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5cf93c2-af80-4384-add5-49cdd601b40c" containerName="kube-rbac-proxy-web" Jul 06 01:22:55.262048 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.262040 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5cf93c2-af80-4384-add5-49cdd601b40c" containerName="prometheus" Jul 06 01:22:55.262110 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.262102 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5cf93c2-af80-4384-add5-49cdd601b40c" containerName="prometheus" Jul 06 01:22:55.262181 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.262173 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5cf93c2-af80-4384-add5-49cdd601b40c" containerName="config-reloader" Jul 06 01:22:55.262268 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.262260 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5cf93c2-af80-4384-add5-49cdd601b40c" containerName="config-reloader" Jul 06 01:22:55.262436 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.262424 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5cf93c2-af80-4384-add5-49cdd601b40c" containerName="init-config-reloader" Jul 06 01:22:55.262525 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.262516 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5cf93c2-af80-4384-add5-49cdd601b40c" containerName="init-config-reloader" Jul 06 01:22:55.262643 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.262632 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5cf93c2-af80-4384-add5-49cdd601b40c" containerName="kube-rbac-proxy-thanos" Jul 06 01:22:55.262707 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.262699 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5cf93c2-af80-4384-add5-49cdd601b40c" containerName="kube-rbac-proxy-thanos" Jul 06 01:22:55.262784 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.262775 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5cf93c2-af80-4384-add5-49cdd601b40c" containerName="thanos-sidecar" Jul 06 01:22:55.262842 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.262834 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5cf93c2-af80-4384-add5-49cdd601b40c" containerName="thanos-sidecar" Jul 06 01:22:55.262899 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.262873 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5cf93c2-af80-4384-add5-49cdd601b40c" containerName="kube-rbac-proxy" Jul 06 01:22:55.262899 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.262887 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5cf93c2-af80-4384-add5-49cdd601b40c" containerName="kube-rbac-proxy" Jul 06 01:22:55.263485 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.262977 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5cf93c2-af80-4384-add5-49cdd601b40c" containerName="kube-rbac-proxy-thanos" Jul 06 01:22:55.263485 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.262992 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5cf93c2-af80-4384-add5-49cdd601b40c" containerName="prometheus" Jul 06 01:22:55.263485 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.263000 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5cf93c2-af80-4384-add5-49cdd601b40c" containerName="config-reloader" Jul 06 01:22:55.263485 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.263009 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5cf93c2-af80-4384-add5-49cdd601b40c" containerName="kube-rbac-proxy" Jul 06 01:22:55.263485 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.263020 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5cf93c2-af80-4384-add5-49cdd601b40c" containerName="kube-rbac-proxy-web" Jul 06 01:22:55.263485 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.263029 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5cf93c2-af80-4384-add5-49cdd601b40c" containerName="thanos-sidecar" Jul 06 01:22:55.267363 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.267337 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.268872 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.268845 2565 scope.go:117] "RemoveContainer" containerID="e69034755d0292d6c7808ef2126f934888ec8cf408afe632dd385d711a8042a4" Jul 06 01:22:55.270268 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.270228 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 06 01:22:55.271252 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.270506 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 06 01:22:55.271252 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.270742 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 06 01:22:55.271252 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.270760 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-mhhpf\"" Jul 06 01:22:55.271252 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.271117 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 06 01:22:55.271252 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.271132 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 06 01:22:55.271252 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.271131 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 06 01:22:55.271252 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.271254 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 06 01:22:55.271801 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.271496 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-fpc9i5daqe19b\"" Jul 06 01:22:55.271801 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.271576 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 06 01:22:55.271962 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.271944 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 06 01:22:55.272104 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.272085 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 06 01:22:55.272220 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.272096 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 06 01:22:55.272220 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.272143 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 06 01:22:55.274112 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.274093 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 06 01:22:55.276291 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.276056 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 06 01:22:55.278472 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.278364 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 01:22:55.279698 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.279677 2565 scope.go:117] "RemoveContainer" containerID="8e35dea0111da30f93c5572e670a19b04a7796de3fe1a390659ebd87a47b4c44" Jul 06 01:22:55.288928 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.288907 2565 scope.go:117] "RemoveContainer" containerID="f3aaad56290da5b1fb8a6fe3ade74d8c27450ed46845facd8b053c768be8de89" Jul 06 01:22:55.296540 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.296521 2565 scope.go:117] "RemoveContainer" containerID="7481585220cf802cc416c54b694536f1550363ba218d891257702cb267073a14" Jul 06 01:22:55.296845 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:22:55.296820 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7481585220cf802cc416c54b694536f1550363ba218d891257702cb267073a14\": container with ID starting with 7481585220cf802cc416c54b694536f1550363ba218d891257702cb267073a14 not found: ID does not exist" containerID="7481585220cf802cc416c54b694536f1550363ba218d891257702cb267073a14" Jul 06 01:22:55.296906 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.296854 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7481585220cf802cc416c54b694536f1550363ba218d891257702cb267073a14"} err="failed to get container status \"7481585220cf802cc416c54b694536f1550363ba218d891257702cb267073a14\": rpc error: code = NotFound desc = could not find container \"7481585220cf802cc416c54b694536f1550363ba218d891257702cb267073a14\": container with ID starting with 7481585220cf802cc416c54b694536f1550363ba218d891257702cb267073a14 not found: ID does not exist" Jul 06 01:22:55.296906 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.296874 2565 scope.go:117] "RemoveContainer" containerID="5d6d090979962c7cbdfea6b12f745ae0d4bc743eb7222b0ea4faf3fced052c80" Jul 06 01:22:55.297140 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:22:55.297123 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d6d090979962c7cbdfea6b12f745ae0d4bc743eb7222b0ea4faf3fced052c80\": container with ID starting with 5d6d090979962c7cbdfea6b12f745ae0d4bc743eb7222b0ea4faf3fced052c80 not found: ID does not exist" containerID="5d6d090979962c7cbdfea6b12f745ae0d4bc743eb7222b0ea4faf3fced052c80" Jul 06 01:22:55.297191 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.297149 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d6d090979962c7cbdfea6b12f745ae0d4bc743eb7222b0ea4faf3fced052c80"} err="failed to get container status \"5d6d090979962c7cbdfea6b12f745ae0d4bc743eb7222b0ea4faf3fced052c80\": rpc error: code = NotFound desc = could not find container \"5d6d090979962c7cbdfea6b12f745ae0d4bc743eb7222b0ea4faf3fced052c80\": container with ID starting with 5d6d090979962c7cbdfea6b12f745ae0d4bc743eb7222b0ea4faf3fced052c80 not found: ID does not exist" Jul 06 01:22:55.297191 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.297170 2565 scope.go:117] "RemoveContainer" containerID="64f901021a44f7d5134297d7e2968ad0cc63ff814ae2c097948bd81392bbd868" Jul 06 01:22:55.297542 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:22:55.297498 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64f901021a44f7d5134297d7e2968ad0cc63ff814ae2c097948bd81392bbd868\": container with ID starting with 64f901021a44f7d5134297d7e2968ad0cc63ff814ae2c097948bd81392bbd868 not found: ID does not exist" containerID="64f901021a44f7d5134297d7e2968ad0cc63ff814ae2c097948bd81392bbd868" Jul 06 01:22:55.297635 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.297557 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64f901021a44f7d5134297d7e2968ad0cc63ff814ae2c097948bd81392bbd868"} err="failed to get container status \"64f901021a44f7d5134297d7e2968ad0cc63ff814ae2c097948bd81392bbd868\": rpc error: code = NotFound desc = could not find container \"64f901021a44f7d5134297d7e2968ad0cc63ff814ae2c097948bd81392bbd868\": container with ID starting with 64f901021a44f7d5134297d7e2968ad0cc63ff814ae2c097948bd81392bbd868 not found: ID does not exist" Jul 06 01:22:55.297635 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.297575 2565 scope.go:117] "RemoveContainer" containerID="608cde94a0548d7f5e181750409054de510aeb53254655cae7e7173048956e18" Jul 06 01:22:55.297911 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:22:55.297891 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"608cde94a0548d7f5e181750409054de510aeb53254655cae7e7173048956e18\": container with ID starting with 608cde94a0548d7f5e181750409054de510aeb53254655cae7e7173048956e18 not found: ID does not exist" containerID="608cde94a0548d7f5e181750409054de510aeb53254655cae7e7173048956e18" Jul 06 01:22:55.297978 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.297933 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"608cde94a0548d7f5e181750409054de510aeb53254655cae7e7173048956e18"} err="failed to get container status \"608cde94a0548d7f5e181750409054de510aeb53254655cae7e7173048956e18\": rpc error: code = NotFound desc = could not find container \"608cde94a0548d7f5e181750409054de510aeb53254655cae7e7173048956e18\": container with ID starting with 608cde94a0548d7f5e181750409054de510aeb53254655cae7e7173048956e18 not found: ID does not exist" Jul 06 01:22:55.297978 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.297954 2565 scope.go:117] "RemoveContainer" containerID="e69034755d0292d6c7808ef2126f934888ec8cf408afe632dd385d711a8042a4" Jul 06 01:22:55.298355 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:22:55.298334 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e69034755d0292d6c7808ef2126f934888ec8cf408afe632dd385d711a8042a4\": container with ID starting with e69034755d0292d6c7808ef2126f934888ec8cf408afe632dd385d711a8042a4 not found: ID does not exist" containerID="e69034755d0292d6c7808ef2126f934888ec8cf408afe632dd385d711a8042a4" Jul 06 01:22:55.298465 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.298366 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e69034755d0292d6c7808ef2126f934888ec8cf408afe632dd385d711a8042a4"} err="failed to get container status \"e69034755d0292d6c7808ef2126f934888ec8cf408afe632dd385d711a8042a4\": rpc error: code = NotFound desc = could not find container \"e69034755d0292d6c7808ef2126f934888ec8cf408afe632dd385d711a8042a4\": container with ID starting with e69034755d0292d6c7808ef2126f934888ec8cf408afe632dd385d711a8042a4 not found: ID does not exist" Jul 06 01:22:55.298465 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.298410 2565 scope.go:117] "RemoveContainer" containerID="8e35dea0111da30f93c5572e670a19b04a7796de3fe1a390659ebd87a47b4c44" Jul 06 01:22:55.298686 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:22:55.298668 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e35dea0111da30f93c5572e670a19b04a7796de3fe1a390659ebd87a47b4c44\": container with ID starting with 8e35dea0111da30f93c5572e670a19b04a7796de3fe1a390659ebd87a47b4c44 not found: ID does not exist" containerID="8e35dea0111da30f93c5572e670a19b04a7796de3fe1a390659ebd87a47b4c44" Jul 06 01:22:55.298735 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.298690 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e35dea0111da30f93c5572e670a19b04a7796de3fe1a390659ebd87a47b4c44"} err="failed to get container status \"8e35dea0111da30f93c5572e670a19b04a7796de3fe1a390659ebd87a47b4c44\": rpc error: code = NotFound desc = could not find container \"8e35dea0111da30f93c5572e670a19b04a7796de3fe1a390659ebd87a47b4c44\": container with ID starting with 8e35dea0111da30f93c5572e670a19b04a7796de3fe1a390659ebd87a47b4c44 not found: ID does not exist" Jul 06 01:22:55.298735 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.298704 2565 scope.go:117] "RemoveContainer" containerID="f3aaad56290da5b1fb8a6fe3ade74d8c27450ed46845facd8b053c768be8de89" Jul 06 01:22:55.298912 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:22:55.298894 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3aaad56290da5b1fb8a6fe3ade74d8c27450ed46845facd8b053c768be8de89\": container with ID starting with f3aaad56290da5b1fb8a6fe3ade74d8c27450ed46845facd8b053c768be8de89 not found: ID does not exist" containerID="f3aaad56290da5b1fb8a6fe3ade74d8c27450ed46845facd8b053c768be8de89" Jul 06 01:22:55.299030 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.298920 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3aaad56290da5b1fb8a6fe3ade74d8c27450ed46845facd8b053c768be8de89"} err="failed to get container status \"f3aaad56290da5b1fb8a6fe3ade74d8c27450ed46845facd8b053c768be8de89\": rpc error: code = NotFound desc = could not find container \"f3aaad56290da5b1fb8a6fe3ade74d8c27450ed46845facd8b053c768be8de89\": container with ID starting with f3aaad56290da5b1fb8a6fe3ade74d8c27450ed46845facd8b053c768be8de89 not found: ID does not exist" Jul 06 01:22:55.299030 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.298939 2565 scope.go:117] "RemoveContainer" containerID="7481585220cf802cc416c54b694536f1550363ba218d891257702cb267073a14" Jul 06 01:22:55.299179 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.299164 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7481585220cf802cc416c54b694536f1550363ba218d891257702cb267073a14"} err="failed to get container status \"7481585220cf802cc416c54b694536f1550363ba218d891257702cb267073a14\": rpc error: code = NotFound desc = could not find container \"7481585220cf802cc416c54b694536f1550363ba218d891257702cb267073a14\": container with ID starting with 7481585220cf802cc416c54b694536f1550363ba218d891257702cb267073a14 not found: ID does not exist" Jul 06 01:22:55.299224 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.299180 2565 scope.go:117] "RemoveContainer" containerID="5d6d090979962c7cbdfea6b12f745ae0d4bc743eb7222b0ea4faf3fced052c80" Jul 06 01:22:55.299422 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.299360 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d6d090979962c7cbdfea6b12f745ae0d4bc743eb7222b0ea4faf3fced052c80"} err="failed to get container status \"5d6d090979962c7cbdfea6b12f745ae0d4bc743eb7222b0ea4faf3fced052c80\": rpc error: code = NotFound desc = could not find container \"5d6d090979962c7cbdfea6b12f745ae0d4bc743eb7222b0ea4faf3fced052c80\": container with ID starting with 5d6d090979962c7cbdfea6b12f745ae0d4bc743eb7222b0ea4faf3fced052c80 not found: ID does not exist" Jul 06 01:22:55.299422 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.299408 2565 scope.go:117] "RemoveContainer" containerID="64f901021a44f7d5134297d7e2968ad0cc63ff814ae2c097948bd81392bbd868" Jul 06 01:22:55.299599 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.299582 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64f901021a44f7d5134297d7e2968ad0cc63ff814ae2c097948bd81392bbd868"} err="failed to get container status \"64f901021a44f7d5134297d7e2968ad0cc63ff814ae2c097948bd81392bbd868\": rpc error: code = NotFound desc = could not find container \"64f901021a44f7d5134297d7e2968ad0cc63ff814ae2c097948bd81392bbd868\": container with ID starting with 64f901021a44f7d5134297d7e2968ad0cc63ff814ae2c097948bd81392bbd868 not found: ID does not exist" Jul 06 01:22:55.299634 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.299600 2565 scope.go:117] "RemoveContainer" containerID="608cde94a0548d7f5e181750409054de510aeb53254655cae7e7173048956e18" Jul 06 01:22:55.299783 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.299763 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"608cde94a0548d7f5e181750409054de510aeb53254655cae7e7173048956e18"} err="failed to get container status \"608cde94a0548d7f5e181750409054de510aeb53254655cae7e7173048956e18\": rpc error: code = NotFound desc = could not find container \"608cde94a0548d7f5e181750409054de510aeb53254655cae7e7173048956e18\": container with ID starting with 608cde94a0548d7f5e181750409054de510aeb53254655cae7e7173048956e18 not found: ID does not exist" Jul 06 01:22:55.299822 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.299784 2565 scope.go:117] "RemoveContainer" containerID="e69034755d0292d6c7808ef2126f934888ec8cf408afe632dd385d711a8042a4" Jul 06 01:22:55.299976 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.299949 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e69034755d0292d6c7808ef2126f934888ec8cf408afe632dd385d711a8042a4"} err="failed to get container status \"e69034755d0292d6c7808ef2126f934888ec8cf408afe632dd385d711a8042a4\": rpc error: code = NotFound desc = could not find container \"e69034755d0292d6c7808ef2126f934888ec8cf408afe632dd385d711a8042a4\": container with ID starting with e69034755d0292d6c7808ef2126f934888ec8cf408afe632dd385d711a8042a4 not found: ID does not exist" Jul 06 01:22:55.299976 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.299969 2565 scope.go:117] "RemoveContainer" containerID="8e35dea0111da30f93c5572e670a19b04a7796de3fe1a390659ebd87a47b4c44" Jul 06 01:22:55.300167 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.300151 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e35dea0111da30f93c5572e670a19b04a7796de3fe1a390659ebd87a47b4c44"} err="failed to get container status \"8e35dea0111da30f93c5572e670a19b04a7796de3fe1a390659ebd87a47b4c44\": rpc error: code = NotFound desc = could not find container \"8e35dea0111da30f93c5572e670a19b04a7796de3fe1a390659ebd87a47b4c44\": container with ID starting with 8e35dea0111da30f93c5572e670a19b04a7796de3fe1a390659ebd87a47b4c44 not found: ID does not exist" Jul 06 01:22:55.300204 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.300168 2565 scope.go:117] "RemoveContainer" containerID="f3aaad56290da5b1fb8a6fe3ade74d8c27450ed46845facd8b053c768be8de89" Jul 06 01:22:55.300329 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.300314 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3aaad56290da5b1fb8a6fe3ade74d8c27450ed46845facd8b053c768be8de89"} err="failed to get container status \"f3aaad56290da5b1fb8a6fe3ade74d8c27450ed46845facd8b053c768be8de89\": rpc error: code = NotFound desc = could not find container \"f3aaad56290da5b1fb8a6fe3ade74d8c27450ed46845facd8b053c768be8de89\": container with ID starting with f3aaad56290da5b1fb8a6fe3ade74d8c27450ed46845facd8b053c768be8de89 not found: ID does not exist" Jul 06 01:22:55.300366 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.300330 2565 scope.go:117] "RemoveContainer" containerID="7481585220cf802cc416c54b694536f1550363ba218d891257702cb267073a14" Jul 06 01:22:55.300526 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.300503 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7481585220cf802cc416c54b694536f1550363ba218d891257702cb267073a14"} err="failed to get container status \"7481585220cf802cc416c54b694536f1550363ba218d891257702cb267073a14\": rpc error: code = NotFound desc = could not find container \"7481585220cf802cc416c54b694536f1550363ba218d891257702cb267073a14\": container with ID starting with 7481585220cf802cc416c54b694536f1550363ba218d891257702cb267073a14 not found: ID does not exist" Jul 06 01:22:55.300583 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.300526 2565 scope.go:117] "RemoveContainer" containerID="5d6d090979962c7cbdfea6b12f745ae0d4bc743eb7222b0ea4faf3fced052c80" Jul 06 01:22:55.300696 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.300680 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d6d090979962c7cbdfea6b12f745ae0d4bc743eb7222b0ea4faf3fced052c80"} err="failed to get container status \"5d6d090979962c7cbdfea6b12f745ae0d4bc743eb7222b0ea4faf3fced052c80\": rpc error: code = NotFound desc = could not find container \"5d6d090979962c7cbdfea6b12f745ae0d4bc743eb7222b0ea4faf3fced052c80\": container with ID starting with 5d6d090979962c7cbdfea6b12f745ae0d4bc743eb7222b0ea4faf3fced052c80 not found: ID does not exist" Jul 06 01:22:55.300735 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.300696 2565 scope.go:117] "RemoveContainer" containerID="64f901021a44f7d5134297d7e2968ad0cc63ff814ae2c097948bd81392bbd868" Jul 06 01:22:55.300870 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.300856 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64f901021a44f7d5134297d7e2968ad0cc63ff814ae2c097948bd81392bbd868"} err="failed to get container status \"64f901021a44f7d5134297d7e2968ad0cc63ff814ae2c097948bd81392bbd868\": rpc error: code = NotFound desc = could not find container \"64f901021a44f7d5134297d7e2968ad0cc63ff814ae2c097948bd81392bbd868\": container with ID starting with 64f901021a44f7d5134297d7e2968ad0cc63ff814ae2c097948bd81392bbd868 not found: ID does not exist" Jul 06 01:22:55.300911 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.300870 2565 scope.go:117] "RemoveContainer" containerID="608cde94a0548d7f5e181750409054de510aeb53254655cae7e7173048956e18" Jul 06 01:22:55.301091 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.301076 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"608cde94a0548d7f5e181750409054de510aeb53254655cae7e7173048956e18"} err="failed to get container status \"608cde94a0548d7f5e181750409054de510aeb53254655cae7e7173048956e18\": rpc error: code = NotFound desc = could not find container \"608cde94a0548d7f5e181750409054de510aeb53254655cae7e7173048956e18\": container with ID starting with 608cde94a0548d7f5e181750409054de510aeb53254655cae7e7173048956e18 not found: ID does not exist" Jul 06 01:22:55.301136 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.301091 2565 scope.go:117] "RemoveContainer" containerID="e69034755d0292d6c7808ef2126f934888ec8cf408afe632dd385d711a8042a4" Jul 06 01:22:55.301307 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.301289 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e69034755d0292d6c7808ef2126f934888ec8cf408afe632dd385d711a8042a4"} err="failed to get container status \"e69034755d0292d6c7808ef2126f934888ec8cf408afe632dd385d711a8042a4\": rpc error: code = NotFound desc = could not find container \"e69034755d0292d6c7808ef2126f934888ec8cf408afe632dd385d711a8042a4\": container with ID starting with e69034755d0292d6c7808ef2126f934888ec8cf408afe632dd385d711a8042a4 not found: ID does not exist" Jul 06 01:22:55.301367 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.301309 2565 scope.go:117] "RemoveContainer" containerID="8e35dea0111da30f93c5572e670a19b04a7796de3fe1a390659ebd87a47b4c44" Jul 06 01:22:55.301566 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.301549 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e35dea0111da30f93c5572e670a19b04a7796de3fe1a390659ebd87a47b4c44"} err="failed to get container status \"8e35dea0111da30f93c5572e670a19b04a7796de3fe1a390659ebd87a47b4c44\": rpc error: code = NotFound desc = could not find container \"8e35dea0111da30f93c5572e670a19b04a7796de3fe1a390659ebd87a47b4c44\": container with ID starting with 8e35dea0111da30f93c5572e670a19b04a7796de3fe1a390659ebd87a47b4c44 not found: ID does not exist" Jul 06 01:22:55.301618 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.301566 2565 scope.go:117] "RemoveContainer" containerID="f3aaad56290da5b1fb8a6fe3ade74d8c27450ed46845facd8b053c768be8de89" Jul 06 01:22:55.301756 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.301740 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3aaad56290da5b1fb8a6fe3ade74d8c27450ed46845facd8b053c768be8de89"} err="failed to get container status \"f3aaad56290da5b1fb8a6fe3ade74d8c27450ed46845facd8b053c768be8de89\": rpc error: code = NotFound desc = could not find container \"f3aaad56290da5b1fb8a6fe3ade74d8c27450ed46845facd8b053c768be8de89\": container with ID starting with f3aaad56290da5b1fb8a6fe3ade74d8c27450ed46845facd8b053c768be8de89 not found: ID does not exist" Jul 06 01:22:55.301794 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.301755 2565 scope.go:117] "RemoveContainer" containerID="7481585220cf802cc416c54b694536f1550363ba218d891257702cb267073a14" Jul 06 01:22:55.301940 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.301919 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7481585220cf802cc416c54b694536f1550363ba218d891257702cb267073a14"} err="failed to get container status \"7481585220cf802cc416c54b694536f1550363ba218d891257702cb267073a14\": rpc error: code = NotFound desc = could not find container \"7481585220cf802cc416c54b694536f1550363ba218d891257702cb267073a14\": container with ID starting with 7481585220cf802cc416c54b694536f1550363ba218d891257702cb267073a14 not found: ID does not exist" Jul 06 01:22:55.302011 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.301942 2565 scope.go:117] "RemoveContainer" containerID="5d6d090979962c7cbdfea6b12f745ae0d4bc743eb7222b0ea4faf3fced052c80" Jul 06 01:22:55.302159 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.302144 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d6d090979962c7cbdfea6b12f745ae0d4bc743eb7222b0ea4faf3fced052c80"} err="failed to get container status \"5d6d090979962c7cbdfea6b12f745ae0d4bc743eb7222b0ea4faf3fced052c80\": rpc error: code = NotFound desc = could not find container \"5d6d090979962c7cbdfea6b12f745ae0d4bc743eb7222b0ea4faf3fced052c80\": container with ID starting with 5d6d090979962c7cbdfea6b12f745ae0d4bc743eb7222b0ea4faf3fced052c80 not found: ID does not exist" Jul 06 01:22:55.302205 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.302159 2565 scope.go:117] "RemoveContainer" containerID="64f901021a44f7d5134297d7e2968ad0cc63ff814ae2c097948bd81392bbd868" Jul 06 01:22:55.302327 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.302311 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64f901021a44f7d5134297d7e2968ad0cc63ff814ae2c097948bd81392bbd868"} err="failed to get container status \"64f901021a44f7d5134297d7e2968ad0cc63ff814ae2c097948bd81392bbd868\": rpc error: code = NotFound desc = could not find container \"64f901021a44f7d5134297d7e2968ad0cc63ff814ae2c097948bd81392bbd868\": container with ID starting with 64f901021a44f7d5134297d7e2968ad0cc63ff814ae2c097948bd81392bbd868 not found: ID does not exist" Jul 06 01:22:55.302327 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.302327 2565 scope.go:117] "RemoveContainer" containerID="608cde94a0548d7f5e181750409054de510aeb53254655cae7e7173048956e18" Jul 06 01:22:55.302526 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.302510 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"608cde94a0548d7f5e181750409054de510aeb53254655cae7e7173048956e18"} err="failed to get container status \"608cde94a0548d7f5e181750409054de510aeb53254655cae7e7173048956e18\": rpc error: code = NotFound desc = could not find container \"608cde94a0548d7f5e181750409054de510aeb53254655cae7e7173048956e18\": container with ID starting with 608cde94a0548d7f5e181750409054de510aeb53254655cae7e7173048956e18 not found: ID does not exist" Jul 06 01:22:55.302573 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.302526 2565 scope.go:117] "RemoveContainer" containerID="e69034755d0292d6c7808ef2126f934888ec8cf408afe632dd385d711a8042a4" Jul 06 01:22:55.302699 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.302681 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e69034755d0292d6c7808ef2126f934888ec8cf408afe632dd385d711a8042a4"} err="failed to get container status \"e69034755d0292d6c7808ef2126f934888ec8cf408afe632dd385d711a8042a4\": rpc error: code = NotFound desc = could not find container \"e69034755d0292d6c7808ef2126f934888ec8cf408afe632dd385d711a8042a4\": container with ID starting with e69034755d0292d6c7808ef2126f934888ec8cf408afe632dd385d711a8042a4 not found: ID does not exist" Jul 06 01:22:55.302761 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.302700 2565 scope.go:117] "RemoveContainer" containerID="8e35dea0111da30f93c5572e670a19b04a7796de3fe1a390659ebd87a47b4c44" Jul 06 01:22:55.302927 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.302908 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e35dea0111da30f93c5572e670a19b04a7796de3fe1a390659ebd87a47b4c44"} err="failed to get container status \"8e35dea0111da30f93c5572e670a19b04a7796de3fe1a390659ebd87a47b4c44\": rpc error: code = NotFound desc = could not find container \"8e35dea0111da30f93c5572e670a19b04a7796de3fe1a390659ebd87a47b4c44\": container with ID starting with 8e35dea0111da30f93c5572e670a19b04a7796de3fe1a390659ebd87a47b4c44 not found: ID does not exist" Jul 06 01:22:55.302964 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.302928 2565 scope.go:117] "RemoveContainer" containerID="f3aaad56290da5b1fb8a6fe3ade74d8c27450ed46845facd8b053c768be8de89" Jul 06 01:22:55.303157 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.303140 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3aaad56290da5b1fb8a6fe3ade74d8c27450ed46845facd8b053c768be8de89"} err="failed to get container status \"f3aaad56290da5b1fb8a6fe3ade74d8c27450ed46845facd8b053c768be8de89\": rpc error: code = NotFound desc = could not find container \"f3aaad56290da5b1fb8a6fe3ade74d8c27450ed46845facd8b053c768be8de89\": container with ID starting with f3aaad56290da5b1fb8a6fe3ade74d8c27450ed46845facd8b053c768be8de89 not found: ID does not exist" Jul 06 01:22:55.303194 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.303159 2565 scope.go:117] "RemoveContainer" containerID="7481585220cf802cc416c54b694536f1550363ba218d891257702cb267073a14" Jul 06 01:22:55.303405 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.303359 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7481585220cf802cc416c54b694536f1550363ba218d891257702cb267073a14"} err="failed to get container status \"7481585220cf802cc416c54b694536f1550363ba218d891257702cb267073a14\": rpc error: code = NotFound desc = could not find container \"7481585220cf802cc416c54b694536f1550363ba218d891257702cb267073a14\": container with ID starting with 7481585220cf802cc416c54b694536f1550363ba218d891257702cb267073a14 not found: ID does not exist" Jul 06 01:22:55.303405 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.303406 2565 scope.go:117] "RemoveContainer" containerID="5d6d090979962c7cbdfea6b12f745ae0d4bc743eb7222b0ea4faf3fced052c80" Jul 06 01:22:55.303668 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.303647 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d6d090979962c7cbdfea6b12f745ae0d4bc743eb7222b0ea4faf3fced052c80"} err="failed to get container status \"5d6d090979962c7cbdfea6b12f745ae0d4bc743eb7222b0ea4faf3fced052c80\": rpc error: code = NotFound desc = could not find container \"5d6d090979962c7cbdfea6b12f745ae0d4bc743eb7222b0ea4faf3fced052c80\": container with ID starting with 5d6d090979962c7cbdfea6b12f745ae0d4bc743eb7222b0ea4faf3fced052c80 not found: ID does not exist" Jul 06 01:22:55.303668 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.303668 2565 scope.go:117] "RemoveContainer" containerID="64f901021a44f7d5134297d7e2968ad0cc63ff814ae2c097948bd81392bbd868" Jul 06 01:22:55.303876 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.303858 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64f901021a44f7d5134297d7e2968ad0cc63ff814ae2c097948bd81392bbd868"} err="failed to get container status \"64f901021a44f7d5134297d7e2968ad0cc63ff814ae2c097948bd81392bbd868\": rpc error: code = NotFound desc = could not find container \"64f901021a44f7d5134297d7e2968ad0cc63ff814ae2c097948bd81392bbd868\": container with ID starting with 64f901021a44f7d5134297d7e2968ad0cc63ff814ae2c097948bd81392bbd868 not found: ID does not exist" Jul 06 01:22:55.303918 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.303877 2565 scope.go:117] "RemoveContainer" containerID="608cde94a0548d7f5e181750409054de510aeb53254655cae7e7173048956e18" Jul 06 01:22:55.304092 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.304074 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"608cde94a0548d7f5e181750409054de510aeb53254655cae7e7173048956e18"} err="failed to get container status \"608cde94a0548d7f5e181750409054de510aeb53254655cae7e7173048956e18\": rpc error: code = NotFound desc = could not find container \"608cde94a0548d7f5e181750409054de510aeb53254655cae7e7173048956e18\": container with ID starting with 608cde94a0548d7f5e181750409054de510aeb53254655cae7e7173048956e18 not found: ID does not exist" Jul 06 01:22:55.304158 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.304093 2565 scope.go:117] "RemoveContainer" containerID="e69034755d0292d6c7808ef2126f934888ec8cf408afe632dd385d711a8042a4" Jul 06 01:22:55.304297 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.304281 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e69034755d0292d6c7808ef2126f934888ec8cf408afe632dd385d711a8042a4"} err="failed to get container status \"e69034755d0292d6c7808ef2126f934888ec8cf408afe632dd385d711a8042a4\": rpc error: code = NotFound desc = could not find container \"e69034755d0292d6c7808ef2126f934888ec8cf408afe632dd385d711a8042a4\": container with ID starting with e69034755d0292d6c7808ef2126f934888ec8cf408afe632dd385d711a8042a4 not found: ID does not exist" Jul 06 01:22:55.304340 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.304298 2565 scope.go:117] "RemoveContainer" containerID="8e35dea0111da30f93c5572e670a19b04a7796de3fe1a390659ebd87a47b4c44" Jul 06 01:22:55.304512 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.304490 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e35dea0111da30f93c5572e670a19b04a7796de3fe1a390659ebd87a47b4c44"} err="failed to get container status \"8e35dea0111da30f93c5572e670a19b04a7796de3fe1a390659ebd87a47b4c44\": rpc error: code = NotFound desc = could not find container \"8e35dea0111da30f93c5572e670a19b04a7796de3fe1a390659ebd87a47b4c44\": container with ID starting with 8e35dea0111da30f93c5572e670a19b04a7796de3fe1a390659ebd87a47b4c44 not found: ID does not exist" Jul 06 01:22:55.304571 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.304514 2565 scope.go:117] "RemoveContainer" containerID="f3aaad56290da5b1fb8a6fe3ade74d8c27450ed46845facd8b053c768be8de89" Jul 06 01:22:55.304719 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.304699 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3aaad56290da5b1fb8a6fe3ade74d8c27450ed46845facd8b053c768be8de89"} err="failed to get container status \"f3aaad56290da5b1fb8a6fe3ade74d8c27450ed46845facd8b053c768be8de89\": rpc error: code = NotFound desc = could not find container \"f3aaad56290da5b1fb8a6fe3ade74d8c27450ed46845facd8b053c768be8de89\": container with ID starting with f3aaad56290da5b1fb8a6fe3ade74d8c27450ed46845facd8b053c768be8de89 not found: ID does not exist" Jul 06 01:22:55.304779 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.304720 2565 scope.go:117] "RemoveContainer" containerID="7481585220cf802cc416c54b694536f1550363ba218d891257702cb267073a14" Jul 06 01:22:55.304942 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.304924 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7481585220cf802cc416c54b694536f1550363ba218d891257702cb267073a14"} err="failed to get container status \"7481585220cf802cc416c54b694536f1550363ba218d891257702cb267073a14\": rpc error: code = NotFound desc = could not find container \"7481585220cf802cc416c54b694536f1550363ba218d891257702cb267073a14\": container with ID starting with 7481585220cf802cc416c54b694536f1550363ba218d891257702cb267073a14 not found: ID does not exist" Jul 06 01:22:55.304987 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.304942 2565 scope.go:117] "RemoveContainer" containerID="5d6d090979962c7cbdfea6b12f745ae0d4bc743eb7222b0ea4faf3fced052c80" Jul 06 01:22:55.305157 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.305141 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d6d090979962c7cbdfea6b12f745ae0d4bc743eb7222b0ea4faf3fced052c80"} err="failed to get container status \"5d6d090979962c7cbdfea6b12f745ae0d4bc743eb7222b0ea4faf3fced052c80\": rpc error: code = NotFound desc = could not find container \"5d6d090979962c7cbdfea6b12f745ae0d4bc743eb7222b0ea4faf3fced052c80\": container with ID starting with 5d6d090979962c7cbdfea6b12f745ae0d4bc743eb7222b0ea4faf3fced052c80 not found: ID does not exist" Jul 06 01:22:55.305248 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.305156 2565 scope.go:117] "RemoveContainer" containerID="64f901021a44f7d5134297d7e2968ad0cc63ff814ae2c097948bd81392bbd868" Jul 06 01:22:55.305369 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.305353 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64f901021a44f7d5134297d7e2968ad0cc63ff814ae2c097948bd81392bbd868"} err="failed to get container status \"64f901021a44f7d5134297d7e2968ad0cc63ff814ae2c097948bd81392bbd868\": rpc error: code = NotFound desc = could not find container \"64f901021a44f7d5134297d7e2968ad0cc63ff814ae2c097948bd81392bbd868\": container with ID starting with 64f901021a44f7d5134297d7e2968ad0cc63ff814ae2c097948bd81392bbd868 not found: ID does not exist" Jul 06 01:22:55.305369 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.305368 2565 scope.go:117] "RemoveContainer" containerID="608cde94a0548d7f5e181750409054de510aeb53254655cae7e7173048956e18" Jul 06 01:22:55.305599 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.305584 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"608cde94a0548d7f5e181750409054de510aeb53254655cae7e7173048956e18"} err="failed to get container status \"608cde94a0548d7f5e181750409054de510aeb53254655cae7e7173048956e18\": rpc error: code = NotFound desc = could not find container \"608cde94a0548d7f5e181750409054de510aeb53254655cae7e7173048956e18\": container with ID starting with 608cde94a0548d7f5e181750409054de510aeb53254655cae7e7173048956e18 not found: ID does not exist" Jul 06 01:22:55.305650 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.305599 2565 scope.go:117] "RemoveContainer" containerID="e69034755d0292d6c7808ef2126f934888ec8cf408afe632dd385d711a8042a4" Jul 06 01:22:55.305768 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.305753 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e69034755d0292d6c7808ef2126f934888ec8cf408afe632dd385d711a8042a4"} err="failed to get container status \"e69034755d0292d6c7808ef2126f934888ec8cf408afe632dd385d711a8042a4\": rpc error: code = NotFound desc = could not find container \"e69034755d0292d6c7808ef2126f934888ec8cf408afe632dd385d711a8042a4\": container with ID starting with e69034755d0292d6c7808ef2126f934888ec8cf408afe632dd385d711a8042a4 not found: ID does not exist" Jul 06 01:22:55.305825 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.305768 2565 scope.go:117] "RemoveContainer" containerID="8e35dea0111da30f93c5572e670a19b04a7796de3fe1a390659ebd87a47b4c44" Jul 06 01:22:55.305970 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.305948 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e35dea0111da30f93c5572e670a19b04a7796de3fe1a390659ebd87a47b4c44"} err="failed to get container status \"8e35dea0111da30f93c5572e670a19b04a7796de3fe1a390659ebd87a47b4c44\": rpc error: code = NotFound desc = could not find container \"8e35dea0111da30f93c5572e670a19b04a7796de3fe1a390659ebd87a47b4c44\": container with ID starting with 8e35dea0111da30f93c5572e670a19b04a7796de3fe1a390659ebd87a47b4c44 not found: ID does not exist" Jul 06 01:22:55.306012 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.305974 2565 scope.go:117] "RemoveContainer" containerID="f3aaad56290da5b1fb8a6fe3ade74d8c27450ed46845facd8b053c768be8de89" Jul 06 01:22:55.306205 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.306189 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3aaad56290da5b1fb8a6fe3ade74d8c27450ed46845facd8b053c768be8de89"} err="failed to get container status \"f3aaad56290da5b1fb8a6fe3ade74d8c27450ed46845facd8b053c768be8de89\": rpc error: code = NotFound desc = could not find container \"f3aaad56290da5b1fb8a6fe3ade74d8c27450ed46845facd8b053c768be8de89\": container with ID starting with f3aaad56290da5b1fb8a6fe3ade74d8c27450ed46845facd8b053c768be8de89 not found: ID does not exist" Jul 06 01:22:55.334065 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.334022 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.334065 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.334068 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-config-out\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.334301 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.334104 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.334301 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.334162 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.334301 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.334237 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-web-config\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.334301 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.334275 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.334301 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.334292 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.334531 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.334310 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.334531 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.334340 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-config\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.334531 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.334401 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.334531 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.334449 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.334531 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.334474 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.334531 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.334490 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.334531 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.334516 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkzkw\" (UniqueName: \"kubernetes.io/projected/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-kube-api-access-zkzkw\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.334747 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.334540 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.334747 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.334556 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.334747 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.334580 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.334747 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.334633 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.334747 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.334664 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.334747 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.334686 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.435648 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.435610 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-web-config\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.435648 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.435655 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.435885 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.435671 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.435885 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.435699 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.435885 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.435727 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-config\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.435885 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.435752 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.435885 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.435776 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.435885 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.435809 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.435885 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.435834 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.435885 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.435859 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkzkw\" (UniqueName: \"kubernetes.io/projected/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-kube-api-access-zkzkw\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.435885 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.435885 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.436370 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.435908 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.436370 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.435931 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.436370 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.435964 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.436370 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.435988 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.436370 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.436012 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.436370 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.436051 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.436370 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.436078 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-config-out\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.436370 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.436116 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.436370 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.436143 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.436370 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.436425 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.436966 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.436801 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.437027 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.436962 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.439242 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.439206 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.439400 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.439269 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-config\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.439521 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.439497 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-web-config\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.439604 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.439512 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.439957 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.439931 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.440070 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.440047 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.440129 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.440107 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.440571 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.440547 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.440708 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.440580 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.441007 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.440789 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.441007 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.440844 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.441007 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.440988 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.441862 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.441841 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-config-out\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.442790 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.442774 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.443002 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.442983 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.443156 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.443140 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.448345 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.448318 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkzkw\" (UniqueName: \"kubernetes.io/projected/0b46c140-08b2-4581-8f42-3ddc1dcb8f6c-kube-api-access-zkzkw\") pod \"prometheus-k8s-0\" (UID: \"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.581366 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.581322 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:22:55.727579 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:55.727548 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 01:22:55.730548 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:22:55.730509 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b46c140_08b2_4581_8f42_3ddc1dcb8f6c.slice/crio-cbe9127b35ab20b9a5ad2130dd5ce2316c666fffdebe3f50db3a6027d563643a WatchSource:0}: Error finding container cbe9127b35ab20b9a5ad2130dd5ce2316c666fffdebe3f50db3a6027d563643a: Status 404 returned error can't find the container with id cbe9127b35ab20b9a5ad2130dd5ce2316c666fffdebe3f50db3a6027d563643a Jul 06 01:22:56.213091 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:56.213049 2565 generic.go:356] "Generic (PLEG): container finished" podID="0b46c140-08b2-4581-8f42-3ddc1dcb8f6c" containerID="c33f242007f595abd19fe179fe2c282e58028d98038241b6e75ed4b7baab970c" exitCode=0 Jul 06 01:22:56.213541 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:56.213135 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c","Type":"ContainerDied","Data":"c33f242007f595abd19fe179fe2c282e58028d98038241b6e75ed4b7baab970c"} Jul 06 01:22:56.213541 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:56.213201 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c","Type":"ContainerStarted","Data":"cbe9127b35ab20b9a5ad2130dd5ce2316c666fffdebe3f50db3a6027d563643a"} Jul 06 01:22:56.323620 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:56.323582 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5cf93c2-af80-4384-add5-49cdd601b40c" path="/var/lib/kubelet/pods/f5cf93c2-af80-4384-add5-49cdd601b40c/volumes" Jul 06 01:22:57.221359 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:57.221314 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c","Type":"ContainerStarted","Data":"e5cd9c7a4f334cfe5d4576aac4cfe735767ed261653a59cde6b3f49d4f6f6a62"} Jul 06 01:22:57.221865 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:57.221429 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c","Type":"ContainerStarted","Data":"25ffb4cd614d6379ba2318fff893bf1d56f5925779e4808dd7471b9f441a9c4a"} Jul 06 01:22:57.221865 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:57.221467 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c","Type":"ContainerStarted","Data":"c58fae6672f130641c9e9beefc9f6b4d5457e88a16bfd4873971141011c0d795"} Jul 06 01:22:57.221865 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:57.221492 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c","Type":"ContainerStarted","Data":"e0836265fc09c7f556afefda62e641b935413031d7afb9ca6521d06eeea42a62"} Jul 06 01:22:57.221865 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:57.221527 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c","Type":"ContainerStarted","Data":"a49984e617a5997a6b878d31e2b05d8abc3337cb87bb47290cdd73d42f2f90be"} Jul 06 01:22:57.221865 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:57.221554 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0b46c140-08b2-4581-8f42-3ddc1dcb8f6c","Type":"ContainerStarted","Data":"17fe7e8c3fb9f68ef1eb1694bd5329ac8e15722918ba71330b40ebd00ed7af98"} Jul 06 01:22:57.248339 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:22:57.248245 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.228957641 podStartE2EDuration="2.24822445s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.266809ms" isStatefulPod=false podCreationTimestamp="2026-07-06 01:22:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:22:57.247218077 +0000 UTC m=+149.388776882" watchObservedRunningTime="2026-07-06 01:22:57.24822445 +0000 UTC m=+149.389783238" Jul 06 01:23:00.581791 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:23:00.581744 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:23:55.581717 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:23:55.581678 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:23:55.598423 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:23:55.598369 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:23:56.486223 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:23:56.486194 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 01:25:28.236961 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:25:28.236936 2565 kubelet.go:1648] "Image garbage collection succeeded" Jul 06 01:25:39.525872 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:25:39.525830 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-dttvj"] Jul 06 01:25:39.529125 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:25:39.529106 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-dttvj" Jul 06 01:25:39.531540 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:25:39.531519 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 06 01:25:39.531540 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:25:39.531530 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-sgszl\"" Jul 06 01:25:39.532355 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:25:39.532339 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 06 01:25:39.539470 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:25:39.539443 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-dttvj"] Jul 06 01:25:39.557287 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:25:39.557255 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/af0ff745-7e92-43e1-b528-4546c2576238-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-dttvj\" (UID: \"af0ff745-7e92-43e1-b528-4546c2576238\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dttvj" Jul 06 01:25:39.557487 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:25:39.557314 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfmm4\" (UniqueName: \"kubernetes.io/projected/af0ff745-7e92-43e1-b528-4546c2576238-kube-api-access-nfmm4\") pod \"cert-manager-cainjector-559cdd8c87-dttvj\" (UID: \"af0ff745-7e92-43e1-b528-4546c2576238\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dttvj" Jul 06 01:25:39.658501 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:25:39.658449 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfmm4\" (UniqueName: \"kubernetes.io/projected/af0ff745-7e92-43e1-b528-4546c2576238-kube-api-access-nfmm4\") pod \"cert-manager-cainjector-559cdd8c87-dttvj\" (UID: \"af0ff745-7e92-43e1-b528-4546c2576238\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dttvj" Jul 06 01:25:39.658707 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:25:39.658554 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/af0ff745-7e92-43e1-b528-4546c2576238-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-dttvj\" (UID: \"af0ff745-7e92-43e1-b528-4546c2576238\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dttvj" Jul 06 01:25:39.666958 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:25:39.666932 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/af0ff745-7e92-43e1-b528-4546c2576238-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-dttvj\" (UID: \"af0ff745-7e92-43e1-b528-4546c2576238\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dttvj" Jul 06 01:25:39.667128 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:25:39.667107 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfmm4\" (UniqueName: \"kubernetes.io/projected/af0ff745-7e92-43e1-b528-4546c2576238-kube-api-access-nfmm4\") pod \"cert-manager-cainjector-559cdd8c87-dttvj\" (UID: \"af0ff745-7e92-43e1-b528-4546c2576238\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dttvj" Jul 06 01:25:39.849747 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:25:39.849661 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-dttvj" Jul 06 01:25:39.986805 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:25:39.986773 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-dttvj"] Jul 06 01:25:39.988686 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:25:39.988651 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaf0ff745_7e92_43e1_b528_4546c2576238.slice/crio-04ad65c3136e12644abf7adc2d6319910888ad5f317a008ed96c586b3d46ffe5 WatchSource:0}: Error finding container 04ad65c3136e12644abf7adc2d6319910888ad5f317a008ed96c586b3d46ffe5: Status 404 returned error can't find the container with id 04ad65c3136e12644abf7adc2d6319910888ad5f317a008ed96c586b3d46ffe5 Jul 06 01:25:39.990757 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:25:39.990736 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 01:25:40.892360 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:25:40.892303 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-dttvj" event={"ID":"af0ff745-7e92-43e1-b528-4546c2576238","Type":"ContainerStarted","Data":"04ad65c3136e12644abf7adc2d6319910888ad5f317a008ed96c586b3d46ffe5"} Jul 06 01:25:43.907064 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:25:43.907010 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-dttvj" event={"ID":"af0ff745-7e92-43e1-b528-4546c2576238","Type":"ContainerStarted","Data":"ee9b0a3f4a766633c5f6080c82e78cc29e8c0efed908f7f9d9afc2c8be727c9e"} Jul 06 01:25:43.921271 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:25:43.921196 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-dttvj" podStartSLOduration=1.651494224 podStartE2EDuration="4.921175597s" totalImagesPullingTime="3.269681373s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:25:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:25:43.921059886 +0000 UTC m=+316.062618673" watchObservedRunningTime="2026-07-06 01:25:43.921175597 +0000 UTC m=+316.062734384" Jul 06 01:25:48.931748 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:25:48.929241 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cqkv2"] Jul 06 01:25:48.933863 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:25:48.933832 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cqkv2" Jul 06 01:25:48.936454 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:25:48.936431 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 06 01:25:48.936454 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:25:48.936431 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 06 01:25:48.937289 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:25:48.937268 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-rcxdp\"" Jul 06 01:25:48.941915 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:25:48.941890 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cqkv2"] Jul 06 01:25:49.036081 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:25:49.036041 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d3ad6ae1-7d23-43fc-9617-e7e084891b69-tmp\") pod \"openshift-lws-operator-fd8ccff4c-cqkv2\" (UID: \"d3ad6ae1-7d23-43fc-9617-e7e084891b69\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cqkv2" Jul 06 01:25:49.036261 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:25:49.036116 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-949lg\" (UniqueName: \"kubernetes.io/projected/d3ad6ae1-7d23-43fc-9617-e7e084891b69-kube-api-access-949lg\") pod \"openshift-lws-operator-fd8ccff4c-cqkv2\" (UID: \"d3ad6ae1-7d23-43fc-9617-e7e084891b69\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cqkv2" Jul 06 01:25:49.136674 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:25:49.136630 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d3ad6ae1-7d23-43fc-9617-e7e084891b69-tmp\") pod \"openshift-lws-operator-fd8ccff4c-cqkv2\" (UID: \"d3ad6ae1-7d23-43fc-9617-e7e084891b69\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cqkv2" Jul 06 01:25:49.136856 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:25:49.136699 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-949lg\" (UniqueName: \"kubernetes.io/projected/d3ad6ae1-7d23-43fc-9617-e7e084891b69-kube-api-access-949lg\") pod \"openshift-lws-operator-fd8ccff4c-cqkv2\" (UID: \"d3ad6ae1-7d23-43fc-9617-e7e084891b69\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cqkv2" Jul 06 01:25:49.137028 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:25:49.137007 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d3ad6ae1-7d23-43fc-9617-e7e084891b69-tmp\") pod \"openshift-lws-operator-fd8ccff4c-cqkv2\" (UID: \"d3ad6ae1-7d23-43fc-9617-e7e084891b69\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cqkv2" Jul 06 01:25:49.144951 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:25:49.144926 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-949lg\" (UniqueName: \"kubernetes.io/projected/d3ad6ae1-7d23-43fc-9617-e7e084891b69-kube-api-access-949lg\") pod \"openshift-lws-operator-fd8ccff4c-cqkv2\" (UID: \"d3ad6ae1-7d23-43fc-9617-e7e084891b69\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cqkv2" Jul 06 01:25:49.243980 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:25:49.243941 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cqkv2" Jul 06 01:25:49.382823 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:25:49.382723 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cqkv2"] Jul 06 01:25:49.385752 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:25:49.385721 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd3ad6ae1_7d23_43fc_9617_e7e084891b69.slice/crio-55e02f102f91af2c89786b1abff441de20ef15ddd2997d69b47f9cfc7e9a8baf WatchSource:0}: Error finding container 55e02f102f91af2c89786b1abff441de20ef15ddd2997d69b47f9cfc7e9a8baf: Status 404 returned error can't find the container with id 55e02f102f91af2c89786b1abff441de20ef15ddd2997d69b47f9cfc7e9a8baf Jul 06 01:25:49.935005 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:25:49.934972 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cqkv2" event={"ID":"d3ad6ae1-7d23-43fc-9617-e7e084891b69","Type":"ContainerStarted","Data":"55e02f102f91af2c89786b1abff441de20ef15ddd2997d69b47f9cfc7e9a8baf"} Jul 06 01:25:52.951511 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:25:52.951475 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cqkv2" event={"ID":"d3ad6ae1-7d23-43fc-9617-e7e084891b69","Type":"ContainerStarted","Data":"17d2db4c2503aba46507ea116b80b208bd2d184b9a65f81af484c4499e96a1b5"} Jul 06 01:25:52.968322 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:25:52.968261 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cqkv2" podStartSLOduration=2.2570194519999998 podStartE2EDuration="4.968242543s" totalImagesPullingTime="2.711223091s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:25:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:25:52.966100718 +0000 UTC m=+325.107659525" watchObservedRunningTime="2026-07-06 01:25:52.968242543 +0000 UTC m=+325.109801328" Jul 06 01:26:05.859719 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:05.859688 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-76dc9d4b5b-wnpn9"] Jul 06 01:26:05.863162 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:05.863145 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-76dc9d4b5b-wnpn9" Jul 06 01:26:05.865757 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:05.865729 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 06 01:26:05.866777 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:05.866755 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 06 01:26:05.866777 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:05.866782 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 06 01:26:05.866974 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:05.866824 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-5x8h4\"" Jul 06 01:26:05.872698 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:05.872670 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-76dc9d4b5b-wnpn9"] Jul 06 01:26:05.987040 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:05.987008 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kt6lk\" (UniqueName: \"kubernetes.io/projected/6849fc29-d12d-4792-95d9-1dca00e22b72-kube-api-access-kt6lk\") pod \"lws-controller-manager-76dc9d4b5b-wnpn9\" (UID: \"6849fc29-d12d-4792-95d9-1dca00e22b72\") " pod="openshift-lws-operator/lws-controller-manager-76dc9d4b5b-wnpn9" Jul 06 01:26:05.987040 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:05.987053 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6849fc29-d12d-4792-95d9-1dca00e22b72-manager-config\") pod \"lws-controller-manager-76dc9d4b5b-wnpn9\" (UID: \"6849fc29-d12d-4792-95d9-1dca00e22b72\") " pod="openshift-lws-operator/lws-controller-manager-76dc9d4b5b-wnpn9" Jul 06 01:26:05.987300 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:05.987079 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6849fc29-d12d-4792-95d9-1dca00e22b72-cert\") pod \"lws-controller-manager-76dc9d4b5b-wnpn9\" (UID: \"6849fc29-d12d-4792-95d9-1dca00e22b72\") " pod="openshift-lws-operator/lws-controller-manager-76dc9d4b5b-wnpn9" Jul 06 01:26:05.987300 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:05.987165 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6849fc29-d12d-4792-95d9-1dca00e22b72-metrics-cert\") pod \"lws-controller-manager-76dc9d4b5b-wnpn9\" (UID: \"6849fc29-d12d-4792-95d9-1dca00e22b72\") " pod="openshift-lws-operator/lws-controller-manager-76dc9d4b5b-wnpn9" Jul 06 01:26:06.087978 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:06.087939 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6849fc29-d12d-4792-95d9-1dca00e22b72-metrics-cert\") pod \"lws-controller-manager-76dc9d4b5b-wnpn9\" (UID: \"6849fc29-d12d-4792-95d9-1dca00e22b72\") " pod="openshift-lws-operator/lws-controller-manager-76dc9d4b5b-wnpn9" Jul 06 01:26:06.088197 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:06.088053 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kt6lk\" (UniqueName: \"kubernetes.io/projected/6849fc29-d12d-4792-95d9-1dca00e22b72-kube-api-access-kt6lk\") pod \"lws-controller-manager-76dc9d4b5b-wnpn9\" (UID: \"6849fc29-d12d-4792-95d9-1dca00e22b72\") " pod="openshift-lws-operator/lws-controller-manager-76dc9d4b5b-wnpn9" Jul 06 01:26:06.088197 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:06.088104 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6849fc29-d12d-4792-95d9-1dca00e22b72-manager-config\") pod \"lws-controller-manager-76dc9d4b5b-wnpn9\" (UID: \"6849fc29-d12d-4792-95d9-1dca00e22b72\") " pod="openshift-lws-operator/lws-controller-manager-76dc9d4b5b-wnpn9" Jul 06 01:26:06.088197 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:06.088125 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6849fc29-d12d-4792-95d9-1dca00e22b72-cert\") pod \"lws-controller-manager-76dc9d4b5b-wnpn9\" (UID: \"6849fc29-d12d-4792-95d9-1dca00e22b72\") " pod="openshift-lws-operator/lws-controller-manager-76dc9d4b5b-wnpn9" Jul 06 01:26:06.088822 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:06.088796 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6849fc29-d12d-4792-95d9-1dca00e22b72-manager-config\") pod \"lws-controller-manager-76dc9d4b5b-wnpn9\" (UID: \"6849fc29-d12d-4792-95d9-1dca00e22b72\") " pod="openshift-lws-operator/lws-controller-manager-76dc9d4b5b-wnpn9" Jul 06 01:26:06.090843 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:06.090818 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6849fc29-d12d-4792-95d9-1dca00e22b72-metrics-cert\") pod \"lws-controller-manager-76dc9d4b5b-wnpn9\" (UID: \"6849fc29-d12d-4792-95d9-1dca00e22b72\") " pod="openshift-lws-operator/lws-controller-manager-76dc9d4b5b-wnpn9" Jul 06 01:26:06.090934 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:06.090862 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6849fc29-d12d-4792-95d9-1dca00e22b72-cert\") pod \"lws-controller-manager-76dc9d4b5b-wnpn9\" (UID: \"6849fc29-d12d-4792-95d9-1dca00e22b72\") " pod="openshift-lws-operator/lws-controller-manager-76dc9d4b5b-wnpn9" Jul 06 01:26:06.098554 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:06.098532 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kt6lk\" (UniqueName: \"kubernetes.io/projected/6849fc29-d12d-4792-95d9-1dca00e22b72-kube-api-access-kt6lk\") pod \"lws-controller-manager-76dc9d4b5b-wnpn9\" (UID: \"6849fc29-d12d-4792-95d9-1dca00e22b72\") " pod="openshift-lws-operator/lws-controller-manager-76dc9d4b5b-wnpn9" Jul 06 01:26:06.173787 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:06.173697 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-76dc9d4b5b-wnpn9" Jul 06 01:26:06.333326 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:06.333301 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-76dc9d4b5b-wnpn9"] Jul 06 01:26:06.335808 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:26:06.335778 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6849fc29_d12d_4792_95d9_1dca00e22b72.slice/crio-2d60ce1c3543014b612a4ff598637f47959aff4ed70948d0581961b12cff00bb WatchSource:0}: Error finding container 2d60ce1c3543014b612a4ff598637f47959aff4ed70948d0581961b12cff00bb: Status 404 returned error can't find the container with id 2d60ce1c3543014b612a4ff598637f47959aff4ed70948d0581961b12cff00bb Jul 06 01:26:07.013807 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:07.013770 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-76dc9d4b5b-wnpn9" event={"ID":"6849fc29-d12d-4792-95d9-1dca00e22b72","Type":"ContainerStarted","Data":"2d60ce1c3543014b612a4ff598637f47959aff4ed70948d0581961b12cff00bb"} Jul 06 01:26:09.024473 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:09.024437 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-76dc9d4b5b-wnpn9" event={"ID":"6849fc29-d12d-4792-95d9-1dca00e22b72","Type":"ContainerStarted","Data":"f68bcfc257cb8bdda699e1127cce2c07556d4253fd634e9283100a38796e52bf"} Jul 06 01:26:09.024923 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:09.024533 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-76dc9d4b5b-wnpn9" Jul 06 01:26:09.041709 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:09.041657 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-76dc9d4b5b-wnpn9" podStartSLOduration=2.172642048 podStartE2EDuration="4.04164003s" totalImagesPullingTime="1.868997982s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:26:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:26:09.040658769 +0000 UTC m=+341.182217555" watchObservedRunningTime="2026-07-06 01:26:09.04164003 +0000 UTC m=+341.183198936" Jul 06 01:26:09.280413 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:09.280319 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d8fbf9755-s7rjg"] Jul 06 01:26:09.283820 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:09.283801 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-s7rjg" Jul 06 01:26:09.287490 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:09.287457 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 06 01:26:09.288001 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:09.287977 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 06 01:26:09.288001 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:09.287991 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-mxjmq\"" Jul 06 01:26:09.288144 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:09.288022 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 06 01:26:09.288532 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:09.288515 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 06 01:26:09.365122 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:09.365086 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d8fbf9755-s7rjg"] Jul 06 01:26:09.418907 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:09.418875 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/787b65e1-3d80-4453-85e3-f4489d16b595-tmp\") pod \"opendatahub-operator-controller-manager-d8fbf9755-s7rjg\" (UID: \"787b65e1-3d80-4453-85e3-f4489d16b595\") " pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-s7rjg" Jul 06 01:26:09.418907 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:09.418913 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/787b65e1-3d80-4453-85e3-f4489d16b595-manifests\") pod \"opendatahub-operator-controller-manager-d8fbf9755-s7rjg\" (UID: \"787b65e1-3d80-4453-85e3-f4489d16b595\") " pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-s7rjg" Jul 06 01:26:09.419147 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:09.418990 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/787b65e1-3d80-4453-85e3-f4489d16b595-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d8fbf9755-s7rjg\" (UID: \"787b65e1-3d80-4453-85e3-f4489d16b595\") " pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-s7rjg" Jul 06 01:26:09.419147 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:09.419021 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msft7\" (UniqueName: \"kubernetes.io/projected/787b65e1-3d80-4453-85e3-f4489d16b595-kube-api-access-msft7\") pod \"opendatahub-operator-controller-manager-d8fbf9755-s7rjg\" (UID: \"787b65e1-3d80-4453-85e3-f4489d16b595\") " pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-s7rjg" Jul 06 01:26:09.419147 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:09.419050 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/787b65e1-3d80-4453-85e3-f4489d16b595-webhook-cert\") pod \"opendatahub-operator-controller-manager-d8fbf9755-s7rjg\" (UID: \"787b65e1-3d80-4453-85e3-f4489d16b595\") " pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-s7rjg" Jul 06 01:26:09.519711 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:09.519669 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/787b65e1-3d80-4453-85e3-f4489d16b595-tmp\") pod \"opendatahub-operator-controller-manager-d8fbf9755-s7rjg\" (UID: \"787b65e1-3d80-4453-85e3-f4489d16b595\") " pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-s7rjg" Jul 06 01:26:09.519922 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:09.519741 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/787b65e1-3d80-4453-85e3-f4489d16b595-manifests\") pod \"opendatahub-operator-controller-manager-d8fbf9755-s7rjg\" (UID: \"787b65e1-3d80-4453-85e3-f4489d16b595\") " pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-s7rjg" Jul 06 01:26:09.519922 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:09.519805 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/787b65e1-3d80-4453-85e3-f4489d16b595-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d8fbf9755-s7rjg\" (UID: \"787b65e1-3d80-4453-85e3-f4489d16b595\") " pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-s7rjg" Jul 06 01:26:09.519922 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:09.519829 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msft7\" (UniqueName: \"kubernetes.io/projected/787b65e1-3d80-4453-85e3-f4489d16b595-kube-api-access-msft7\") pod \"opendatahub-operator-controller-manager-d8fbf9755-s7rjg\" (UID: \"787b65e1-3d80-4453-85e3-f4489d16b595\") " pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-s7rjg" Jul 06 01:26:09.519922 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:09.519864 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/787b65e1-3d80-4453-85e3-f4489d16b595-webhook-cert\") pod \"opendatahub-operator-controller-manager-d8fbf9755-s7rjg\" (UID: \"787b65e1-3d80-4453-85e3-f4489d16b595\") " pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-s7rjg" Jul 06 01:26:09.520176 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:09.520144 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/787b65e1-3d80-4453-85e3-f4489d16b595-manifests\") pod \"opendatahub-operator-controller-manager-d8fbf9755-s7rjg\" (UID: \"787b65e1-3d80-4453-85e3-f4489d16b595\") " pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-s7rjg" Jul 06 01:26:09.522295 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:09.522255 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/787b65e1-3d80-4453-85e3-f4489d16b595-tmp\") pod \"opendatahub-operator-controller-manager-d8fbf9755-s7rjg\" (UID: \"787b65e1-3d80-4453-85e3-f4489d16b595\") " pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-s7rjg" Jul 06 01:26:09.522457 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:09.522304 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/787b65e1-3d80-4453-85e3-f4489d16b595-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d8fbf9755-s7rjg\" (UID: \"787b65e1-3d80-4453-85e3-f4489d16b595\") " pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-s7rjg" Jul 06 01:26:09.522457 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:09.522392 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/787b65e1-3d80-4453-85e3-f4489d16b595-webhook-cert\") pod \"opendatahub-operator-controller-manager-d8fbf9755-s7rjg\" (UID: \"787b65e1-3d80-4453-85e3-f4489d16b595\") " pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-s7rjg" Jul 06 01:26:09.536013 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:09.535937 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-msft7\" (UniqueName: \"kubernetes.io/projected/787b65e1-3d80-4453-85e3-f4489d16b595-kube-api-access-msft7\") pod \"opendatahub-operator-controller-manager-d8fbf9755-s7rjg\" (UID: \"787b65e1-3d80-4453-85e3-f4489d16b595\") " pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-s7rjg" Jul 06 01:26:09.593930 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:09.593886 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-s7rjg" Jul 06 01:26:09.750736 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:09.750709 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d8fbf9755-s7rjg"] Jul 06 01:26:09.753512 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:26:09.753483 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod787b65e1_3d80_4453_85e3_f4489d16b595.slice/crio-b733b075e356b3594f5df1081b1bdd7367366b951ac0535b42e8eae66887a76f WatchSource:0}: Error finding container b733b075e356b3594f5df1081b1bdd7367366b951ac0535b42e8eae66887a76f: Status 404 returned error can't find the container with id b733b075e356b3594f5df1081b1bdd7367366b951ac0535b42e8eae66887a76f Jul 06 01:26:10.030601 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:10.030567 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-s7rjg" event={"ID":"787b65e1-3d80-4453-85e3-f4489d16b595","Type":"ContainerStarted","Data":"b733b075e356b3594f5df1081b1bdd7367366b951ac0535b42e8eae66887a76f"} Jul 06 01:26:13.051322 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:13.051281 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-s7rjg" event={"ID":"787b65e1-3d80-4453-85e3-f4489d16b595","Type":"ContainerStarted","Data":"9dc0a6987cfa9239f287a376f725f62f430bb5538769e5976d9e1822d574edfd"} Jul 06 01:26:14.057941 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:14.057902 2565 generic.go:356] "Generic (PLEG): container finished" podID="787b65e1-3d80-4453-85e3-f4489d16b595" containerID="9dc0a6987cfa9239f287a376f725f62f430bb5538769e5976d9e1822d574edfd" exitCode=0 Jul 06 01:26:14.058341 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:14.057987 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-s7rjg" event={"ID":"787b65e1-3d80-4453-85e3-f4489d16b595","Type":"ContainerDied","Data":"9dc0a6987cfa9239f287a376f725f62f430bb5538769e5976d9e1822d574edfd"} Jul 06 01:26:15.065368 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:15.065333 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-s7rjg" event={"ID":"787b65e1-3d80-4453-85e3-f4489d16b595","Type":"ContainerStarted","Data":"52298ddf16c0150bba7757938c438f940893fe399bbc227ae943fecab07b256d"} Jul 06 01:26:15.065793 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:15.065628 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-s7rjg" Jul 06 01:26:15.086335 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:15.086275 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-s7rjg" podStartSLOduration=1.771706906 podStartE2EDuration="6.086257602s" totalImagesPullingTime="3.243564388s" totalInitContainerRuntime="1.070986308s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:26:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:26:15.083680123 +0000 UTC m=+347.225238921" watchObservedRunningTime="2026-07-06 01:26:15.086257602 +0000 UTC m=+347.227816388" Jul 06 01:26:20.034455 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:20.034349 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-76dc9d4b5b-wnpn9" Jul 06 01:26:26.075569 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:26.075407 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-d8fbf9755-s7rjg" Jul 06 01:26:29.453993 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:29.453940 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-799868c9c7-h75ft"] Jul 06 01:26:29.460510 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:29.460477 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-799868c9c7-h75ft" Jul 06 01:26:29.463944 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:29.463914 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 06 01:26:29.464095 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:29.463920 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 06 01:26:29.464844 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:29.464821 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 06 01:26:29.464844 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:29.464839 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 06 01:26:29.465036 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:29.464993 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-hdcln\"" Jul 06 01:26:29.471767 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:29.471743 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-799868c9c7-h75ft"] Jul 06 01:26:29.504884 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:29.504844 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/979609f9-f877-47b2-a3b7-17d491a53d23-tmp\") pod \"kube-auth-proxy-799868c9c7-h75ft\" (UID: \"979609f9-f877-47b2-a3b7-17d491a53d23\") " pod="openshift-ingress/kube-auth-proxy-799868c9c7-h75ft" Jul 06 01:26:29.505056 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:29.504890 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97qdj\" (UniqueName: \"kubernetes.io/projected/979609f9-f877-47b2-a3b7-17d491a53d23-kube-api-access-97qdj\") pod \"kube-auth-proxy-799868c9c7-h75ft\" (UID: \"979609f9-f877-47b2-a3b7-17d491a53d23\") " pod="openshift-ingress/kube-auth-proxy-799868c9c7-h75ft" Jul 06 01:26:29.505056 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:29.504979 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/979609f9-f877-47b2-a3b7-17d491a53d23-tls-certs\") pod \"kube-auth-proxy-799868c9c7-h75ft\" (UID: \"979609f9-f877-47b2-a3b7-17d491a53d23\") " pod="openshift-ingress/kube-auth-proxy-799868c9c7-h75ft" Jul 06 01:26:29.606306 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:29.606266 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/979609f9-f877-47b2-a3b7-17d491a53d23-tmp\") pod \"kube-auth-proxy-799868c9c7-h75ft\" (UID: \"979609f9-f877-47b2-a3b7-17d491a53d23\") " pod="openshift-ingress/kube-auth-proxy-799868c9c7-h75ft" Jul 06 01:26:29.606512 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:29.606332 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97qdj\" (UniqueName: \"kubernetes.io/projected/979609f9-f877-47b2-a3b7-17d491a53d23-kube-api-access-97qdj\") pod \"kube-auth-proxy-799868c9c7-h75ft\" (UID: \"979609f9-f877-47b2-a3b7-17d491a53d23\") " pod="openshift-ingress/kube-auth-proxy-799868c9c7-h75ft" Jul 06 01:26:29.606512 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:29.606431 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/979609f9-f877-47b2-a3b7-17d491a53d23-tls-certs\") pod \"kube-auth-proxy-799868c9c7-h75ft\" (UID: \"979609f9-f877-47b2-a3b7-17d491a53d23\") " pod="openshift-ingress/kube-auth-proxy-799868c9c7-h75ft" Jul 06 01:26:29.608942 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:29.608914 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/979609f9-f877-47b2-a3b7-17d491a53d23-tmp\") pod \"kube-auth-proxy-799868c9c7-h75ft\" (UID: \"979609f9-f877-47b2-a3b7-17d491a53d23\") " pod="openshift-ingress/kube-auth-proxy-799868c9c7-h75ft" Jul 06 01:26:29.609467 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:29.609446 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/979609f9-f877-47b2-a3b7-17d491a53d23-tls-certs\") pod \"kube-auth-proxy-799868c9c7-h75ft\" (UID: \"979609f9-f877-47b2-a3b7-17d491a53d23\") " pod="openshift-ingress/kube-auth-proxy-799868c9c7-h75ft" Jul 06 01:26:29.618116 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:29.618083 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-97qdj\" (UniqueName: \"kubernetes.io/projected/979609f9-f877-47b2-a3b7-17d491a53d23-kube-api-access-97qdj\") pod \"kube-auth-proxy-799868c9c7-h75ft\" (UID: \"979609f9-f877-47b2-a3b7-17d491a53d23\") " pod="openshift-ingress/kube-auth-proxy-799868c9c7-h75ft" Jul 06 01:26:29.771661 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:29.771631 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-799868c9c7-h75ft" Jul 06 01:26:29.912153 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:29.912125 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-799868c9c7-h75ft"] Jul 06 01:26:29.914641 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:26:29.914613 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod979609f9_f877_47b2_a3b7_17d491a53d23.slice/crio-a7ef9e0be6152fce3080f2253d25b9407144a0d69ec65376fc2ab5ab3f4732a1 WatchSource:0}: Error finding container a7ef9e0be6152fce3080f2253d25b9407144a0d69ec65376fc2ab5ab3f4732a1: Status 404 returned error can't find the container with id a7ef9e0be6152fce3080f2253d25b9407144a0d69ec65376fc2ab5ab3f4732a1 Jul 06 01:26:30.145869 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:30.145765 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-799868c9c7-h75ft" event={"ID":"979609f9-f877-47b2-a3b7-17d491a53d23","Type":"ContainerStarted","Data":"a7ef9e0be6152fce3080f2253d25b9407144a0d69ec65376fc2ab5ab3f4732a1"} Jul 06 01:26:33.167708 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:33.167660 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-799868c9c7-h75ft" event={"ID":"979609f9-f877-47b2-a3b7-17d491a53d23","Type":"ContainerStarted","Data":"47c88f393804f8a051aa481fe9f00584f312d223c5613b45545cf314f9e77532"} Jul 06 01:26:33.184575 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:26:33.184522 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-799868c9c7-h75ft" podStartSLOduration=1.675504136 podStartE2EDuration="4.184504328s" totalImagesPullingTime="2.509000192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:26:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:26:33.182636252 +0000 UTC m=+365.324195062" watchObservedRunningTime="2026-07-06 01:26:33.184504328 +0000 UTC m=+365.326063113" Jul 06 01:28:03.643234 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:03.643144 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c7d44bb8f-jc6tc"] Jul 06 01:28:03.651206 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:03.651171 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c7d44bb8f-jc6tc" Jul 06 01:28:03.653719 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:03.653693 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 06 01:28:03.654033 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:03.654010 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 06 01:28:03.654637 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:03.654615 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 06 01:28:03.654730 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:03.654646 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 06 01:28:03.654730 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:03.654615 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-vjkln\"" Jul 06 01:28:03.654900 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:03.654888 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 06 01:28:03.659054 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:03.659031 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c7d44bb8f-jc6tc"] Jul 06 01:28:03.659678 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:03.659646 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 06 01:28:03.766679 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:03.766624 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/35f08148-08b1-4267-908c-e24d039e73d6-console-serving-cert\") pod \"console-6c7d44bb8f-jc6tc\" (UID: \"35f08148-08b1-4267-908c-e24d039e73d6\") " pod="openshift-console/console-6c7d44bb8f-jc6tc" Jul 06 01:28:03.766679 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:03.766687 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/35f08148-08b1-4267-908c-e24d039e73d6-console-config\") pod \"console-6c7d44bb8f-jc6tc\" (UID: \"35f08148-08b1-4267-908c-e24d039e73d6\") " pod="openshift-console/console-6c7d44bb8f-jc6tc" Jul 06 01:28:03.766905 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:03.766753 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/35f08148-08b1-4267-908c-e24d039e73d6-console-oauth-config\") pod \"console-6c7d44bb8f-jc6tc\" (UID: \"35f08148-08b1-4267-908c-e24d039e73d6\") " pod="openshift-console/console-6c7d44bb8f-jc6tc" Jul 06 01:28:03.766905 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:03.766787 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/35f08148-08b1-4267-908c-e24d039e73d6-oauth-serving-cert\") pod \"console-6c7d44bb8f-jc6tc\" (UID: \"35f08148-08b1-4267-908c-e24d039e73d6\") " pod="openshift-console/console-6c7d44bb8f-jc6tc" Jul 06 01:28:03.766905 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:03.766833 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/35f08148-08b1-4267-908c-e24d039e73d6-service-ca\") pod \"console-6c7d44bb8f-jc6tc\" (UID: \"35f08148-08b1-4267-908c-e24d039e73d6\") " pod="openshift-console/console-6c7d44bb8f-jc6tc" Jul 06 01:28:03.767008 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:03.766913 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35f08148-08b1-4267-908c-e24d039e73d6-trusted-ca-bundle\") pod \"console-6c7d44bb8f-jc6tc\" (UID: \"35f08148-08b1-4267-908c-e24d039e73d6\") " pod="openshift-console/console-6c7d44bb8f-jc6tc" Jul 06 01:28:03.767008 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:03.766959 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzlsv\" (UniqueName: \"kubernetes.io/projected/35f08148-08b1-4267-908c-e24d039e73d6-kube-api-access-lzlsv\") pod \"console-6c7d44bb8f-jc6tc\" (UID: \"35f08148-08b1-4267-908c-e24d039e73d6\") " pod="openshift-console/console-6c7d44bb8f-jc6tc" Jul 06 01:28:03.867983 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:03.867929 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzlsv\" (UniqueName: \"kubernetes.io/projected/35f08148-08b1-4267-908c-e24d039e73d6-kube-api-access-lzlsv\") pod \"console-6c7d44bb8f-jc6tc\" (UID: \"35f08148-08b1-4267-908c-e24d039e73d6\") " pod="openshift-console/console-6c7d44bb8f-jc6tc" Jul 06 01:28:03.867983 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:03.867990 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/35f08148-08b1-4267-908c-e24d039e73d6-console-serving-cert\") pod \"console-6c7d44bb8f-jc6tc\" (UID: \"35f08148-08b1-4267-908c-e24d039e73d6\") " pod="openshift-console/console-6c7d44bb8f-jc6tc" Jul 06 01:28:03.868228 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:03.868045 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/35f08148-08b1-4267-908c-e24d039e73d6-console-config\") pod \"console-6c7d44bb8f-jc6tc\" (UID: \"35f08148-08b1-4267-908c-e24d039e73d6\") " pod="openshift-console/console-6c7d44bb8f-jc6tc" Jul 06 01:28:03.868228 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:03.868081 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/35f08148-08b1-4267-908c-e24d039e73d6-console-oauth-config\") pod \"console-6c7d44bb8f-jc6tc\" (UID: \"35f08148-08b1-4267-908c-e24d039e73d6\") " pod="openshift-console/console-6c7d44bb8f-jc6tc" Jul 06 01:28:03.868228 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:03.868110 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/35f08148-08b1-4267-908c-e24d039e73d6-oauth-serving-cert\") pod \"console-6c7d44bb8f-jc6tc\" (UID: \"35f08148-08b1-4267-908c-e24d039e73d6\") " pod="openshift-console/console-6c7d44bb8f-jc6tc" Jul 06 01:28:03.868228 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:03.868163 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/35f08148-08b1-4267-908c-e24d039e73d6-service-ca\") pod \"console-6c7d44bb8f-jc6tc\" (UID: \"35f08148-08b1-4267-908c-e24d039e73d6\") " pod="openshift-console/console-6c7d44bb8f-jc6tc" Jul 06 01:28:03.868228 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:03.868210 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35f08148-08b1-4267-908c-e24d039e73d6-trusted-ca-bundle\") pod \"console-6c7d44bb8f-jc6tc\" (UID: \"35f08148-08b1-4267-908c-e24d039e73d6\") " pod="openshift-console/console-6c7d44bb8f-jc6tc" Jul 06 01:28:03.868942 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:03.868918 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/35f08148-08b1-4267-908c-e24d039e73d6-oauth-serving-cert\") pod \"console-6c7d44bb8f-jc6tc\" (UID: \"35f08148-08b1-4267-908c-e24d039e73d6\") " pod="openshift-console/console-6c7d44bb8f-jc6tc" Jul 06 01:28:03.868942 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:03.868935 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/35f08148-08b1-4267-908c-e24d039e73d6-console-config\") pod \"console-6c7d44bb8f-jc6tc\" (UID: \"35f08148-08b1-4267-908c-e24d039e73d6\") " pod="openshift-console/console-6c7d44bb8f-jc6tc" Jul 06 01:28:03.869073 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:03.868935 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/35f08148-08b1-4267-908c-e24d039e73d6-service-ca\") pod \"console-6c7d44bb8f-jc6tc\" (UID: \"35f08148-08b1-4267-908c-e24d039e73d6\") " pod="openshift-console/console-6c7d44bb8f-jc6tc" Jul 06 01:28:03.869345 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:03.869324 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35f08148-08b1-4267-908c-e24d039e73d6-trusted-ca-bundle\") pod \"console-6c7d44bb8f-jc6tc\" (UID: \"35f08148-08b1-4267-908c-e24d039e73d6\") " pod="openshift-console/console-6c7d44bb8f-jc6tc" Jul 06 01:28:03.870634 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:03.870611 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/35f08148-08b1-4267-908c-e24d039e73d6-console-oauth-config\") pod \"console-6c7d44bb8f-jc6tc\" (UID: \"35f08148-08b1-4267-908c-e24d039e73d6\") " pod="openshift-console/console-6c7d44bb8f-jc6tc" Jul 06 01:28:03.870895 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:03.870880 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/35f08148-08b1-4267-908c-e24d039e73d6-console-serving-cert\") pod \"console-6c7d44bb8f-jc6tc\" (UID: \"35f08148-08b1-4267-908c-e24d039e73d6\") " pod="openshift-console/console-6c7d44bb8f-jc6tc" Jul 06 01:28:03.883881 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:03.883849 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzlsv\" (UniqueName: \"kubernetes.io/projected/35f08148-08b1-4267-908c-e24d039e73d6-kube-api-access-lzlsv\") pod \"console-6c7d44bb8f-jc6tc\" (UID: \"35f08148-08b1-4267-908c-e24d039e73d6\") " pod="openshift-console/console-6c7d44bb8f-jc6tc" Jul 06 01:28:03.962953 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:03.962862 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c7d44bb8f-jc6tc" Jul 06 01:28:04.109494 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:04.109459 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c7d44bb8f-jc6tc"] Jul 06 01:28:04.112896 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:28:04.112868 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35f08148_08b1_4267_908c_e24d039e73d6.slice/crio-044f6adc8aa445ca2d2c7aac8ea74611b4d36861e687107bd4436496c822673f WatchSource:0}: Error finding container 044f6adc8aa445ca2d2c7aac8ea74611b4d36861e687107bd4436496c822673f: Status 404 returned error can't find the container with id 044f6adc8aa445ca2d2c7aac8ea74611b4d36861e687107bd4436496c822673f Jul 06 01:28:04.586349 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:04.586311 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c7d44bb8f-jc6tc" event={"ID":"35f08148-08b1-4267-908c-e24d039e73d6","Type":"ContainerStarted","Data":"deb86ea2005220f3124b7ef4068d12faf87e53758b3fff011cef738d87de1fe5"} Jul 06 01:28:04.586525 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:04.586367 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c7d44bb8f-jc6tc" event={"ID":"35f08148-08b1-4267-908c-e24d039e73d6","Type":"ContainerStarted","Data":"044f6adc8aa445ca2d2c7aac8ea74611b4d36861e687107bd4436496c822673f"} Jul 06 01:28:04.604972 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:04.604920 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6c7d44bb8f-jc6tc" podStartSLOduration=1.604903207 podStartE2EDuration="1.604903207s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:28:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:28:04.603195765 +0000 UTC m=+456.744754550" watchObservedRunningTime="2026-07-06 01:28:04.604903207 +0000 UTC m=+456.746461993" Jul 06 01:28:13.963528 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:13.963484 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c7d44bb8f-jc6tc" Jul 06 01:28:13.963977 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:13.963554 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c7d44bb8f-jc6tc" Jul 06 01:28:13.968370 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:13.968345 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c7d44bb8f-jc6tc" Jul 06 01:28:14.644610 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:14.644579 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c7d44bb8f-jc6tc" Jul 06 01:28:28.711878 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:28.711845 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2sdb8"] Jul 06 01:28:28.715169 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:28.715153 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2sdb8" Jul 06 01:28:28.717792 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:28.717768 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-v2k2b\"" Jul 06 01:28:28.717989 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:28.717973 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 01:28:28.718340 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:28.718326 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 01:28:28.728262 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:28.728235 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2sdb8"] Jul 06 01:28:28.795294 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:28.795251 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb67c0ca-88f3-44b4-bb56-4eb286f5d74c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-2sdb8\" (UID: \"bb67c0ca-88f3-44b4-bb56-4eb286f5d74c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2sdb8" Jul 06 01:28:28.795595 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:28.795315 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cjjz\" (UniqueName: \"kubernetes.io/projected/bb67c0ca-88f3-44b4-bb56-4eb286f5d74c-kube-api-access-4cjjz\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-2sdb8\" (UID: \"bb67c0ca-88f3-44b4-bb56-4eb286f5d74c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2sdb8" Jul 06 01:28:28.819893 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:28.819844 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2sdb8"] Jul 06 01:28:28.820178 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:28:28.820159 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-4cjjz], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2sdb8" podUID="bb67c0ca-88f3-44b4-bb56-4eb286f5d74c" Jul 06 01:28:28.896155 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:28.896115 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb67c0ca-88f3-44b4-bb56-4eb286f5d74c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-2sdb8\" (UID: \"bb67c0ca-88f3-44b4-bb56-4eb286f5d74c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2sdb8" Jul 06 01:28:28.896342 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:28.896211 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cjjz\" (UniqueName: \"kubernetes.io/projected/bb67c0ca-88f3-44b4-bb56-4eb286f5d74c-kube-api-access-4cjjz\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-2sdb8\" (UID: \"bb67c0ca-88f3-44b4-bb56-4eb286f5d74c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2sdb8" Jul 06 01:28:28.896583 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:28.896566 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb67c0ca-88f3-44b4-bb56-4eb286f5d74c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-2sdb8\" (UID: \"bb67c0ca-88f3-44b4-bb56-4eb286f5d74c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2sdb8" Jul 06 01:28:28.905530 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:28.905504 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cjjz\" (UniqueName: \"kubernetes.io/projected/bb67c0ca-88f3-44b4-bb56-4eb286f5d74c-kube-api-access-4cjjz\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-2sdb8\" (UID: \"bb67c0ca-88f3-44b4-bb56-4eb286f5d74c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2sdb8" Jul 06 01:28:29.267952 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:29.267916 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2sdb8"] Jul 06 01:28:29.273859 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:29.273829 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2sdb8"] Jul 06 01:28:29.310282 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:29.310246 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p89gs"] Jul 06 01:28:29.313961 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:29.313912 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p89gs" Jul 06 01:28:29.328631 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:29.328552 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p89gs"] Jul 06 01:28:29.400914 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:29.400875 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/314c66a9-1495-4367-856d-a4dc664760db-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-p89gs\" (UID: \"314c66a9-1495-4367-856d-a4dc664760db\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p89gs" Jul 06 01:28:29.401089 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:29.400923 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7gzh\" (UniqueName: \"kubernetes.io/projected/314c66a9-1495-4367-856d-a4dc664760db-kube-api-access-h7gzh\") pod \"kuadrant-operator-controller-manager-8684c9ff49-p89gs\" (UID: \"314c66a9-1495-4367-856d-a4dc664760db\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p89gs" Jul 06 01:28:29.501779 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:29.501737 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/314c66a9-1495-4367-856d-a4dc664760db-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-p89gs\" (UID: \"314c66a9-1495-4367-856d-a4dc664760db\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p89gs" Jul 06 01:28:29.501981 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:29.501792 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7gzh\" (UniqueName: \"kubernetes.io/projected/314c66a9-1495-4367-856d-a4dc664760db-kube-api-access-h7gzh\") pod \"kuadrant-operator-controller-manager-8684c9ff49-p89gs\" (UID: \"314c66a9-1495-4367-856d-a4dc664760db\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p89gs" Jul 06 01:28:29.502161 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:29.502136 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/314c66a9-1495-4367-856d-a4dc664760db-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-p89gs\" (UID: \"314c66a9-1495-4367-856d-a4dc664760db\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p89gs" Jul 06 01:28:29.516484 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:29.516457 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7gzh\" (UniqueName: \"kubernetes.io/projected/314c66a9-1495-4367-856d-a4dc664760db-kube-api-access-h7gzh\") pod \"kuadrant-operator-controller-manager-8684c9ff49-p89gs\" (UID: \"314c66a9-1495-4367-856d-a4dc664760db\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p89gs" Jul 06 01:28:29.626297 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:29.626204 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p89gs" Jul 06 01:28:29.707261 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:29.707230 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2sdb8" Jul 06 01:28:29.709715 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:28:29.709677 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-2sdb8\" is forbidden: User \"system:node:ip-10-0-143-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-159.ec2.internal' and this object" podUID="bb67c0ca-88f3-44b4-bb56-4eb286f5d74c" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2sdb8" Jul 06 01:28:29.714763 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:29.714737 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2sdb8" Jul 06 01:28:29.716925 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:28:29.716895 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-2sdb8\" is forbidden: User \"system:node:ip-10-0-143-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-159.ec2.internal' and this object" podUID="bb67c0ca-88f3-44b4-bb56-4eb286f5d74c" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2sdb8" Jul 06 01:28:29.772718 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:29.772671 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p89gs"] Jul 06 01:28:29.775011 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:28:29.774983 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod314c66a9_1495_4367_856d_a4dc664760db.slice/crio-3ab762d0b897eedd9a77ec1b86d5537b5209c8a2e4d834ec4faad28edaab70d1 WatchSource:0}: Error finding container 3ab762d0b897eedd9a77ec1b86d5537b5209c8a2e4d834ec4faad28edaab70d1: Status 404 returned error can't find the container with id 3ab762d0b897eedd9a77ec1b86d5537b5209c8a2e4d834ec4faad28edaab70d1 Jul 06 01:28:29.804328 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:29.804294 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4cjjz\" (UniqueName: \"kubernetes.io/projected/bb67c0ca-88f3-44b4-bb56-4eb286f5d74c-kube-api-access-4cjjz\") pod \"bb67c0ca-88f3-44b4-bb56-4eb286f5d74c\" (UID: \"bb67c0ca-88f3-44b4-bb56-4eb286f5d74c\") " Jul 06 01:28:29.804526 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:29.804421 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb67c0ca-88f3-44b4-bb56-4eb286f5d74c-extensions-socket-volume\") pod \"bb67c0ca-88f3-44b4-bb56-4eb286f5d74c\" (UID: \"bb67c0ca-88f3-44b4-bb56-4eb286f5d74c\") " Jul 06 01:28:29.804691 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:29.804669 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb67c0ca-88f3-44b4-bb56-4eb286f5d74c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "bb67c0ca-88f3-44b4-bb56-4eb286f5d74c" (UID: "bb67c0ca-88f3-44b4-bb56-4eb286f5d74c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:28:29.804754 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:29.804742 2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb67c0ca-88f3-44b4-bb56-4eb286f5d74c-extensions-socket-volume\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:28:29.806722 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:29.806699 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb67c0ca-88f3-44b4-bb56-4eb286f5d74c-kube-api-access-4cjjz" (OuterVolumeSpecName: "kube-api-access-4cjjz") pod "bb67c0ca-88f3-44b4-bb56-4eb286f5d74c" (UID: "bb67c0ca-88f3-44b4-bb56-4eb286f5d74c"). InnerVolumeSpecName "kube-api-access-4cjjz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:28:29.906074 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:29.905983 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4cjjz\" (UniqueName: \"kubernetes.io/projected/bb67c0ca-88f3-44b4-bb56-4eb286f5d74c-kube-api-access-4cjjz\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:28:30.324940 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:30.324905 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb67c0ca-88f3-44b4-bb56-4eb286f5d74c" path="/var/lib/kubelet/pods/bb67c0ca-88f3-44b4-bb56-4eb286f5d74c/volumes" Jul 06 01:28:30.715077 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:30.714998 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2sdb8" Jul 06 01:28:30.715537 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:30.714996 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p89gs" event={"ID":"314c66a9-1495-4367-856d-a4dc664760db","Type":"ContainerStarted","Data":"3ab762d0b897eedd9a77ec1b86d5537b5209c8a2e4d834ec4faad28edaab70d1"} Jul 06 01:28:30.719269 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:28:30.719233 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-2sdb8\" is forbidden: User \"system:node:ip-10-0-143-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-159.ec2.internal' and this object" podUID="bb67c0ca-88f3-44b4-bb56-4eb286f5d74c" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2sdb8" Jul 06 01:28:34.737708 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:34.737678 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p89gs" event={"ID":"314c66a9-1495-4367-856d-a4dc664760db","Type":"ContainerStarted","Data":"350f29cd5b603c74ad657cda2785e4416d732332b6ca96b2b9420b909cf45dea"} Jul 06 01:28:34.738184 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:34.737766 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p89gs" Jul 06 01:28:34.756089 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:34.755881 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p89gs" podStartSLOduration=0.977838707 podStartE2EDuration="5.755862732s" totalImagesPullingTime="4.778024025s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:28:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:28:34.754698089 +0000 UTC m=+486.896256911" watchObservedRunningTime="2026-07-06 01:28:34.755862732 +0000 UTC m=+486.897421705" Jul 06 01:28:45.745824 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:45.745792 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p89gs" Jul 06 01:28:53.886975 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:53.886932 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p89gs"] Jul 06 01:28:53.887491 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:53.887190 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p89gs" podUID="314c66a9-1495-4367-856d-a4dc664760db" containerName="manager" containerID="cri-o://350f29cd5b603c74ad657cda2785e4416d732332b6ca96b2b9420b909cf45dea" gracePeriod=10 Jul 06 01:28:54.135498 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:54.135471 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p89gs" Jul 06 01:28:54.229545 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:54.229511 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/314c66a9-1495-4367-856d-a4dc664760db-extensions-socket-volume\") pod \"314c66a9-1495-4367-856d-a4dc664760db\" (UID: \"314c66a9-1495-4367-856d-a4dc664760db\") " Jul 06 01:28:54.229736 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:54.229552 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h7gzh\" (UniqueName: \"kubernetes.io/projected/314c66a9-1495-4367-856d-a4dc664760db-kube-api-access-h7gzh\") pod \"314c66a9-1495-4367-856d-a4dc664760db\" (UID: \"314c66a9-1495-4367-856d-a4dc664760db\") " Jul 06 01:28:54.230000 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:54.229975 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/314c66a9-1495-4367-856d-a4dc664760db-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "314c66a9-1495-4367-856d-a4dc664760db" (UID: "314c66a9-1495-4367-856d-a4dc664760db"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:28:54.231971 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:54.231948 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/314c66a9-1495-4367-856d-a4dc664760db-kube-api-access-h7gzh" (OuterVolumeSpecName: "kube-api-access-h7gzh") pod "314c66a9-1495-4367-856d-a4dc664760db" (UID: "314c66a9-1495-4367-856d-a4dc664760db"). InnerVolumeSpecName "kube-api-access-h7gzh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:28:54.331048 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:54.330836 2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/314c66a9-1495-4367-856d-a4dc664760db-extensions-socket-volume\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:28:54.331048 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:54.330855 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h7gzh\" (UniqueName: \"kubernetes.io/projected/314c66a9-1495-4367-856d-a4dc664760db-kube-api-access-h7gzh\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:28:54.834736 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:54.834697 2565 generic.go:356] "Generic (PLEG): container finished" podID="314c66a9-1495-4367-856d-a4dc664760db" containerID="350f29cd5b603c74ad657cda2785e4416d732332b6ca96b2b9420b909cf45dea" exitCode=0 Jul 06 01:28:54.834736 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:54.834738 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p89gs" Jul 06 01:28:54.834947 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:54.834793 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p89gs" event={"ID":"314c66a9-1495-4367-856d-a4dc664760db","Type":"ContainerDied","Data":"350f29cd5b603c74ad657cda2785e4416d732332b6ca96b2b9420b909cf45dea"} Jul 06 01:28:54.834947 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:54.834845 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p89gs" event={"ID":"314c66a9-1495-4367-856d-a4dc664760db","Type":"ContainerDied","Data":"3ab762d0b897eedd9a77ec1b86d5537b5209c8a2e4d834ec4faad28edaab70d1"} Jul 06 01:28:54.834947 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:54.834867 2565 scope.go:117] "RemoveContainer" containerID="350f29cd5b603c74ad657cda2785e4416d732332b6ca96b2b9420b909cf45dea" Jul 06 01:28:54.844841 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:54.844824 2565 scope.go:117] "RemoveContainer" containerID="350f29cd5b603c74ad657cda2785e4416d732332b6ca96b2b9420b909cf45dea" Jul 06 01:28:54.845137 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:28:54.845115 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"350f29cd5b603c74ad657cda2785e4416d732332b6ca96b2b9420b909cf45dea\": container with ID starting with 350f29cd5b603c74ad657cda2785e4416d732332b6ca96b2b9420b909cf45dea not found: ID does not exist" containerID="350f29cd5b603c74ad657cda2785e4416d732332b6ca96b2b9420b909cf45dea" Jul 06 01:28:54.845248 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:54.845145 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"350f29cd5b603c74ad657cda2785e4416d732332b6ca96b2b9420b909cf45dea"} err="failed to get container status \"350f29cd5b603c74ad657cda2785e4416d732332b6ca96b2b9420b909cf45dea\": rpc error: code = NotFound desc = could not find container \"350f29cd5b603c74ad657cda2785e4416d732332b6ca96b2b9420b909cf45dea\": container with ID starting with 350f29cd5b603c74ad657cda2785e4416d732332b6ca96b2b9420b909cf45dea not found: ID does not exist" Jul 06 01:28:54.855459 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:54.855435 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p89gs"] Jul 06 01:28:54.861671 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:54.861647 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p89gs"] Jul 06 01:28:56.324607 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:28:56.324571 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="314c66a9-1495-4367-856d-a4dc664760db" path="/var/lib/kubelet/pods/314c66a9-1495-4367-856d-a4dc664760db/volumes" Jul 06 01:29:15.218777 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:15.218740 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-5v2mn"] Jul 06 01:29:15.219256 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:15.219104 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="314c66a9-1495-4367-856d-a4dc664760db" containerName="manager" Jul 06 01:29:15.219256 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:15.219115 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="314c66a9-1495-4367-856d-a4dc664760db" containerName="manager" Jul 06 01:29:15.219256 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:15.219179 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="314c66a9-1495-4367-856d-a4dc664760db" containerName="manager" Jul 06 01:29:15.224525 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:15.224494 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-5v2mn" Jul 06 01:29:15.227261 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:15.227214 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 01:29:15.228219 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:15.228195 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 06 01:29:15.228546 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:15.228529 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 01:29:15.228647 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:15.228530 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-wsd9j\"" Jul 06 01:29:15.229782 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:15.229761 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-5v2mn"] Jul 06 01:29:15.318932 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:15.318898 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/da81cda8-7153-4f41-ac37-7875647ef840-config-file\") pod \"limitador-limitador-64c94fd64f-5v2mn\" (UID: \"da81cda8-7153-4f41-ac37-7875647ef840\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-5v2mn" Jul 06 01:29:15.319147 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:15.318964 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vplwz\" (UniqueName: \"kubernetes.io/projected/da81cda8-7153-4f41-ac37-7875647ef840-kube-api-access-vplwz\") pod \"limitador-limitador-64c94fd64f-5v2mn\" (UID: \"da81cda8-7153-4f41-ac37-7875647ef840\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-5v2mn" Jul 06 01:29:15.334745 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:15.334710 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-5v2mn"] Jul 06 01:29:15.419433 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:15.419349 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vplwz\" (UniqueName: \"kubernetes.io/projected/da81cda8-7153-4f41-ac37-7875647ef840-kube-api-access-vplwz\") pod \"limitador-limitador-64c94fd64f-5v2mn\" (UID: \"da81cda8-7153-4f41-ac37-7875647ef840\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-5v2mn" Jul 06 01:29:15.419626 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:15.419499 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/da81cda8-7153-4f41-ac37-7875647ef840-config-file\") pod \"limitador-limitador-64c94fd64f-5v2mn\" (UID: \"da81cda8-7153-4f41-ac37-7875647ef840\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-5v2mn" Jul 06 01:29:15.420135 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:15.420116 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/da81cda8-7153-4f41-ac37-7875647ef840-config-file\") pod \"limitador-limitador-64c94fd64f-5v2mn\" (UID: \"da81cda8-7153-4f41-ac37-7875647ef840\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-5v2mn" Jul 06 01:29:15.427953 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:15.427924 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vplwz\" (UniqueName: \"kubernetes.io/projected/da81cda8-7153-4f41-ac37-7875647ef840-kube-api-access-vplwz\") pod \"limitador-limitador-64c94fd64f-5v2mn\" (UID: \"da81cda8-7153-4f41-ac37-7875647ef840\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-5v2mn" Jul 06 01:29:15.536803 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:15.536751 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-5v2mn" Jul 06 01:29:15.682882 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:15.682802 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-5v2mn"] Jul 06 01:29:15.686036 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:29:15.686008 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda81cda8_7153_4f41_ac37_7875647ef840.slice/crio-4fe97c1991cd968d3e823abcdb333cba181389314fbd02dfb0ce25fc63cb2c79 WatchSource:0}: Error finding container 4fe97c1991cd968d3e823abcdb333cba181389314fbd02dfb0ce25fc63cb2c79: Status 404 returned error can't find the container with id 4fe97c1991cd968d3e823abcdb333cba181389314fbd02dfb0ce25fc63cb2c79 Jul 06 01:29:15.943267 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:15.943175 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-5v2mn" event={"ID":"da81cda8-7153-4f41-ac37-7875647ef840","Type":"ContainerStarted","Data":"4fe97c1991cd968d3e823abcdb333cba181389314fbd02dfb0ce25fc63cb2c79"} Jul 06 01:29:16.034849 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:16.034815 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-vkr5h"] Jul 06 01:29:16.039652 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:16.039624 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vkr5h" Jul 06 01:29:16.042262 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:16.042237 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-6qm4q\"" Jul 06 01:29:16.043990 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:16.043964 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-vkr5h"] Jul 06 01:29:16.126092 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:16.126049 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c64zd\" (UniqueName: \"kubernetes.io/projected/a14ea7c9-e983-47d9-a702-fa5fe0b8074b-kube-api-access-c64zd\") pod \"authorino-58cd955675-vkr5h\" (UID: \"a14ea7c9-e983-47d9-a702-fa5fe0b8074b\") " pod="kuadrant-system/authorino-58cd955675-vkr5h" Jul 06 01:29:16.226594 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:16.226558 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c64zd\" (UniqueName: \"kubernetes.io/projected/a14ea7c9-e983-47d9-a702-fa5fe0b8074b-kube-api-access-c64zd\") pod \"authorino-58cd955675-vkr5h\" (UID: \"a14ea7c9-e983-47d9-a702-fa5fe0b8074b\") " pod="kuadrant-system/authorino-58cd955675-vkr5h" Jul 06 01:29:16.235572 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:16.235545 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c64zd\" (UniqueName: \"kubernetes.io/projected/a14ea7c9-e983-47d9-a702-fa5fe0b8074b-kube-api-access-c64zd\") pod \"authorino-58cd955675-vkr5h\" (UID: \"a14ea7c9-e983-47d9-a702-fa5fe0b8074b\") " pod="kuadrant-system/authorino-58cd955675-vkr5h" Jul 06 01:29:16.350594 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:16.350561 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vkr5h" Jul 06 01:29:16.516895 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:16.516650 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-vkr5h"] Jul 06 01:29:16.521586 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:29:16.521396 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda14ea7c9_e983_47d9_a702_fa5fe0b8074b.slice/crio-de61498d7498ff51e571bc7bab9d5048914977e04b4a770938c491239fa35d8d WatchSource:0}: Error finding container de61498d7498ff51e571bc7bab9d5048914977e04b4a770938c491239fa35d8d: Status 404 returned error can't find the container with id de61498d7498ff51e571bc7bab9d5048914977e04b4a770938c491239fa35d8d Jul 06 01:29:16.950279 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:16.950190 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vkr5h" event={"ID":"a14ea7c9-e983-47d9-a702-fa5fe0b8074b","Type":"ContainerStarted","Data":"de61498d7498ff51e571bc7bab9d5048914977e04b4a770938c491239fa35d8d"} Jul 06 01:29:20.977007 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:20.976962 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vkr5h" event={"ID":"a14ea7c9-e983-47d9-a702-fa5fe0b8074b","Type":"ContainerStarted","Data":"c7230e20e5ee5ffdb4b72191719e320922c4ee61bf1aff7ac5b1025b81f70181"} Jul 06 01:29:20.978966 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:20.978936 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-5v2mn" event={"ID":"da81cda8-7153-4f41-ac37-7875647ef840","Type":"ContainerStarted","Data":"97e538f8040d4e5f50194ef25a6720a41581a13040df4b24a724030f56cac730"} Jul 06 01:29:20.979110 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:20.979006 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-5v2mn" Jul 06 01:29:20.993516 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:20.993481 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-vkr5h"] Jul 06 01:29:21.003850 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:21.003794 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-vkr5h" podStartSLOduration=1.049934538 podStartE2EDuration="5.003778192s" totalImagesPullingTime="3.953843654s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:29:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:29:21.003062703 +0000 UTC m=+533.144621498" watchObservedRunningTime="2026-07-06 01:29:21.003778192 +0000 UTC m=+533.145336976" Jul 06 01:29:21.022474 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:21.022422 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-5v2mn" podStartSLOduration=1.240933504 podStartE2EDuration="6.022405448s" totalImagesPullingTime="4.781471944s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:29:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:29:21.021886314 +0000 UTC m=+533.163445099" watchObservedRunningTime="2026-07-06 01:29:21.022405448 +0000 UTC m=+533.163964234" Jul 06 01:29:22.988488 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:22.988448 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-vkr5h" podUID="a14ea7c9-e983-47d9-a702-fa5fe0b8074b" containerName="authorino" containerID="cri-o://c7230e20e5ee5ffdb4b72191719e320922c4ee61bf1aff7ac5b1025b81f70181" gracePeriod=30 Jul 06 01:29:23.244046 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:23.243990 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vkr5h" Jul 06 01:29:23.297031 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:23.296989 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c64zd\" (UniqueName: \"kubernetes.io/projected/a14ea7c9-e983-47d9-a702-fa5fe0b8074b-kube-api-access-c64zd\") pod \"a14ea7c9-e983-47d9-a702-fa5fe0b8074b\" (UID: \"a14ea7c9-e983-47d9-a702-fa5fe0b8074b\") " Jul 06 01:29:23.299236 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:23.299209 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a14ea7c9-e983-47d9-a702-fa5fe0b8074b-kube-api-access-c64zd" (OuterVolumeSpecName: "kube-api-access-c64zd") pod "a14ea7c9-e983-47d9-a702-fa5fe0b8074b" (UID: "a14ea7c9-e983-47d9-a702-fa5fe0b8074b"). InnerVolumeSpecName "kube-api-access-c64zd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:29:23.398732 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:23.398688 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c64zd\" (UniqueName: \"kubernetes.io/projected/a14ea7c9-e983-47d9-a702-fa5fe0b8074b-kube-api-access-c64zd\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:29:23.994042 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:23.994008 2565 generic.go:356] "Generic (PLEG): container finished" podID="a14ea7c9-e983-47d9-a702-fa5fe0b8074b" containerID="c7230e20e5ee5ffdb4b72191719e320922c4ee61bf1aff7ac5b1025b81f70181" exitCode=0 Jul 06 01:29:23.994467 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:23.994053 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vkr5h" Jul 06 01:29:23.994467 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:23.994100 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vkr5h" event={"ID":"a14ea7c9-e983-47d9-a702-fa5fe0b8074b","Type":"ContainerDied","Data":"c7230e20e5ee5ffdb4b72191719e320922c4ee61bf1aff7ac5b1025b81f70181"} Jul 06 01:29:23.994467 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:23.994138 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vkr5h" event={"ID":"a14ea7c9-e983-47d9-a702-fa5fe0b8074b","Type":"ContainerDied","Data":"de61498d7498ff51e571bc7bab9d5048914977e04b4a770938c491239fa35d8d"} Jul 06 01:29:23.994467 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:23.994154 2565 scope.go:117] "RemoveContainer" containerID="c7230e20e5ee5ffdb4b72191719e320922c4ee61bf1aff7ac5b1025b81f70181" Jul 06 01:29:24.003507 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:24.003488 2565 scope.go:117] "RemoveContainer" containerID="c7230e20e5ee5ffdb4b72191719e320922c4ee61bf1aff7ac5b1025b81f70181" Jul 06 01:29:24.003811 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:29:24.003789 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7230e20e5ee5ffdb4b72191719e320922c4ee61bf1aff7ac5b1025b81f70181\": container with ID starting with c7230e20e5ee5ffdb4b72191719e320922c4ee61bf1aff7ac5b1025b81f70181 not found: ID does not exist" containerID="c7230e20e5ee5ffdb4b72191719e320922c4ee61bf1aff7ac5b1025b81f70181" Jul 06 01:29:24.003849 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:24.003823 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7230e20e5ee5ffdb4b72191719e320922c4ee61bf1aff7ac5b1025b81f70181"} err="failed to get container status \"c7230e20e5ee5ffdb4b72191719e320922c4ee61bf1aff7ac5b1025b81f70181\": rpc error: code = NotFound desc = could not find container \"c7230e20e5ee5ffdb4b72191719e320922c4ee61bf1aff7ac5b1025b81f70181\": container with ID starting with c7230e20e5ee5ffdb4b72191719e320922c4ee61bf1aff7ac5b1025b81f70181 not found: ID does not exist" Jul 06 01:29:24.015058 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:24.015033 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-vkr5h"] Jul 06 01:29:24.020610 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:24.020585 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-vkr5h"] Jul 06 01:29:24.324257 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:24.324175 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a14ea7c9-e983-47d9-a702-fa5fe0b8074b" path="/var/lib/kubelet/pods/a14ea7c9-e983-47d9-a702-fa5fe0b8074b/volumes" Jul 06 01:29:31.984993 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:31.984967 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-5v2mn" Jul 06 01:29:32.014789 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:32.014751 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-5v2mn"] Jul 06 01:29:32.033361 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:32.033300 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-5v2mn" podUID="da81cda8-7153-4f41-ac37-7875647ef840" containerName="limitador" containerID="cri-o://97e538f8040d4e5f50194ef25a6720a41581a13040df4b24a724030f56cac730" gracePeriod=30 Jul 06 01:29:32.585016 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:32.584991 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-5v2mn" Jul 06 01:29:32.685849 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:32.685745 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/da81cda8-7153-4f41-ac37-7875647ef840-config-file\") pod \"da81cda8-7153-4f41-ac37-7875647ef840\" (UID: \"da81cda8-7153-4f41-ac37-7875647ef840\") " Jul 06 01:29:32.685849 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:32.685851 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vplwz\" (UniqueName: \"kubernetes.io/projected/da81cda8-7153-4f41-ac37-7875647ef840-kube-api-access-vplwz\") pod \"da81cda8-7153-4f41-ac37-7875647ef840\" (UID: \"da81cda8-7153-4f41-ac37-7875647ef840\") " Jul 06 01:29:32.686123 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:32.686098 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da81cda8-7153-4f41-ac37-7875647ef840-config-file" (OuterVolumeSpecName: "config-file") pod "da81cda8-7153-4f41-ac37-7875647ef840" (UID: "da81cda8-7153-4f41-ac37-7875647ef840"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 01:29:32.688076 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:32.688053 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da81cda8-7153-4f41-ac37-7875647ef840-kube-api-access-vplwz" (OuterVolumeSpecName: "kube-api-access-vplwz") pod "da81cda8-7153-4f41-ac37-7875647ef840" (UID: "da81cda8-7153-4f41-ac37-7875647ef840"). InnerVolumeSpecName "kube-api-access-vplwz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:29:32.787503 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:32.787448 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vplwz\" (UniqueName: \"kubernetes.io/projected/da81cda8-7153-4f41-ac37-7875647ef840-kube-api-access-vplwz\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:29:32.787503 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:32.787493 2565 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/da81cda8-7153-4f41-ac37-7875647ef840-config-file\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:29:33.038710 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:33.038673 2565 generic.go:356] "Generic (PLEG): container finished" podID="da81cda8-7153-4f41-ac37-7875647ef840" containerID="97e538f8040d4e5f50194ef25a6720a41581a13040df4b24a724030f56cac730" exitCode=0 Jul 06 01:29:33.039106 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:33.038718 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-5v2mn" Jul 06 01:29:33.039106 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:33.038761 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-5v2mn" event={"ID":"da81cda8-7153-4f41-ac37-7875647ef840","Type":"ContainerDied","Data":"97e538f8040d4e5f50194ef25a6720a41581a13040df4b24a724030f56cac730"} Jul 06 01:29:33.039106 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:33.038816 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-5v2mn" event={"ID":"da81cda8-7153-4f41-ac37-7875647ef840","Type":"ContainerDied","Data":"4fe97c1991cd968d3e823abcdb333cba181389314fbd02dfb0ce25fc63cb2c79"} Jul 06 01:29:33.039106 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:33.038840 2565 scope.go:117] "RemoveContainer" containerID="97e538f8040d4e5f50194ef25a6720a41581a13040df4b24a724030f56cac730" Jul 06 01:29:33.048047 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:33.048028 2565 scope.go:117] "RemoveContainer" containerID="97e538f8040d4e5f50194ef25a6720a41581a13040df4b24a724030f56cac730" Jul 06 01:29:33.048341 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:29:33.048320 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97e538f8040d4e5f50194ef25a6720a41581a13040df4b24a724030f56cac730\": container with ID starting with 97e538f8040d4e5f50194ef25a6720a41581a13040df4b24a724030f56cac730 not found: ID does not exist" containerID="97e538f8040d4e5f50194ef25a6720a41581a13040df4b24a724030f56cac730" Jul 06 01:29:33.048414 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:33.048351 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97e538f8040d4e5f50194ef25a6720a41581a13040df4b24a724030f56cac730"} err="failed to get container status \"97e538f8040d4e5f50194ef25a6720a41581a13040df4b24a724030f56cac730\": rpc error: code = NotFound desc = could not find container \"97e538f8040d4e5f50194ef25a6720a41581a13040df4b24a724030f56cac730\": container with ID starting with 97e538f8040d4e5f50194ef25a6720a41581a13040df4b24a724030f56cac730 not found: ID does not exist" Jul 06 01:29:33.058967 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:33.058936 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-5v2mn"] Jul 06 01:29:33.064864 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:33.064836 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-5v2mn"] Jul 06 01:29:34.324350 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:34.324314 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da81cda8-7153-4f41-ac37-7875647ef840" path="/var/lib/kubelet/pods/da81cda8-7153-4f41-ac37-7875647ef840/volumes" Jul 06 01:29:34.380627 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:34.380588 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-jk4pl"] Jul 06 01:29:34.381181 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:34.381162 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da81cda8-7153-4f41-ac37-7875647ef840" containerName="limitador" Jul 06 01:29:34.381247 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:34.381186 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="da81cda8-7153-4f41-ac37-7875647ef840" containerName="limitador" Jul 06 01:29:34.381247 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:34.381212 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a14ea7c9-e983-47d9-a702-fa5fe0b8074b" containerName="authorino" Jul 06 01:29:34.381247 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:34.381222 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a14ea7c9-e983-47d9-a702-fa5fe0b8074b" containerName="authorino" Jul 06 01:29:34.381354 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:34.381342 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a14ea7c9-e983-47d9-a702-fa5fe0b8074b" containerName="authorino" Jul 06 01:29:34.381405 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:34.381362 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="da81cda8-7153-4f41-ac37-7875647ef840" containerName="limitador" Jul 06 01:29:34.386236 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:34.386214 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-jk4pl" Jul 06 01:29:34.388550 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:34.388526 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 06 01:29:34.388715 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:34.388688 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-rv7pr\"" Jul 06 01:29:34.392491 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:34.392466 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-jk4pl"] Jul 06 01:29:34.501790 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:34.501746 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/1138dd2e-074c-4dc2-a681-f02a1ea002e7-data\") pod \"postgres-69fb4594cf-jk4pl\" (UID: \"1138dd2e-074c-4dc2-a681-f02a1ea002e7\") " pod="opendatahub/postgres-69fb4594cf-jk4pl" Jul 06 01:29:34.501992 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:34.501929 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hv4gz\" (UniqueName: \"kubernetes.io/projected/1138dd2e-074c-4dc2-a681-f02a1ea002e7-kube-api-access-hv4gz\") pod \"postgres-69fb4594cf-jk4pl\" (UID: \"1138dd2e-074c-4dc2-a681-f02a1ea002e7\") " pod="opendatahub/postgres-69fb4594cf-jk4pl" Jul 06 01:29:34.602824 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:34.602731 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hv4gz\" (UniqueName: \"kubernetes.io/projected/1138dd2e-074c-4dc2-a681-f02a1ea002e7-kube-api-access-hv4gz\") pod \"postgres-69fb4594cf-jk4pl\" (UID: \"1138dd2e-074c-4dc2-a681-f02a1ea002e7\") " pod="opendatahub/postgres-69fb4594cf-jk4pl" Jul 06 01:29:34.602824 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:34.602769 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/1138dd2e-074c-4dc2-a681-f02a1ea002e7-data\") pod \"postgres-69fb4594cf-jk4pl\" (UID: \"1138dd2e-074c-4dc2-a681-f02a1ea002e7\") " pod="opendatahub/postgres-69fb4594cf-jk4pl" Jul 06 01:29:34.603116 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:34.603100 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/1138dd2e-074c-4dc2-a681-f02a1ea002e7-data\") pod \"postgres-69fb4594cf-jk4pl\" (UID: \"1138dd2e-074c-4dc2-a681-f02a1ea002e7\") " pod="opendatahub/postgres-69fb4594cf-jk4pl" Jul 06 01:29:34.611537 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:34.611506 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hv4gz\" (UniqueName: \"kubernetes.io/projected/1138dd2e-074c-4dc2-a681-f02a1ea002e7-kube-api-access-hv4gz\") pod \"postgres-69fb4594cf-jk4pl\" (UID: \"1138dd2e-074c-4dc2-a681-f02a1ea002e7\") " pod="opendatahub/postgres-69fb4594cf-jk4pl" Jul 06 01:29:34.698211 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:34.698169 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-jk4pl" Jul 06 01:29:34.841632 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:34.841607 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-jk4pl"] Jul 06 01:29:34.844197 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:29:34.844167 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1138dd2e_074c_4dc2_a681_f02a1ea002e7.slice/crio-e6f067d6c96f192d91e3451018341cdd3fc598d0ff3674cd10ce5fba57c54838 WatchSource:0}: Error finding container e6f067d6c96f192d91e3451018341cdd3fc598d0ff3674cd10ce5fba57c54838: Status 404 returned error can't find the container with id e6f067d6c96f192d91e3451018341cdd3fc598d0ff3674cd10ce5fba57c54838 Jul 06 01:29:35.049291 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:35.049257 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-jk4pl" event={"ID":"1138dd2e-074c-4dc2-a681-f02a1ea002e7","Type":"ContainerStarted","Data":"e6f067d6c96f192d91e3451018341cdd3fc598d0ff3674cd10ce5fba57c54838"} Jul 06 01:29:41.087388 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:41.087351 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-jk4pl" event={"ID":"1138dd2e-074c-4dc2-a681-f02a1ea002e7","Type":"ContainerStarted","Data":"f6af34f89781739e9865bbbae9bc12ff67e8f3e72215442d8d3381858eaaf565"} Jul 06 01:29:41.087792 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:41.087456 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-jk4pl" Jul 06 01:29:41.104517 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:41.104462 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-jk4pl" podStartSLOduration=1.7344036489999999 podStartE2EDuration="7.104446208s" totalImagesPullingTime="5.370042559s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:29:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:29:41.101752765 +0000 UTC m=+553.243311567" watchObservedRunningTime="2026-07-06 01:29:41.104446208 +0000 UTC m=+553.246004993" Jul 06 01:29:47.123586 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:47.123555 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-jk4pl" Jul 06 01:29:47.631358 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:47.631317 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-k78k5"] Jul 06 01:29:47.637885 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:47.637854 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-k78k5" Jul 06 01:29:47.640518 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:47.640488 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 01:29:47.640670 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:47.640545 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-6qm4q\"" Jul 06 01:29:47.640864 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:47.640845 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-k78k5"] Jul 06 01:29:47.641356 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:47.641338 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 01:29:47.822806 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:47.822761 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fh7q\" (UniqueName: \"kubernetes.io/projected/69a4e37a-36e9-41e7-969c-637acde12498-kube-api-access-7fh7q\") pod \"authorino-5d4b67c446-k78k5\" (UID: \"69a4e37a-36e9-41e7-969c-637acde12498\") " pod="kuadrant-system/authorino-5d4b67c446-k78k5" Jul 06 01:29:47.860198 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:47.860149 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-k78k5"] Jul 06 01:29:47.860488 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:29:47.860467 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-7fh7q], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-k78k5" podUID="69a4e37a-36e9-41e7-969c-637acde12498" Jul 06 01:29:47.881444 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:47.881335 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-574d754759-z68p6"] Jul 06 01:29:47.885430 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:47.885406 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-574d754759-z68p6" Jul 06 01:29:47.888011 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:47.887987 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-574d754759-z68p6"] Jul 06 01:29:47.923983 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:47.923943 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fh7q\" (UniqueName: \"kubernetes.io/projected/69a4e37a-36e9-41e7-969c-637acde12498-kube-api-access-7fh7q\") pod \"authorino-5d4b67c446-k78k5\" (UID: \"69a4e37a-36e9-41e7-969c-637acde12498\") " pod="kuadrant-system/authorino-5d4b67c446-k78k5" Jul 06 01:29:47.931941 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:47.931912 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fh7q\" (UniqueName: \"kubernetes.io/projected/69a4e37a-36e9-41e7-969c-637acde12498-kube-api-access-7fh7q\") pod \"authorino-5d4b67c446-k78k5\" (UID: \"69a4e37a-36e9-41e7-969c-637acde12498\") " pod="kuadrant-system/authorino-5d4b67c446-k78k5" Jul 06 01:29:47.999669 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:47.999628 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-574d754759-z68p6"] Jul 06 01:29:47.999991 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:29:47.999970 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-4qjvs], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-574d754759-z68p6" podUID="74de3d44-4092-4689-be3d-0b83792830f0" Jul 06 01:29:48.029194 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:48.026954 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-78f5c5cf67-pb5kx"] Jul 06 01:29:48.030001 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:48.029971 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qjvs\" (UniqueName: \"kubernetes.io/projected/74de3d44-4092-4689-be3d-0b83792830f0-kube-api-access-4qjvs\") pod \"authorino-574d754759-z68p6\" (UID: \"74de3d44-4092-4689-be3d-0b83792830f0\") " pod="kuadrant-system/authorino-574d754759-z68p6" Jul 06 01:29:48.033017 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:48.032986 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78f5c5cf67-pb5kx" Jul 06 01:29:48.034514 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:48.034487 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-78f5c5cf67-pb5kx"] Jul 06 01:29:48.035498 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:48.035478 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 06 01:29:48.120957 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:48.120922 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-k78k5" Jul 06 01:29:48.121145 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:48.120922 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-574d754759-z68p6" Jul 06 01:29:48.128366 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:48.128343 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-k78k5" Jul 06 01:29:48.130647 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:48.130618 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qjvs\" (UniqueName: \"kubernetes.io/projected/74de3d44-4092-4689-be3d-0b83792830f0-kube-api-access-4qjvs\") pod \"authorino-574d754759-z68p6\" (UID: \"74de3d44-4092-4689-be3d-0b83792830f0\") " pod="kuadrant-system/authorino-574d754759-z68p6" Jul 06 01:29:48.131743 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:48.131704 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-574d754759-z68p6" Jul 06 01:29:48.139752 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:48.139725 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qjvs\" (UniqueName: \"kubernetes.io/projected/74de3d44-4092-4689-be3d-0b83792830f0-kube-api-access-4qjvs\") pod \"authorino-574d754759-z68p6\" (UID: \"74de3d44-4092-4689-be3d-0b83792830f0\") " pod="kuadrant-system/authorino-574d754759-z68p6" Jul 06 01:29:48.231480 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:48.231438 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7fh7q\" (UniqueName: \"kubernetes.io/projected/69a4e37a-36e9-41e7-969c-637acde12498-kube-api-access-7fh7q\") pod \"69a4e37a-36e9-41e7-969c-637acde12498\" (UID: \"69a4e37a-36e9-41e7-969c-637acde12498\") " Jul 06 01:29:48.231641 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:48.231582 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4r4xs\" (UniqueName: \"kubernetes.io/projected/d2c52d2a-3776-467f-afbb-3ba67ed11529-kube-api-access-4r4xs\") pod \"authorino-78f5c5cf67-pb5kx\" (UID: \"d2c52d2a-3776-467f-afbb-3ba67ed11529\") " pod="kuadrant-system/authorino-78f5c5cf67-pb5kx" Jul 06 01:29:48.231685 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:48.231667 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d2c52d2a-3776-467f-afbb-3ba67ed11529-tls-cert\") pod \"authorino-78f5c5cf67-pb5kx\" (UID: \"d2c52d2a-3776-467f-afbb-3ba67ed11529\") " pod="kuadrant-system/authorino-78f5c5cf67-pb5kx" Jul 06 01:29:48.233838 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:48.233803 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69a4e37a-36e9-41e7-969c-637acde12498-kube-api-access-7fh7q" (OuterVolumeSpecName: "kube-api-access-7fh7q") pod "69a4e37a-36e9-41e7-969c-637acde12498" (UID: "69a4e37a-36e9-41e7-969c-637acde12498"). InnerVolumeSpecName "kube-api-access-7fh7q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:29:48.332286 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:48.332258 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4qjvs\" (UniqueName: \"kubernetes.io/projected/74de3d44-4092-4689-be3d-0b83792830f0-kube-api-access-4qjvs\") pod \"74de3d44-4092-4689-be3d-0b83792830f0\" (UID: \"74de3d44-4092-4689-be3d-0b83792830f0\") " Jul 06 01:29:48.332748 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:48.332722 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d2c52d2a-3776-467f-afbb-3ba67ed11529-tls-cert\") pod \"authorino-78f5c5cf67-pb5kx\" (UID: \"d2c52d2a-3776-467f-afbb-3ba67ed11529\") " pod="kuadrant-system/authorino-78f5c5cf67-pb5kx" Jul 06 01:29:48.333109 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:48.333082 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r4xs\" (UniqueName: \"kubernetes.io/projected/d2c52d2a-3776-467f-afbb-3ba67ed11529-kube-api-access-4r4xs\") pod \"authorino-78f5c5cf67-pb5kx\" (UID: \"d2c52d2a-3776-467f-afbb-3ba67ed11529\") " pod="kuadrant-system/authorino-78f5c5cf67-pb5kx" Jul 06 01:29:48.333259 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:48.333229 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7fh7q\" (UniqueName: \"kubernetes.io/projected/69a4e37a-36e9-41e7-969c-637acde12498-kube-api-access-7fh7q\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:29:48.334534 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:48.334513 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74de3d44-4092-4689-be3d-0b83792830f0-kube-api-access-4qjvs" (OuterVolumeSpecName: "kube-api-access-4qjvs") pod "74de3d44-4092-4689-be3d-0b83792830f0" (UID: "74de3d44-4092-4689-be3d-0b83792830f0"). InnerVolumeSpecName "kube-api-access-4qjvs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:29:48.335206 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:48.335191 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d2c52d2a-3776-467f-afbb-3ba67ed11529-tls-cert\") pod \"authorino-78f5c5cf67-pb5kx\" (UID: \"d2c52d2a-3776-467f-afbb-3ba67ed11529\") " pod="kuadrant-system/authorino-78f5c5cf67-pb5kx" Jul 06 01:29:48.340177 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:48.340154 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r4xs\" (UniqueName: \"kubernetes.io/projected/d2c52d2a-3776-467f-afbb-3ba67ed11529-kube-api-access-4r4xs\") pod \"authorino-78f5c5cf67-pb5kx\" (UID: \"d2c52d2a-3776-467f-afbb-3ba67ed11529\") " pod="kuadrant-system/authorino-78f5c5cf67-pb5kx" Jul 06 01:29:48.343021 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:48.342990 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78f5c5cf67-pb5kx" Jul 06 01:29:48.434021 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:48.433987 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qjvs\" (UniqueName: \"kubernetes.io/projected/74de3d44-4092-4689-be3d-0b83792830f0-kube-api-access-4qjvs\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:29:48.488570 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:48.488544 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-78f5c5cf67-pb5kx"] Jul 06 01:29:48.490535 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:29:48.490505 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd2c52d2a_3776_467f_afbb_3ba67ed11529.slice/crio-0aa63608119ed4695f24088d52dc9fc8c52e19b63c05fcdfcff7dd81974cc968 WatchSource:0}: Error finding container 0aa63608119ed4695f24088d52dc9fc8c52e19b63c05fcdfcff7dd81974cc968: Status 404 returned error can't find the container with id 0aa63608119ed4695f24088d52dc9fc8c52e19b63c05fcdfcff7dd81974cc968 Jul 06 01:29:49.127403 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:49.127346 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-78f5c5cf67-pb5kx" event={"ID":"d2c52d2a-3776-467f-afbb-3ba67ed11529","Type":"ContainerStarted","Data":"0aa63608119ed4695f24088d52dc9fc8c52e19b63c05fcdfcff7dd81974cc968"} Jul 06 01:29:49.127403 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:49.127411 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-574d754759-z68p6" Jul 06 01:29:49.127626 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:49.127408 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-k78k5" Jul 06 01:29:49.153934 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:49.153902 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-k78k5"] Jul 06 01:29:49.159617 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:49.159589 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-k78k5"] Jul 06 01:29:49.178762 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:49.178728 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-574d754759-z68p6"] Jul 06 01:29:49.181933 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:49.181906 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-574d754759-z68p6"] Jul 06 01:29:50.133230 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:50.133188 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-78f5c5cf67-pb5kx" event={"ID":"d2c52d2a-3776-467f-afbb-3ba67ed11529","Type":"ContainerStarted","Data":"8f2788ddcd5119550c4e780fe266964cb8c493be9568242196dd279ce005c9f0"} Jul 06 01:29:50.150033 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:50.149978 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-78f5c5cf67-pb5kx" podStartSLOduration=1.5384406309999998 podStartE2EDuration="2.149959782s" totalImagesPullingTime="611.519151ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:29:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:29:50.148193404 +0000 UTC m=+562.289752190" watchObservedRunningTime="2026-07-06 01:29:50.149959782 +0000 UTC m=+562.291518566" Jul 06 01:29:50.324333 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:50.324300 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69a4e37a-36e9-41e7-969c-637acde12498" path="/var/lib/kubelet/pods/69a4e37a-36e9-41e7-969c-637acde12498/volumes" Jul 06 01:29:50.324717 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:50.324573 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74de3d44-4092-4689-be3d-0b83792830f0" path="/var/lib/kubelet/pods/74de3d44-4092-4689-be3d-0b83792830f0/volumes" Jul 06 01:29:54.573778 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:54.573743 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-645586966b-jmnjz"] Jul 06 01:29:54.586926 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:54.586902 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-645586966b-jmnjz" Jul 06 01:29:54.591175 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:54.591145 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 06 01:29:54.591326 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:54.591146 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 06 01:29:54.591326 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:54.591210 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-cqwgs\"" Jul 06 01:29:54.597081 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:54.597007 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-645586966b-jmnjz"] Jul 06 01:29:54.695011 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:54.694967 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xhgk\" (UniqueName: \"kubernetes.io/projected/0abcc062-4c22-4219-ad4d-3049272539d6-kube-api-access-5xhgk\") pod \"maas-controller-645586966b-jmnjz\" (UID: \"0abcc062-4c22-4219-ad4d-3049272539d6\") " pod="opendatahub/maas-controller-645586966b-jmnjz" Jul 06 01:29:54.695176 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:54.695029 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0abcc062-4c22-4219-ad4d-3049272539d6-webhook-cert\") pod \"maas-controller-645586966b-jmnjz\" (UID: \"0abcc062-4c22-4219-ad4d-3049272539d6\") " pod="opendatahub/maas-controller-645586966b-jmnjz" Jul 06 01:29:54.770490 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:54.770452 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-797b48b58c-zsdtc"] Jul 06 01:29:54.772868 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:54.772846 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-797b48b58c-zsdtc" Jul 06 01:29:54.781675 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:54.781644 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-797b48b58c-zsdtc"] Jul 06 01:29:54.795821 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:54.795779 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsx2v\" (UniqueName: \"kubernetes.io/projected/26c168fa-05d6-462a-92b7-923ea184833c-kube-api-access-qsx2v\") pod \"maas-controller-797b48b58c-zsdtc\" (UID: \"26c168fa-05d6-462a-92b7-923ea184833c\") " pod="opendatahub/maas-controller-797b48b58c-zsdtc" Jul 06 01:29:54.796013 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:54.795863 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xhgk\" (UniqueName: \"kubernetes.io/projected/0abcc062-4c22-4219-ad4d-3049272539d6-kube-api-access-5xhgk\") pod \"maas-controller-645586966b-jmnjz\" (UID: \"0abcc062-4c22-4219-ad4d-3049272539d6\") " pod="opendatahub/maas-controller-645586966b-jmnjz" Jul 06 01:29:54.796013 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:54.795938 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/26c168fa-05d6-462a-92b7-923ea184833c-webhook-cert\") pod \"maas-controller-797b48b58c-zsdtc\" (UID: \"26c168fa-05d6-462a-92b7-923ea184833c\") " pod="opendatahub/maas-controller-797b48b58c-zsdtc" Jul 06 01:29:54.796013 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:54.795987 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0abcc062-4c22-4219-ad4d-3049272539d6-webhook-cert\") pod \"maas-controller-645586966b-jmnjz\" (UID: \"0abcc062-4c22-4219-ad4d-3049272539d6\") " pod="opendatahub/maas-controller-645586966b-jmnjz" Jul 06 01:29:54.798761 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:54.798737 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0abcc062-4c22-4219-ad4d-3049272539d6-webhook-cert\") pod \"maas-controller-645586966b-jmnjz\" (UID: \"0abcc062-4c22-4219-ad4d-3049272539d6\") " pod="opendatahub/maas-controller-645586966b-jmnjz" Jul 06 01:29:54.808077 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:54.808054 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xhgk\" (UniqueName: \"kubernetes.io/projected/0abcc062-4c22-4219-ad4d-3049272539d6-kube-api-access-5xhgk\") pod \"maas-controller-645586966b-jmnjz\" (UID: \"0abcc062-4c22-4219-ad4d-3049272539d6\") " pod="opendatahub/maas-controller-645586966b-jmnjz" Jul 06 01:29:54.897433 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:54.897302 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/26c168fa-05d6-462a-92b7-923ea184833c-webhook-cert\") pod \"maas-controller-797b48b58c-zsdtc\" (UID: \"26c168fa-05d6-462a-92b7-923ea184833c\") " pod="opendatahub/maas-controller-797b48b58c-zsdtc" Jul 06 01:29:54.897606 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:54.897447 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsx2v\" (UniqueName: \"kubernetes.io/projected/26c168fa-05d6-462a-92b7-923ea184833c-kube-api-access-qsx2v\") pod \"maas-controller-797b48b58c-zsdtc\" (UID: \"26c168fa-05d6-462a-92b7-923ea184833c\") " pod="opendatahub/maas-controller-797b48b58c-zsdtc" Jul 06 01:29:54.899688 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:54.899657 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-645586966b-jmnjz" Jul 06 01:29:54.899945 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:54.899923 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/26c168fa-05d6-462a-92b7-923ea184833c-webhook-cert\") pod \"maas-controller-797b48b58c-zsdtc\" (UID: \"26c168fa-05d6-462a-92b7-923ea184833c\") " pod="opendatahub/maas-controller-797b48b58c-zsdtc" Jul 06 01:29:54.905765 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:54.905736 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsx2v\" (UniqueName: \"kubernetes.io/projected/26c168fa-05d6-462a-92b7-923ea184833c-kube-api-access-qsx2v\") pod \"maas-controller-797b48b58c-zsdtc\" (UID: \"26c168fa-05d6-462a-92b7-923ea184833c\") " pod="opendatahub/maas-controller-797b48b58c-zsdtc" Jul 06 01:29:55.038579 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:55.038548 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-645586966b-jmnjz"] Jul 06 01:29:55.041263 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:29:55.041224 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0abcc062_4c22_4219_ad4d_3049272539d6.slice/crio-caac10ae1654fbf44807b90d8509af319c49f1e21d27061bbb17bd8f0a902569 WatchSource:0}: Error finding container caac10ae1654fbf44807b90d8509af319c49f1e21d27061bbb17bd8f0a902569: Status 404 returned error can't find the container with id caac10ae1654fbf44807b90d8509af319c49f1e21d27061bbb17bd8f0a902569 Jul 06 01:29:55.083866 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:55.083825 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-797b48b58c-zsdtc" Jul 06 01:29:55.159356 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:55.159277 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-645586966b-jmnjz" event={"ID":"0abcc062-4c22-4219-ad4d-3049272539d6","Type":"ContainerStarted","Data":"caac10ae1654fbf44807b90d8509af319c49f1e21d27061bbb17bd8f0a902569"} Jul 06 01:29:55.222927 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:55.222890 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-797b48b58c-zsdtc"] Jul 06 01:29:55.226518 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:29:55.226488 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26c168fa_05d6_462a_92b7_923ea184833c.slice/crio-94bc6a2f265ab44315683b9596c08914e7b02df963515995645748e887667e77 WatchSource:0}: Error finding container 94bc6a2f265ab44315683b9596c08914e7b02df963515995645748e887667e77: Status 404 returned error can't find the container with id 94bc6a2f265ab44315683b9596c08914e7b02df963515995645748e887667e77 Jul 06 01:29:56.172617 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:56.172519 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-797b48b58c-zsdtc" event={"ID":"26c168fa-05d6-462a-92b7-923ea184833c","Type":"ContainerStarted","Data":"94bc6a2f265ab44315683b9596c08914e7b02df963515995645748e887667e77"} Jul 06 01:29:58.184481 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:58.184437 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-645586966b-jmnjz" event={"ID":"0abcc062-4c22-4219-ad4d-3049272539d6","Type":"ContainerStarted","Data":"39ef01701c957216f15d896b54092b706f3e140764a99ba472b01fb5951c3ace"} Jul 06 01:29:58.184957 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:58.184602 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-645586966b-jmnjz" Jul 06 01:29:58.185952 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:58.185929 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-797b48b58c-zsdtc" event={"ID":"26c168fa-05d6-462a-92b7-923ea184833c","Type":"ContainerStarted","Data":"62dd376a44b898496adda32651ca37ba7d34c792e0730b8457ad7184af1a7a86"} Jul 06 01:29:58.186093 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:58.186080 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-797b48b58c-zsdtc" Jul 06 01:29:58.206317 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:58.206262 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-645586966b-jmnjz" podStartSLOduration=2.083935608 podStartE2EDuration="4.206246287s" totalImagesPullingTime="2.122310679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:29:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:29:58.203988092 +0000 UTC m=+570.345546876" watchObservedRunningTime="2026-07-06 01:29:58.206246287 +0000 UTC m=+570.347805071" Jul 06 01:29:58.220814 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:29:58.220655 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-797b48b58c-zsdtc" podStartSLOduration=2.273217274 podStartE2EDuration="4.220633905s" totalImagesPullingTime="1.947416631s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:29:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:29:58.218300869 +0000 UTC m=+570.359859703" watchObservedRunningTime="2026-07-06 01:29:58.220633905 +0000 UTC m=+570.362192697" Jul 06 01:30:09.197480 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:09.197450 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-797b48b58c-zsdtc" Jul 06 01:30:09.197967 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:09.197539 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-645586966b-jmnjz" Jul 06 01:30:09.253705 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:09.253665 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-645586966b-jmnjz"] Jul 06 01:30:09.253954 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:09.253900 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-645586966b-jmnjz" podUID="0abcc062-4c22-4219-ad4d-3049272539d6" containerName="manager" containerID="cri-o://39ef01701c957216f15d896b54092b706f3e140764a99ba472b01fb5951c3ace" gracePeriod=30 Jul 06 01:30:09.508851 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:09.508824 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-645586966b-jmnjz" Jul 06 01:30:09.639845 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:09.639805 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5xhgk\" (UniqueName: \"kubernetes.io/projected/0abcc062-4c22-4219-ad4d-3049272539d6-kube-api-access-5xhgk\") pod \"0abcc062-4c22-4219-ad4d-3049272539d6\" (UID: \"0abcc062-4c22-4219-ad4d-3049272539d6\") " Jul 06 01:30:09.640027 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:09.639900 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0abcc062-4c22-4219-ad4d-3049272539d6-webhook-cert\") pod \"0abcc062-4c22-4219-ad4d-3049272539d6\" (UID: \"0abcc062-4c22-4219-ad4d-3049272539d6\") " Jul 06 01:30:09.642250 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:09.642219 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0abcc062-4c22-4219-ad4d-3049272539d6-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "0abcc062-4c22-4219-ad4d-3049272539d6" (UID: "0abcc062-4c22-4219-ad4d-3049272539d6"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 01:30:09.642367 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:09.642270 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0abcc062-4c22-4219-ad4d-3049272539d6-kube-api-access-5xhgk" (OuterVolumeSpecName: "kube-api-access-5xhgk") pod "0abcc062-4c22-4219-ad4d-3049272539d6" (UID: "0abcc062-4c22-4219-ad4d-3049272539d6"). InnerVolumeSpecName "kube-api-access-5xhgk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:30:09.740750 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:09.740649 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5xhgk\" (UniqueName: \"kubernetes.io/projected/0abcc062-4c22-4219-ad4d-3049272539d6-kube-api-access-5xhgk\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:30:09.740750 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:09.740684 2565 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0abcc062-4c22-4219-ad4d-3049272539d6-webhook-cert\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:30:10.246833 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:10.246793 2565 generic.go:356] "Generic (PLEG): container finished" podID="0abcc062-4c22-4219-ad4d-3049272539d6" containerID="39ef01701c957216f15d896b54092b706f3e140764a99ba472b01fb5951c3ace" exitCode=0 Jul 06 01:30:10.247241 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:10.246855 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-645586966b-jmnjz" Jul 06 01:30:10.247241 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:10.246874 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-645586966b-jmnjz" event={"ID":"0abcc062-4c22-4219-ad4d-3049272539d6","Type":"ContainerDied","Data":"39ef01701c957216f15d896b54092b706f3e140764a99ba472b01fb5951c3ace"} Jul 06 01:30:10.247241 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:10.246952 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-645586966b-jmnjz" event={"ID":"0abcc062-4c22-4219-ad4d-3049272539d6","Type":"ContainerDied","Data":"caac10ae1654fbf44807b90d8509af319c49f1e21d27061bbb17bd8f0a902569"} Jul 06 01:30:10.247241 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:10.246974 2565 scope.go:117] "RemoveContainer" containerID="39ef01701c957216f15d896b54092b706f3e140764a99ba472b01fb5951c3ace" Jul 06 01:30:10.256672 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:10.256501 2565 scope.go:117] "RemoveContainer" containerID="39ef01701c957216f15d896b54092b706f3e140764a99ba472b01fb5951c3ace" Jul 06 01:30:10.256836 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:30:10.256815 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39ef01701c957216f15d896b54092b706f3e140764a99ba472b01fb5951c3ace\": container with ID starting with 39ef01701c957216f15d896b54092b706f3e140764a99ba472b01fb5951c3ace not found: ID does not exist" containerID="39ef01701c957216f15d896b54092b706f3e140764a99ba472b01fb5951c3ace" Jul 06 01:30:10.256880 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:10.256847 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39ef01701c957216f15d896b54092b706f3e140764a99ba472b01fb5951c3ace"} err="failed to get container status \"39ef01701c957216f15d896b54092b706f3e140764a99ba472b01fb5951c3ace\": rpc error: code = NotFound desc = could not find container \"39ef01701c957216f15d896b54092b706f3e140764a99ba472b01fb5951c3ace\": container with ID starting with 39ef01701c957216f15d896b54092b706f3e140764a99ba472b01fb5951c3ace not found: ID does not exist" Jul 06 01:30:10.267673 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:10.267641 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-645586966b-jmnjz"] Jul 06 01:30:10.271074 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:10.271047 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-645586966b-jmnjz"] Jul 06 01:30:10.324305 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:10.324268 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0abcc062-4c22-4219-ad4d-3049272539d6" path="/var/lib/kubelet/pods/0abcc062-4c22-4219-ad4d-3049272539d6/volumes" Jul 06 01:30:28.958345 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:28.958306 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-6787fdf5f-8fjs9"] Jul 06 01:30:28.958780 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:28.958715 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0abcc062-4c22-4219-ad4d-3049272539d6" containerName="manager" Jul 06 01:30:28.958780 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:28.958727 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="0abcc062-4c22-4219-ad4d-3049272539d6" containerName="manager" Jul 06 01:30:28.958852 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:28.958794 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="0abcc062-4c22-4219-ad4d-3049272539d6" containerName="manager" Jul 06 01:30:28.963255 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:28.963237 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6787fdf5f-8fjs9" Jul 06 01:30:28.965561 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:28.965540 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 06 01:30:28.965561 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:28.965556 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-rhllt\"" Jul 06 01:30:28.970833 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:28.970804 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-6787fdf5f-8fjs9"] Jul 06 01:30:29.018196 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:29.018164 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59b58f6a-6c28-47b0-b99b-dd462f59d915-openshift-service-ca\") pod \"maas-api-6787fdf5f-8fjs9\" (UID: \"59b58f6a-6c28-47b0-b99b-dd462f59d915\") " pod="opendatahub/maas-api-6787fdf5f-8fjs9" Jul 06 01:30:29.018360 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:29.018250 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59b58f6a-6c28-47b0-b99b-dd462f59d915-maas-api-tls\") pod \"maas-api-6787fdf5f-8fjs9\" (UID: \"59b58f6a-6c28-47b0-b99b-dd462f59d915\") " pod="opendatahub/maas-api-6787fdf5f-8fjs9" Jul 06 01:30:29.018360 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:29.018285 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qf6mj\" (UniqueName: \"kubernetes.io/projected/59b58f6a-6c28-47b0-b99b-dd462f59d915-kube-api-access-qf6mj\") pod \"maas-api-6787fdf5f-8fjs9\" (UID: \"59b58f6a-6c28-47b0-b99b-dd462f59d915\") " pod="opendatahub/maas-api-6787fdf5f-8fjs9" Jul 06 01:30:29.119559 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:29.119521 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59b58f6a-6c28-47b0-b99b-dd462f59d915-openshift-service-ca\") pod \"maas-api-6787fdf5f-8fjs9\" (UID: \"59b58f6a-6c28-47b0-b99b-dd462f59d915\") " pod="opendatahub/maas-api-6787fdf5f-8fjs9" Jul 06 01:30:29.119559 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:29.119577 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59b58f6a-6c28-47b0-b99b-dd462f59d915-maas-api-tls\") pod \"maas-api-6787fdf5f-8fjs9\" (UID: \"59b58f6a-6c28-47b0-b99b-dd462f59d915\") " pod="opendatahub/maas-api-6787fdf5f-8fjs9" Jul 06 01:30:29.119799 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:29.119604 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qf6mj\" (UniqueName: \"kubernetes.io/projected/59b58f6a-6c28-47b0-b99b-dd462f59d915-kube-api-access-qf6mj\") pod \"maas-api-6787fdf5f-8fjs9\" (UID: \"59b58f6a-6c28-47b0-b99b-dd462f59d915\") " pod="opendatahub/maas-api-6787fdf5f-8fjs9" Jul 06 01:30:29.120280 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:29.120258 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59b58f6a-6c28-47b0-b99b-dd462f59d915-openshift-service-ca\") pod \"maas-api-6787fdf5f-8fjs9\" (UID: \"59b58f6a-6c28-47b0-b99b-dd462f59d915\") " pod="opendatahub/maas-api-6787fdf5f-8fjs9" Jul 06 01:30:29.122287 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:29.122262 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59b58f6a-6c28-47b0-b99b-dd462f59d915-maas-api-tls\") pod \"maas-api-6787fdf5f-8fjs9\" (UID: \"59b58f6a-6c28-47b0-b99b-dd462f59d915\") " pod="opendatahub/maas-api-6787fdf5f-8fjs9" Jul 06 01:30:29.131987 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:29.131961 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qf6mj\" (UniqueName: \"kubernetes.io/projected/59b58f6a-6c28-47b0-b99b-dd462f59d915-kube-api-access-qf6mj\") pod \"maas-api-6787fdf5f-8fjs9\" (UID: \"59b58f6a-6c28-47b0-b99b-dd462f59d915\") " pod="opendatahub/maas-api-6787fdf5f-8fjs9" Jul 06 01:30:29.275029 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:29.274996 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6787fdf5f-8fjs9" Jul 06 01:30:29.423667 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:29.423640 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-6787fdf5f-8fjs9"] Jul 06 01:30:29.426309 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:30:29.426268 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59b58f6a_6c28_47b0_b99b_dd462f59d915.slice/crio-627011e2715af7e34efb328f1917abd90c9109b4a2737b985ab16ad6a01f5d35 WatchSource:0}: Error finding container 627011e2715af7e34efb328f1917abd90c9109b4a2737b985ab16ad6a01f5d35: Status 404 returned error can't find the container with id 627011e2715af7e34efb328f1917abd90c9109b4a2737b985ab16ad6a01f5d35 Jul 06 01:30:29.569932 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:29.569846 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-6prj5"] Jul 06 01:30:29.574665 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:29.574642 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-6prj5" Jul 06 01:30:29.577314 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:29.577292 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 06 01:30:29.577946 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:29.577925 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-q8wzf\"" Jul 06 01:30:29.587545 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:29.587518 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-6prj5"] Jul 06 01:30:29.623984 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:29.623943 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/767bc03c-400a-4f1b-a4e5-f5c50c3a8ad1-plugins-config-volume\") pod \"payload-processing-58f86c7798-6prj5\" (UID: \"767bc03c-400a-4f1b-a4e5-f5c50c3a8ad1\") " pod="openshift-ingress/payload-processing-58f86c7798-6prj5" Jul 06 01:30:29.624174 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:29.624083 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gz2p9\" (UniqueName: \"kubernetes.io/projected/767bc03c-400a-4f1b-a4e5-f5c50c3a8ad1-kube-api-access-gz2p9\") pod \"payload-processing-58f86c7798-6prj5\" (UID: \"767bc03c-400a-4f1b-a4e5-f5c50c3a8ad1\") " pod="openshift-ingress/payload-processing-58f86c7798-6prj5" Jul 06 01:30:29.724865 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:29.724822 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gz2p9\" (UniqueName: \"kubernetes.io/projected/767bc03c-400a-4f1b-a4e5-f5c50c3a8ad1-kube-api-access-gz2p9\") pod \"payload-processing-58f86c7798-6prj5\" (UID: \"767bc03c-400a-4f1b-a4e5-f5c50c3a8ad1\") " pod="openshift-ingress/payload-processing-58f86c7798-6prj5" Jul 06 01:30:29.725070 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:29.724887 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/767bc03c-400a-4f1b-a4e5-f5c50c3a8ad1-plugins-config-volume\") pod \"payload-processing-58f86c7798-6prj5\" (UID: \"767bc03c-400a-4f1b-a4e5-f5c50c3a8ad1\") " pod="openshift-ingress/payload-processing-58f86c7798-6prj5" Jul 06 01:30:29.725672 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:29.725645 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/767bc03c-400a-4f1b-a4e5-f5c50c3a8ad1-plugins-config-volume\") pod \"payload-processing-58f86c7798-6prj5\" (UID: \"767bc03c-400a-4f1b-a4e5-f5c50c3a8ad1\") " pod="openshift-ingress/payload-processing-58f86c7798-6prj5" Jul 06 01:30:29.732607 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:29.732582 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gz2p9\" (UniqueName: \"kubernetes.io/projected/767bc03c-400a-4f1b-a4e5-f5c50c3a8ad1-kube-api-access-gz2p9\") pod \"payload-processing-58f86c7798-6prj5\" (UID: \"767bc03c-400a-4f1b-a4e5-f5c50c3a8ad1\") " pod="openshift-ingress/payload-processing-58f86c7798-6prj5" Jul 06 01:30:29.888435 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:29.888318 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-6prj5" Jul 06 01:30:30.030024 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:30.029998 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-6prj5"] Jul 06 01:30:30.032988 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:30:30.032961 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod767bc03c_400a_4f1b_a4e5_f5c50c3a8ad1.slice/crio-9369986a9da2e954aa2b3eedee2b607efab351a1f63668f9a3173a3dd18bea7e WatchSource:0}: Error finding container 9369986a9da2e954aa2b3eedee2b607efab351a1f63668f9a3173a3dd18bea7e: Status 404 returned error can't find the container with id 9369986a9da2e954aa2b3eedee2b607efab351a1f63668f9a3173a3dd18bea7e Jul 06 01:30:30.357566 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:30.357521 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6787fdf5f-8fjs9" event={"ID":"59b58f6a-6c28-47b0-b99b-dd462f59d915","Type":"ContainerStarted","Data":"627011e2715af7e34efb328f1917abd90c9109b4a2737b985ab16ad6a01f5d35"} Jul 06 01:30:30.358614 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:30.358585 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-6prj5" event={"ID":"767bc03c-400a-4f1b-a4e5-f5c50c3a8ad1","Type":"ContainerStarted","Data":"9369986a9da2e954aa2b3eedee2b607efab351a1f63668f9a3173a3dd18bea7e"} Jul 06 01:30:31.365093 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:31.365020 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6787fdf5f-8fjs9" event={"ID":"59b58f6a-6c28-47b0-b99b-dd462f59d915","Type":"ContainerStarted","Data":"738db463f6d6c8c70d85a37504405dec51c3cbe12f5370b748c5b097dcc1c0d8"} Jul 06 01:30:31.365482 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:31.365115 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-6787fdf5f-8fjs9" Jul 06 01:30:31.381872 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:31.381822 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-6787fdf5f-8fjs9" podStartSLOduration=1.809346516 podStartE2EDuration="3.381804476s" totalImagesPullingTime="1.57245796s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:30:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:30:31.380634301 +0000 UTC m=+603.522193086" watchObservedRunningTime="2026-07-06 01:30:31.381804476 +0000 UTC m=+603.523363259" Jul 06 01:30:32.371580 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:32.371542 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-6prj5" event={"ID":"767bc03c-400a-4f1b-a4e5-f5c50c3a8ad1","Type":"ContainerStarted","Data":"aa46da194753d9c7bee73816d390217c121d8be8dc543ad8ac1407def4eebc85"} Jul 06 01:30:32.372063 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:32.371805 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-6prj5" Jul 06 01:30:32.387577 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:32.387518 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-6prj5" podStartSLOduration=2.052701393 podStartE2EDuration="3.387501949s" totalImagesPullingTime="1.334800556s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:30:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:30:32.386249292 +0000 UTC m=+604.527808091" watchObservedRunningTime="2026-07-06 01:30:32.387501949 +0000 UTC m=+604.529060733" Jul 06 01:30:37.377438 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:37.377407 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-6787fdf5f-8fjs9" Jul 06 01:30:43.379734 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:30:43.379703 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-6prj5" Jul 06 01:31:13.056472 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:13.056432 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-vdpkn"] Jul 06 01:31:13.062324 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:13.062296 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vdpkn" Jul 06 01:31:13.064873 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:13.064847 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-f2mpd\"" Jul 06 01:31:13.064994 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:13.064878 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 06 01:31:13.066536 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:13.065871 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 06 01:31:13.066536 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:13.066153 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 06 01:31:13.069179 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:13.069153 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-vdpkn"] Jul 06 01:31:13.132129 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:13.132090 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf969efe-8d72-4116-a5ce-dd1c4598986c-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vdpkn\" (UID: \"bf969efe-8d72-4116-a5ce-dd1c4598986c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vdpkn" Jul 06 01:31:13.132330 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:13.132138 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf969efe-8d72-4116-a5ce-dd1c4598986c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vdpkn\" (UID: \"bf969efe-8d72-4116-a5ce-dd1c4598986c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vdpkn" Jul 06 01:31:13.132330 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:13.132216 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9gh7\" (UniqueName: \"kubernetes.io/projected/bf969efe-8d72-4116-a5ce-dd1c4598986c-kube-api-access-v9gh7\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vdpkn\" (UID: \"bf969efe-8d72-4116-a5ce-dd1c4598986c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vdpkn" Jul 06 01:31:13.132330 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:13.132250 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf969efe-8d72-4116-a5ce-dd1c4598986c-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vdpkn\" (UID: \"bf969efe-8d72-4116-a5ce-dd1c4598986c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vdpkn" Jul 06 01:31:13.132330 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:13.132278 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf969efe-8d72-4116-a5ce-dd1c4598986c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vdpkn\" (UID: \"bf969efe-8d72-4116-a5ce-dd1c4598986c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vdpkn" Jul 06 01:31:13.132533 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:13.132359 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf969efe-8d72-4116-a5ce-dd1c4598986c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vdpkn\" (UID: \"bf969efe-8d72-4116-a5ce-dd1c4598986c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vdpkn" Jul 06 01:31:13.233314 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:13.233264 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf969efe-8d72-4116-a5ce-dd1c4598986c-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vdpkn\" (UID: \"bf969efe-8d72-4116-a5ce-dd1c4598986c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vdpkn" Jul 06 01:31:13.233314 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:13.233315 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf969efe-8d72-4116-a5ce-dd1c4598986c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vdpkn\" (UID: \"bf969efe-8d72-4116-a5ce-dd1c4598986c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vdpkn" Jul 06 01:31:13.233556 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:13.233402 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf969efe-8d72-4116-a5ce-dd1c4598986c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vdpkn\" (UID: \"bf969efe-8d72-4116-a5ce-dd1c4598986c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vdpkn" Jul 06 01:31:13.233556 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:13.233460 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf969efe-8d72-4116-a5ce-dd1c4598986c-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vdpkn\" (UID: \"bf969efe-8d72-4116-a5ce-dd1c4598986c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vdpkn" Jul 06 01:31:13.233556 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:13.233489 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf969efe-8d72-4116-a5ce-dd1c4598986c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vdpkn\" (UID: \"bf969efe-8d72-4116-a5ce-dd1c4598986c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vdpkn" Jul 06 01:31:13.233556 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:13.233543 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9gh7\" (UniqueName: \"kubernetes.io/projected/bf969efe-8d72-4116-a5ce-dd1c4598986c-kube-api-access-v9gh7\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vdpkn\" (UID: \"bf969efe-8d72-4116-a5ce-dd1c4598986c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vdpkn" Jul 06 01:31:13.233833 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:13.233800 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf969efe-8d72-4116-a5ce-dd1c4598986c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vdpkn\" (UID: \"bf969efe-8d72-4116-a5ce-dd1c4598986c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vdpkn" Jul 06 01:31:13.233961 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:13.233851 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf969efe-8d72-4116-a5ce-dd1c4598986c-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vdpkn\" (UID: \"bf969efe-8d72-4116-a5ce-dd1c4598986c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vdpkn" Jul 06 01:31:13.233961 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:13.233872 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf969efe-8d72-4116-a5ce-dd1c4598986c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vdpkn\" (UID: \"bf969efe-8d72-4116-a5ce-dd1c4598986c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vdpkn" Jul 06 01:31:13.235848 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:13.235813 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf969efe-8d72-4116-a5ce-dd1c4598986c-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vdpkn\" (UID: \"bf969efe-8d72-4116-a5ce-dd1c4598986c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vdpkn" Jul 06 01:31:13.235985 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:13.235966 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf969efe-8d72-4116-a5ce-dd1c4598986c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vdpkn\" (UID: \"bf969efe-8d72-4116-a5ce-dd1c4598986c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vdpkn" Jul 06 01:31:13.241151 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:13.241122 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9gh7\" (UniqueName: \"kubernetes.io/projected/bf969efe-8d72-4116-a5ce-dd1c4598986c-kube-api-access-v9gh7\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vdpkn\" (UID: \"bf969efe-8d72-4116-a5ce-dd1c4598986c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vdpkn" Jul 06 01:31:13.375872 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:13.375784 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vdpkn" Jul 06 01:31:13.522578 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:13.522549 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-vdpkn"] Jul 06 01:31:13.525494 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:31:13.525462 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf969efe_8d72_4116_a5ce_dd1c4598986c.slice/crio-8023bc3981999b3cfb9fb7b5bd11fc5f6eaf7ecc68ffcfc1facb2b6feffd803f WatchSource:0}: Error finding container 8023bc3981999b3cfb9fb7b5bd11fc5f6eaf7ecc68ffcfc1facb2b6feffd803f: Status 404 returned error can't find the container with id 8023bc3981999b3cfb9fb7b5bd11fc5f6eaf7ecc68ffcfc1facb2b6feffd803f Jul 06 01:31:13.530064 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:13.530038 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 01:31:13.575621 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:13.575587 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vdpkn" event={"ID":"bf969efe-8d72-4116-a5ce-dd1c4598986c","Type":"ContainerStarted","Data":"8023bc3981999b3cfb9fb7b5bd11fc5f6eaf7ecc68ffcfc1facb2b6feffd803f"} Jul 06 01:31:15.588433 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:15.588396 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vdpkn" event={"ID":"bf969efe-8d72-4116-a5ce-dd1c4598986c","Type":"ContainerStarted","Data":"012ea464fd976fc6023c15acd8b1c59935c29eafbde8fc4545a0dc2737cd47af"} Jul 06 01:31:15.606955 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:15.606901 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vdpkn" podStartSLOduration=1.564626898 podStartE2EDuration="2.606884908s" totalImagesPullingTime="1.04225801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:31:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:31:15.604567058 +0000 UTC m=+647.746125843" watchObservedRunningTime="2026-07-06 01:31:15.606884908 +0000 UTC m=+647.748443693" Jul 06 01:31:23.376026 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:23.375983 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vdpkn" Jul 06 01:31:23.376481 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:23.376051 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vdpkn" Jul 06 01:31:23.388743 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:23.388714 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vdpkn" Jul 06 01:31:23.646884 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:23.646796 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vdpkn" Jul 06 01:31:29.863131 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:29.863073 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-qdpfl"] Jul 06 01:31:29.866980 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:29.866956 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdpfl" Jul 06 01:31:29.869255 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:29.869226 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 06 01:31:29.884308 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:29.884281 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-qdpfl"] Jul 06 01:31:29.997681 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:29.997650 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/14f71f8c-1a5d-4048-8df9-0bd19bf39eed-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qdpfl\" (UID: \"14f71f8c-1a5d-4048-8df9-0bd19bf39eed\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdpfl" Jul 06 01:31:29.997681 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:29.997680 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/14f71f8c-1a5d-4048-8df9-0bd19bf39eed-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qdpfl\" (UID: \"14f71f8c-1a5d-4048-8df9-0bd19bf39eed\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdpfl" Jul 06 01:31:29.997903 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:29.997705 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14f71f8c-1a5d-4048-8df9-0bd19bf39eed-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qdpfl\" (UID: \"14f71f8c-1a5d-4048-8df9-0bd19bf39eed\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdpfl" Jul 06 01:31:29.997903 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:29.997811 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/14f71f8c-1a5d-4048-8df9-0bd19bf39eed-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qdpfl\" (UID: \"14f71f8c-1a5d-4048-8df9-0bd19bf39eed\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdpfl" Jul 06 01:31:29.997903 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:29.997865 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fsdx\" (UniqueName: \"kubernetes.io/projected/14f71f8c-1a5d-4048-8df9-0bd19bf39eed-kube-api-access-8fsdx\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qdpfl\" (UID: \"14f71f8c-1a5d-4048-8df9-0bd19bf39eed\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdpfl" Jul 06 01:31:29.997903 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:29.997888 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14f71f8c-1a5d-4048-8df9-0bd19bf39eed-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qdpfl\" (UID: \"14f71f8c-1a5d-4048-8df9-0bd19bf39eed\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdpfl" Jul 06 01:31:30.099313 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:30.099279 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/14f71f8c-1a5d-4048-8df9-0bd19bf39eed-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qdpfl\" (UID: \"14f71f8c-1a5d-4048-8df9-0bd19bf39eed\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdpfl" Jul 06 01:31:30.099313 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:30.099315 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/14f71f8c-1a5d-4048-8df9-0bd19bf39eed-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qdpfl\" (UID: \"14f71f8c-1a5d-4048-8df9-0bd19bf39eed\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdpfl" Jul 06 01:31:30.099613 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:30.099340 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14f71f8c-1a5d-4048-8df9-0bd19bf39eed-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qdpfl\" (UID: \"14f71f8c-1a5d-4048-8df9-0bd19bf39eed\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdpfl" Jul 06 01:31:30.099613 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:30.099407 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/14f71f8c-1a5d-4048-8df9-0bd19bf39eed-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qdpfl\" (UID: \"14f71f8c-1a5d-4048-8df9-0bd19bf39eed\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdpfl" Jul 06 01:31:30.099613 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:30.099457 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fsdx\" (UniqueName: \"kubernetes.io/projected/14f71f8c-1a5d-4048-8df9-0bd19bf39eed-kube-api-access-8fsdx\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qdpfl\" (UID: \"14f71f8c-1a5d-4048-8df9-0bd19bf39eed\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdpfl" Jul 06 01:31:30.099613 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:30.099477 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14f71f8c-1a5d-4048-8df9-0bd19bf39eed-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qdpfl\" (UID: \"14f71f8c-1a5d-4048-8df9-0bd19bf39eed\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdpfl" Jul 06 01:31:30.099836 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:30.099771 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/14f71f8c-1a5d-4048-8df9-0bd19bf39eed-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qdpfl\" (UID: \"14f71f8c-1a5d-4048-8df9-0bd19bf39eed\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdpfl" Jul 06 01:31:30.099836 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:30.099782 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/14f71f8c-1a5d-4048-8df9-0bd19bf39eed-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qdpfl\" (UID: \"14f71f8c-1a5d-4048-8df9-0bd19bf39eed\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdpfl" Jul 06 01:31:30.099836 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:30.099827 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14f71f8c-1a5d-4048-8df9-0bd19bf39eed-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qdpfl\" (UID: \"14f71f8c-1a5d-4048-8df9-0bd19bf39eed\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdpfl" Jul 06 01:31:30.101904 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:30.101882 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/14f71f8c-1a5d-4048-8df9-0bd19bf39eed-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qdpfl\" (UID: \"14f71f8c-1a5d-4048-8df9-0bd19bf39eed\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdpfl" Jul 06 01:31:30.102264 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:30.102247 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14f71f8c-1a5d-4048-8df9-0bd19bf39eed-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qdpfl\" (UID: \"14f71f8c-1a5d-4048-8df9-0bd19bf39eed\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdpfl" Jul 06 01:31:30.106965 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:30.106942 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fsdx\" (UniqueName: \"kubernetes.io/projected/14f71f8c-1a5d-4048-8df9-0bd19bf39eed-kube-api-access-8fsdx\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qdpfl\" (UID: \"14f71f8c-1a5d-4048-8df9-0bd19bf39eed\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdpfl" Jul 06 01:31:30.179857 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:30.179761 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdpfl" Jul 06 01:31:30.322581 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:31:30.322545 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14f71f8c_1a5d_4048_8df9_0bd19bf39eed.slice/crio-fff1419f6ac163984d585f874159db499b07a03a626d239962105918a6b8e1da WatchSource:0}: Error finding container fff1419f6ac163984d585f874159db499b07a03a626d239962105918a6b8e1da: Status 404 returned error can't find the container with id fff1419f6ac163984d585f874159db499b07a03a626d239962105918a6b8e1da Jul 06 01:31:30.324233 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:30.324194 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-qdpfl"] Jul 06 01:31:30.673344 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:30.673307 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdpfl" event={"ID":"14f71f8c-1a5d-4048-8df9-0bd19bf39eed","Type":"ContainerStarted","Data":"28ef7cd598efefe762b4336393ef289e09d3252867e3f68f1eef7b88373a085f"} Jul 06 01:31:30.673538 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:30.673371 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdpfl" event={"ID":"14f71f8c-1a5d-4048-8df9-0bd19bf39eed","Type":"ContainerStarted","Data":"fff1419f6ac163984d585f874159db499b07a03a626d239962105918a6b8e1da"} Jul 06 01:31:30.689790 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:30.689731 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdpfl" podStartSLOduration=1.519354286 podStartE2EDuration="1.689710328s" totalImagesPullingTime="170.356042ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:31:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:31:30.689491644 +0000 UTC m=+662.831050460" watchObservedRunningTime="2026-07-06 01:31:30.689710328 +0000 UTC m=+662.831269114" Jul 06 01:31:40.179919 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:40.179878 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdpfl" Jul 06 01:31:40.180323 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:40.179977 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdpfl" Jul 06 01:31:40.192528 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:40.192500 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdpfl" Jul 06 01:31:40.738860 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:31:40.738829 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdpfl" Jul 06 01:32:16.808777 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:32:16.808729 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-78f5c5cf67-pb5kx"] Jul 06 01:32:16.809418 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:32:16.808919 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-78f5c5cf67-pb5kx" podUID="d2c52d2a-3776-467f-afbb-3ba67ed11529" containerName="authorino" containerID="cri-o://8f2788ddcd5119550c4e780fe266964cb8c493be9568242196dd279ce005c9f0" gracePeriod=30 Jul 06 01:32:17.062012 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:32:17.061943 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78f5c5cf67-pb5kx" Jul 06 01:32:17.150300 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:32:17.150260 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4r4xs\" (UniqueName: \"kubernetes.io/projected/d2c52d2a-3776-467f-afbb-3ba67ed11529-kube-api-access-4r4xs\") pod \"d2c52d2a-3776-467f-afbb-3ba67ed11529\" (UID: \"d2c52d2a-3776-467f-afbb-3ba67ed11529\") " Jul 06 01:32:17.150514 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:32:17.150469 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d2c52d2a-3776-467f-afbb-3ba67ed11529-tls-cert\") pod \"d2c52d2a-3776-467f-afbb-3ba67ed11529\" (UID: \"d2c52d2a-3776-467f-afbb-3ba67ed11529\") " Jul 06 01:32:17.152623 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:32:17.152597 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2c52d2a-3776-467f-afbb-3ba67ed11529-kube-api-access-4r4xs" (OuterVolumeSpecName: "kube-api-access-4r4xs") pod "d2c52d2a-3776-467f-afbb-3ba67ed11529" (UID: "d2c52d2a-3776-467f-afbb-3ba67ed11529"). InnerVolumeSpecName "kube-api-access-4r4xs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:32:17.161500 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:32:17.161471 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2c52d2a-3776-467f-afbb-3ba67ed11529-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "d2c52d2a-3776-467f-afbb-3ba67ed11529" (UID: "d2c52d2a-3776-467f-afbb-3ba67ed11529"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 01:32:17.252064 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:32:17.252021 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4r4xs\" (UniqueName: \"kubernetes.io/projected/d2c52d2a-3776-467f-afbb-3ba67ed11529-kube-api-access-4r4xs\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:32:17.252064 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:32:17.252063 2565 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d2c52d2a-3776-467f-afbb-3ba67ed11529-tls-cert\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:32:17.914535 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:32:17.914494 2565 generic.go:356] "Generic (PLEG): container finished" podID="d2c52d2a-3776-467f-afbb-3ba67ed11529" containerID="8f2788ddcd5119550c4e780fe266964cb8c493be9568242196dd279ce005c9f0" exitCode=0 Jul 06 01:32:17.914535 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:32:17.914536 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78f5c5cf67-pb5kx" Jul 06 01:32:17.915063 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:32:17.914559 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-78f5c5cf67-pb5kx" event={"ID":"d2c52d2a-3776-467f-afbb-3ba67ed11529","Type":"ContainerDied","Data":"8f2788ddcd5119550c4e780fe266964cb8c493be9568242196dd279ce005c9f0"} Jul 06 01:32:17.915063 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:32:17.914600 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-78f5c5cf67-pb5kx" event={"ID":"d2c52d2a-3776-467f-afbb-3ba67ed11529","Type":"ContainerDied","Data":"0aa63608119ed4695f24088d52dc9fc8c52e19b63c05fcdfcff7dd81974cc968"} Jul 06 01:32:17.915063 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:32:17.914623 2565 scope.go:117] "RemoveContainer" containerID="8f2788ddcd5119550c4e780fe266964cb8c493be9568242196dd279ce005c9f0" Jul 06 01:32:17.924626 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:32:17.924608 2565 scope.go:117] "RemoveContainer" containerID="8f2788ddcd5119550c4e780fe266964cb8c493be9568242196dd279ce005c9f0" Jul 06 01:32:17.924887 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:32:17.924859 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f2788ddcd5119550c4e780fe266964cb8c493be9568242196dd279ce005c9f0\": container with ID starting with 8f2788ddcd5119550c4e780fe266964cb8c493be9568242196dd279ce005c9f0 not found: ID does not exist" containerID="8f2788ddcd5119550c4e780fe266964cb8c493be9568242196dd279ce005c9f0" Jul 06 01:32:17.924985 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:32:17.924892 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f2788ddcd5119550c4e780fe266964cb8c493be9568242196dd279ce005c9f0"} err="failed to get container status \"8f2788ddcd5119550c4e780fe266964cb8c493be9568242196dd279ce005c9f0\": rpc error: code = NotFound desc = could not find container \"8f2788ddcd5119550c4e780fe266964cb8c493be9568242196dd279ce005c9f0\": container with ID starting with 8f2788ddcd5119550c4e780fe266964cb8c493be9568242196dd279ce005c9f0 not found: ID does not exist" Jul 06 01:32:17.937397 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:32:17.937357 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-78f5c5cf67-pb5kx"] Jul 06 01:32:17.946061 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:32:17.946039 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-78f5c5cf67-pb5kx"] Jul 06 01:32:18.324584 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:32:18.324541 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2c52d2a-3776-467f-afbb-3ba67ed11529" path="/var/lib/kubelet/pods/d2c52d2a-3776-467f-afbb-3ba67ed11529/volumes" Jul 06 01:33:28.863124 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:28.863087 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-797b48b58c-zsdtc"] Jul 06 01:33:28.863707 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:28.863316 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-797b48b58c-zsdtc" podUID="26c168fa-05d6-462a-92b7-923ea184833c" containerName="manager" containerID="cri-o://62dd376a44b898496adda32651ca37ba7d34c792e0730b8457ad7184af1a7a86" gracePeriod=30 Jul 06 01:33:29.109839 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:29.109815 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-797b48b58c-zsdtc" Jul 06 01:33:29.117870 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:29.117812 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qsx2v\" (UniqueName: \"kubernetes.io/projected/26c168fa-05d6-462a-92b7-923ea184833c-kube-api-access-qsx2v\") pod \"26c168fa-05d6-462a-92b7-923ea184833c\" (UID: \"26c168fa-05d6-462a-92b7-923ea184833c\") " Jul 06 01:33:29.120125 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:29.120091 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26c168fa-05d6-462a-92b7-923ea184833c-kube-api-access-qsx2v" (OuterVolumeSpecName: "kube-api-access-qsx2v") pod "26c168fa-05d6-462a-92b7-923ea184833c" (UID: "26c168fa-05d6-462a-92b7-923ea184833c"). InnerVolumeSpecName "kube-api-access-qsx2v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:33:29.218890 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:29.218848 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/26c168fa-05d6-462a-92b7-923ea184833c-webhook-cert\") pod \"26c168fa-05d6-462a-92b7-923ea184833c\" (UID: \"26c168fa-05d6-462a-92b7-923ea184833c\") " Jul 06 01:33:29.219089 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:29.219073 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qsx2v\" (UniqueName: \"kubernetes.io/projected/26c168fa-05d6-462a-92b7-923ea184833c-kube-api-access-qsx2v\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:33:29.220999 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:29.220977 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26c168fa-05d6-462a-92b7-923ea184833c-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "26c168fa-05d6-462a-92b7-923ea184833c" (UID: "26c168fa-05d6-462a-92b7-923ea184833c"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 01:33:29.273125 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:29.273095 2565 generic.go:356] "Generic (PLEG): container finished" podID="26c168fa-05d6-462a-92b7-923ea184833c" containerID="62dd376a44b898496adda32651ca37ba7d34c792e0730b8457ad7184af1a7a86" exitCode=0 Jul 06 01:33:29.273239 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:29.273139 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-797b48b58c-zsdtc" Jul 06 01:33:29.273239 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:29.273152 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-797b48b58c-zsdtc" event={"ID":"26c168fa-05d6-462a-92b7-923ea184833c","Type":"ContainerDied","Data":"62dd376a44b898496adda32651ca37ba7d34c792e0730b8457ad7184af1a7a86"} Jul 06 01:33:29.273239 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:29.273193 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-797b48b58c-zsdtc" event={"ID":"26c168fa-05d6-462a-92b7-923ea184833c","Type":"ContainerDied","Data":"94bc6a2f265ab44315683b9596c08914e7b02df963515995645748e887667e77"} Jul 06 01:33:29.273239 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:29.273214 2565 scope.go:117] "RemoveContainer" containerID="62dd376a44b898496adda32651ca37ba7d34c792e0730b8457ad7184af1a7a86" Jul 06 01:33:29.282275 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:29.282261 2565 scope.go:117] "RemoveContainer" containerID="62dd376a44b898496adda32651ca37ba7d34c792e0730b8457ad7184af1a7a86" Jul 06 01:33:29.282546 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:33:29.282527 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62dd376a44b898496adda32651ca37ba7d34c792e0730b8457ad7184af1a7a86\": container with ID starting with 62dd376a44b898496adda32651ca37ba7d34c792e0730b8457ad7184af1a7a86 not found: ID does not exist" containerID="62dd376a44b898496adda32651ca37ba7d34c792e0730b8457ad7184af1a7a86" Jul 06 01:33:29.282599 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:29.282554 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62dd376a44b898496adda32651ca37ba7d34c792e0730b8457ad7184af1a7a86"} err="failed to get container status \"62dd376a44b898496adda32651ca37ba7d34c792e0730b8457ad7184af1a7a86\": rpc error: code = NotFound desc = could not find container \"62dd376a44b898496adda32651ca37ba7d34c792e0730b8457ad7184af1a7a86\": container with ID starting with 62dd376a44b898496adda32651ca37ba7d34c792e0730b8457ad7184af1a7a86 not found: ID does not exist" Jul 06 01:33:29.294527 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:29.294503 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-797b48b58c-zsdtc"] Jul 06 01:33:29.298592 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:29.298572 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-797b48b58c-zsdtc"] Jul 06 01:33:29.320056 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:29.320034 2565 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/26c168fa-05d6-462a-92b7-923ea184833c-webhook-cert\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:33:30.073845 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:30.073812 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-797b48b58c-45b4g"] Jul 06 01:33:30.074221 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:30.074205 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2c52d2a-3776-467f-afbb-3ba67ed11529" containerName="authorino" Jul 06 01:33:30.074221 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:30.074217 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2c52d2a-3776-467f-afbb-3ba67ed11529" containerName="authorino" Jul 06 01:33:30.074293 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:30.074228 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26c168fa-05d6-462a-92b7-923ea184833c" containerName="manager" Jul 06 01:33:30.074293 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:30.074234 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="26c168fa-05d6-462a-92b7-923ea184833c" containerName="manager" Jul 06 01:33:30.074353 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:30.074302 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2c52d2a-3776-467f-afbb-3ba67ed11529" containerName="authorino" Jul 06 01:33:30.074353 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:30.074310 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="26c168fa-05d6-462a-92b7-923ea184833c" containerName="manager" Jul 06 01:33:30.078614 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:30.078594 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-797b48b58c-45b4g" Jul 06 01:33:30.081018 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:30.080997 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-cqwgs\"" Jul 06 01:33:30.081971 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:30.081952 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 06 01:33:30.082076 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:30.082000 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 06 01:33:30.085687 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:30.085662 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-797b48b58c-45b4g"] Jul 06 01:33:30.124216 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:30.124190 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvkts\" (UniqueName: \"kubernetes.io/projected/2d9be716-b0b9-4983-a9eb-b0912f4d854f-kube-api-access-qvkts\") pod \"maas-controller-797b48b58c-45b4g\" (UID: \"2d9be716-b0b9-4983-a9eb-b0912f4d854f\") " pod="opendatahub/maas-controller-797b48b58c-45b4g" Jul 06 01:33:30.124216 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:30.124219 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2d9be716-b0b9-4983-a9eb-b0912f4d854f-webhook-cert\") pod \"maas-controller-797b48b58c-45b4g\" (UID: \"2d9be716-b0b9-4983-a9eb-b0912f4d854f\") " pod="opendatahub/maas-controller-797b48b58c-45b4g" Jul 06 01:33:30.225455 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:30.225426 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvkts\" (UniqueName: \"kubernetes.io/projected/2d9be716-b0b9-4983-a9eb-b0912f4d854f-kube-api-access-qvkts\") pod \"maas-controller-797b48b58c-45b4g\" (UID: \"2d9be716-b0b9-4983-a9eb-b0912f4d854f\") " pod="opendatahub/maas-controller-797b48b58c-45b4g" Jul 06 01:33:30.225455 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:30.225459 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2d9be716-b0b9-4983-a9eb-b0912f4d854f-webhook-cert\") pod \"maas-controller-797b48b58c-45b4g\" (UID: \"2d9be716-b0b9-4983-a9eb-b0912f4d854f\") " pod="opendatahub/maas-controller-797b48b58c-45b4g" Jul 06 01:33:30.227934 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:30.227907 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2d9be716-b0b9-4983-a9eb-b0912f4d854f-webhook-cert\") pod \"maas-controller-797b48b58c-45b4g\" (UID: \"2d9be716-b0b9-4983-a9eb-b0912f4d854f\") " pod="opendatahub/maas-controller-797b48b58c-45b4g" Jul 06 01:33:30.232961 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:30.232939 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvkts\" (UniqueName: \"kubernetes.io/projected/2d9be716-b0b9-4983-a9eb-b0912f4d854f-kube-api-access-qvkts\") pod \"maas-controller-797b48b58c-45b4g\" (UID: \"2d9be716-b0b9-4983-a9eb-b0912f4d854f\") " pod="opendatahub/maas-controller-797b48b58c-45b4g" Jul 06 01:33:30.324281 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:30.324208 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26c168fa-05d6-462a-92b7-923ea184833c" path="/var/lib/kubelet/pods/26c168fa-05d6-462a-92b7-923ea184833c/volumes" Jul 06 01:33:30.390656 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:30.390627 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-797b48b58c-45b4g" Jul 06 01:33:30.526532 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:30.526424 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-797b48b58c-45b4g"] Jul 06 01:33:30.529585 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:33:30.529557 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d9be716_b0b9_4983_a9eb_b0912f4d854f.slice/crio-13ddeac7fd743fc78b6ec9fc0e9ef7267c461ac1ca6cf7b3756e0767b54565f0 WatchSource:0}: Error finding container 13ddeac7fd743fc78b6ec9fc0e9ef7267c461ac1ca6cf7b3756e0767b54565f0: Status 404 returned error can't find the container with id 13ddeac7fd743fc78b6ec9fc0e9ef7267c461ac1ca6cf7b3756e0767b54565f0 Jul 06 01:33:31.285322 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:31.285227 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-797b48b58c-45b4g" event={"ID":"2d9be716-b0b9-4983-a9eb-b0912f4d854f","Type":"ContainerStarted","Data":"4ccc2560aa9331ca46dbb5ade30f5cfa32edadc2670a9825fc74ef580e0d2220"} Jul 06 01:33:31.285322 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:31.285277 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-797b48b58c-45b4g" event={"ID":"2d9be716-b0b9-4983-a9eb-b0912f4d854f","Type":"ContainerStarted","Data":"13ddeac7fd743fc78b6ec9fc0e9ef7267c461ac1ca6cf7b3756e0767b54565f0"} Jul 06 01:33:31.285773 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:31.285403 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-797b48b58c-45b4g" Jul 06 01:33:31.303322 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:31.303276 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-797b48b58c-45b4g" podStartSLOduration=0.846329045 podStartE2EDuration="1.30325751s" totalImagesPullingTime="456.928465ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:33:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:33:31.301988625 +0000 UTC m=+783.443547427" watchObservedRunningTime="2026-07-06 01:33:31.30325751 +0000 UTC m=+783.444816295" Jul 06 01:33:42.295498 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:33:42.295462 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-797b48b58c-45b4g" Jul 06 01:39:16.531523 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:39:16.531486 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8mgg7"] Jul 06 01:39:16.535028 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:39:16.535012 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8mgg7" Jul 06 01:39:16.537445 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:39:16.537422 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 01:39:16.537580 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:39:16.537491 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-v2k2b\"" Jul 06 01:39:16.537732 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:39:16.537718 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 01:39:16.545121 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:39:16.545094 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8mgg7"] Jul 06 01:39:16.611755 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:39:16.611709 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txgbq\" (UniqueName: \"kubernetes.io/projected/24550d84-e84e-4d58-9918-cbe0b6808cba-kube-api-access-txgbq\") pod \"kuadrant-operator-controller-manager-86878c769f-8mgg7\" (UID: \"24550d84-e84e-4d58-9918-cbe0b6808cba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8mgg7" Jul 06 01:39:16.611968 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:39:16.611854 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/24550d84-e84e-4d58-9918-cbe0b6808cba-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8mgg7\" (UID: \"24550d84-e84e-4d58-9918-cbe0b6808cba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8mgg7" Jul 06 01:39:16.712409 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:39:16.712329 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/24550d84-e84e-4d58-9918-cbe0b6808cba-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8mgg7\" (UID: \"24550d84-e84e-4d58-9918-cbe0b6808cba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8mgg7" Jul 06 01:39:16.712587 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:39:16.712434 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txgbq\" (UniqueName: \"kubernetes.io/projected/24550d84-e84e-4d58-9918-cbe0b6808cba-kube-api-access-txgbq\") pod \"kuadrant-operator-controller-manager-86878c769f-8mgg7\" (UID: \"24550d84-e84e-4d58-9918-cbe0b6808cba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8mgg7" Jul 06 01:39:16.712737 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:39:16.712714 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/24550d84-e84e-4d58-9918-cbe0b6808cba-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8mgg7\" (UID: \"24550d84-e84e-4d58-9918-cbe0b6808cba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8mgg7" Jul 06 01:39:16.721240 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:39:16.721212 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-txgbq\" (UniqueName: \"kubernetes.io/projected/24550d84-e84e-4d58-9918-cbe0b6808cba-kube-api-access-txgbq\") pod \"kuadrant-operator-controller-manager-86878c769f-8mgg7\" (UID: \"24550d84-e84e-4d58-9918-cbe0b6808cba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8mgg7" Jul 06 01:39:16.846551 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:39:16.846458 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8mgg7" Jul 06 01:39:16.993880 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:39:16.993853 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8mgg7"] Jul 06 01:39:16.996897 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:39:16.996866 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24550d84_e84e_4d58_9918_cbe0b6808cba.slice/crio-70c0f5e683479313b653a8d84a6da46af3559ea1a4a56a9e19431f84beb94958 WatchSource:0}: Error finding container 70c0f5e683479313b653a8d84a6da46af3559ea1a4a56a9e19431f84beb94958: Status 404 returned error can't find the container with id 70c0f5e683479313b653a8d84a6da46af3559ea1a4a56a9e19431f84beb94958 Jul 06 01:39:17.027330 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:39:17.027302 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8mgg7" event={"ID":"24550d84-e84e-4d58-9918-cbe0b6808cba","Type":"ContainerStarted","Data":"70c0f5e683479313b653a8d84a6da46af3559ea1a4a56a9e19431f84beb94958"} Jul 06 01:39:18.035741 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:39:18.035700 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8mgg7" event={"ID":"24550d84-e84e-4d58-9918-cbe0b6808cba","Type":"ContainerStarted","Data":"02d51309da3a05c169724aabc62a19a07405c09ebf92df20aeae548be2be06da"} Jul 06 01:39:18.036150 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:39:18.035773 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8mgg7" Jul 06 01:39:18.055924 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:39:18.055867 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8mgg7" podStartSLOduration=2.055850602 podStartE2EDuration="2.055850602s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:39:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:39:18.054581209 +0000 UTC m=+1130.196140017" watchObservedRunningTime="2026-07-06 01:39:18.055850602 +0000 UTC m=+1130.197409387" Jul 06 01:39:29.042882 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:39:29.042846 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8mgg7" Jul 06 01:51:50.264437 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:51:50.264330 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-7c01b8/test-model-7c01b8-kserve-576fdc687f-m7tfd"] Jul 06 01:51:50.268148 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:51:50.268125 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7c01b8/test-model-7c01b8-kserve-576fdc687f-m7tfd" Jul 06 01:51:50.271444 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:51:50.271417 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-7c01b8\"/\"test-model-7c01b8-kserve-self-signed-certs\"" Jul 06 01:51:50.271582 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:51:50.271514 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-7c01b8\"/\"default-dockercfg-8n2kj\"" Jul 06 01:51:50.271582 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:51:50.271549 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-7c01b8\"/\"openshift-service-ca.crt\"" Jul 06 01:51:50.271698 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:51:50.271661 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-7c01b8\"/\"kube-root-ca.crt\"" Jul 06 01:51:50.277708 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:51:50.277684 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-7c01b8/test-model-7c01b8-kserve-576fdc687f-m7tfd"] Jul 06 01:51:50.375464 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:51:50.375436 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2cf42f94-6d93-40bc-9576-520d8da917b1-tls-certs\") pod \"test-model-7c01b8-kserve-576fdc687f-m7tfd\" (UID: \"2cf42f94-6d93-40bc-9576-520d8da917b1\") " pod="ai-tenant-e2e-inf-b-7c01b8/test-model-7c01b8-kserve-576fdc687f-m7tfd" Jul 06 01:51:50.375676 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:51:50.375472 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2cf42f94-6d93-40bc-9576-520d8da917b1-dshm\") pod \"test-model-7c01b8-kserve-576fdc687f-m7tfd\" (UID: \"2cf42f94-6d93-40bc-9576-520d8da917b1\") " pod="ai-tenant-e2e-inf-b-7c01b8/test-model-7c01b8-kserve-576fdc687f-m7tfd" Jul 06 01:51:50.375676 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:51:50.375492 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2cf42f94-6d93-40bc-9576-520d8da917b1-model-cache\") pod \"test-model-7c01b8-kserve-576fdc687f-m7tfd\" (UID: \"2cf42f94-6d93-40bc-9576-520d8da917b1\") " pod="ai-tenant-e2e-inf-b-7c01b8/test-model-7c01b8-kserve-576fdc687f-m7tfd" Jul 06 01:51:50.375676 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:51:50.375525 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2cf42f94-6d93-40bc-9576-520d8da917b1-home\") pod \"test-model-7c01b8-kserve-576fdc687f-m7tfd\" (UID: \"2cf42f94-6d93-40bc-9576-520d8da917b1\") " pod="ai-tenant-e2e-inf-b-7c01b8/test-model-7c01b8-kserve-576fdc687f-m7tfd" Jul 06 01:51:50.375676 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:51:50.375598 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9ffw\" (UniqueName: \"kubernetes.io/projected/2cf42f94-6d93-40bc-9576-520d8da917b1-kube-api-access-x9ffw\") pod \"test-model-7c01b8-kserve-576fdc687f-m7tfd\" (UID: \"2cf42f94-6d93-40bc-9576-520d8da917b1\") " pod="ai-tenant-e2e-inf-b-7c01b8/test-model-7c01b8-kserve-576fdc687f-m7tfd" Jul 06 01:51:50.375848 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:51:50.375693 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2cf42f94-6d93-40bc-9576-520d8da917b1-tmp-dir\") pod \"test-model-7c01b8-kserve-576fdc687f-m7tfd\" (UID: \"2cf42f94-6d93-40bc-9576-520d8da917b1\") " pod="ai-tenant-e2e-inf-b-7c01b8/test-model-7c01b8-kserve-576fdc687f-m7tfd" Jul 06 01:51:50.476429 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:51:50.476371 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2cf42f94-6d93-40bc-9576-520d8da917b1-tmp-dir\") pod \"test-model-7c01b8-kserve-576fdc687f-m7tfd\" (UID: \"2cf42f94-6d93-40bc-9576-520d8da917b1\") " pod="ai-tenant-e2e-inf-b-7c01b8/test-model-7c01b8-kserve-576fdc687f-m7tfd" Jul 06 01:51:50.476612 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:51:50.476461 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2cf42f94-6d93-40bc-9576-520d8da917b1-tls-certs\") pod \"test-model-7c01b8-kserve-576fdc687f-m7tfd\" (UID: \"2cf42f94-6d93-40bc-9576-520d8da917b1\") " pod="ai-tenant-e2e-inf-b-7c01b8/test-model-7c01b8-kserve-576fdc687f-m7tfd" Jul 06 01:51:50.476612 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:51:50.476483 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2cf42f94-6d93-40bc-9576-520d8da917b1-dshm\") pod \"test-model-7c01b8-kserve-576fdc687f-m7tfd\" (UID: \"2cf42f94-6d93-40bc-9576-520d8da917b1\") " pod="ai-tenant-e2e-inf-b-7c01b8/test-model-7c01b8-kserve-576fdc687f-m7tfd" Jul 06 01:51:50.476612 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:51:50.476498 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2cf42f94-6d93-40bc-9576-520d8da917b1-model-cache\") pod \"test-model-7c01b8-kserve-576fdc687f-m7tfd\" (UID: \"2cf42f94-6d93-40bc-9576-520d8da917b1\") " pod="ai-tenant-e2e-inf-b-7c01b8/test-model-7c01b8-kserve-576fdc687f-m7tfd" Jul 06 01:51:50.476612 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:51:50.476519 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2cf42f94-6d93-40bc-9576-520d8da917b1-home\") pod \"test-model-7c01b8-kserve-576fdc687f-m7tfd\" (UID: \"2cf42f94-6d93-40bc-9576-520d8da917b1\") " pod="ai-tenant-e2e-inf-b-7c01b8/test-model-7c01b8-kserve-576fdc687f-m7tfd" Jul 06 01:51:50.476612 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:51:50.476545 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9ffw\" (UniqueName: \"kubernetes.io/projected/2cf42f94-6d93-40bc-9576-520d8da917b1-kube-api-access-x9ffw\") pod \"test-model-7c01b8-kserve-576fdc687f-m7tfd\" (UID: \"2cf42f94-6d93-40bc-9576-520d8da917b1\") " pod="ai-tenant-e2e-inf-b-7c01b8/test-model-7c01b8-kserve-576fdc687f-m7tfd" Jul 06 01:51:50.476861 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:51:50.476777 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2cf42f94-6d93-40bc-9576-520d8da917b1-tmp-dir\") pod \"test-model-7c01b8-kserve-576fdc687f-m7tfd\" (UID: \"2cf42f94-6d93-40bc-9576-520d8da917b1\") " pod="ai-tenant-e2e-inf-b-7c01b8/test-model-7c01b8-kserve-576fdc687f-m7tfd" Jul 06 01:51:50.476901 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:51:50.476878 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2cf42f94-6d93-40bc-9576-520d8da917b1-model-cache\") pod \"test-model-7c01b8-kserve-576fdc687f-m7tfd\" (UID: \"2cf42f94-6d93-40bc-9576-520d8da917b1\") " pod="ai-tenant-e2e-inf-b-7c01b8/test-model-7c01b8-kserve-576fdc687f-m7tfd" Jul 06 01:51:50.476960 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:51:50.476924 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2cf42f94-6d93-40bc-9576-520d8da917b1-home\") pod \"test-model-7c01b8-kserve-576fdc687f-m7tfd\" (UID: \"2cf42f94-6d93-40bc-9576-520d8da917b1\") " pod="ai-tenant-e2e-inf-b-7c01b8/test-model-7c01b8-kserve-576fdc687f-m7tfd" Jul 06 01:51:50.478805 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:51:50.478783 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2cf42f94-6d93-40bc-9576-520d8da917b1-dshm\") pod \"test-model-7c01b8-kserve-576fdc687f-m7tfd\" (UID: \"2cf42f94-6d93-40bc-9576-520d8da917b1\") " pod="ai-tenant-e2e-inf-b-7c01b8/test-model-7c01b8-kserve-576fdc687f-m7tfd" Jul 06 01:51:50.479033 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:51:50.479017 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2cf42f94-6d93-40bc-9576-520d8da917b1-tls-certs\") pod \"test-model-7c01b8-kserve-576fdc687f-m7tfd\" (UID: \"2cf42f94-6d93-40bc-9576-520d8da917b1\") " pod="ai-tenant-e2e-inf-b-7c01b8/test-model-7c01b8-kserve-576fdc687f-m7tfd" Jul 06 01:51:50.484988 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:51:50.484966 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9ffw\" (UniqueName: \"kubernetes.io/projected/2cf42f94-6d93-40bc-9576-520d8da917b1-kube-api-access-x9ffw\") pod \"test-model-7c01b8-kserve-576fdc687f-m7tfd\" (UID: \"2cf42f94-6d93-40bc-9576-520d8da917b1\") " pod="ai-tenant-e2e-inf-b-7c01b8/test-model-7c01b8-kserve-576fdc687f-m7tfd" Jul 06 01:51:50.578659 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:51:50.578587 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7c01b8/test-model-7c01b8-kserve-576fdc687f-m7tfd" Jul 06 01:51:50.716157 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:51:50.716132 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-7c01b8/test-model-7c01b8-kserve-576fdc687f-m7tfd"] Jul 06 01:51:50.718259 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:51:50.718223 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2cf42f94_6d93_40bc_9576_520d8da917b1.slice/crio-4537575ed198f7ce556250206b15486f3190c2b2acd1b5567a1d8f7a9e2d9f4d WatchSource:0}: Error finding container 4537575ed198f7ce556250206b15486f3190c2b2acd1b5567a1d8f7a9e2d9f4d: Status 404 returned error can't find the container with id 4537575ed198f7ce556250206b15486f3190c2b2acd1b5567a1d8f7a9e2d9f4d Jul 06 01:51:50.852979 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:51:50.852886 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7c01b8/test-model-7c01b8-kserve-576fdc687f-m7tfd" event={"ID":"2cf42f94-6d93-40bc-9576-520d8da917b1","Type":"ContainerStarted","Data":"d78b0be6a5dd9edb8b8d30affa21a7e313ebe3ab3ba8eee224f58690cad03df2"} Jul 06 01:51:50.852979 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:51:50.852933 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7c01b8/test-model-7c01b8-kserve-576fdc687f-m7tfd" event={"ID":"2cf42f94-6d93-40bc-9576-520d8da917b1","Type":"ContainerStarted","Data":"4537575ed198f7ce556250206b15486f3190c2b2acd1b5567a1d8f7a9e2d9f4d"} Jul 06 01:51:50.869900 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:51:50.869848 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-7c01b8/test-model-7c01b8-kserve-576fdc687f-m7tfd" podStartSLOduration=0.86983374 podStartE2EDuration="869.83374ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:51:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:51:50.868184269 +0000 UTC m=+1883.009743054" watchObservedRunningTime="2026-07-06 01:51:50.86983374 +0000 UTC m=+1883.011392523" Jul 06 01:52:00.578998 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:52:00.578955 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-7c01b8/test-model-7c01b8-kserve-576fdc687f-m7tfd" Jul 06 01:52:00.578998 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:52:00.579013 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-7c01b8/test-model-7c01b8-kserve-576fdc687f-m7tfd" Jul 06 01:52:00.591410 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:52:00.591362 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-7c01b8/test-model-7c01b8-kserve-576fdc687f-m7tfd" Jul 06 01:52:00.923138 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:52:00.923042 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-7c01b8/test-model-7c01b8-kserve-576fdc687f-m7tfd" Jul 06 01:52:09.059187 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:52:09.059157 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-7c01b8/test-model-7c01b8-kserve-576fdc687f-m7tfd"] Jul 06 01:52:24.080487 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:52:24.080428 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-7c01b8/test-model-7c01b8-kserve-576fdc687f-m7tfd" podUID="2cf42f94-6d93-40bc-9576-520d8da917b1" containerName="main" containerID="cri-o://d78b0be6a5dd9edb8b8d30affa21a7e313ebe3ab3ba8eee224f58690cad03df2" gracePeriod=45 Jul 06 01:52:24.337480 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:52:24.337419 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7c01b8/test-model-7c01b8-kserve-576fdc687f-m7tfd" Jul 06 01:52:24.391628 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:52:24.391600 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2cf42f94-6d93-40bc-9576-520d8da917b1-tls-certs\") pod \"2cf42f94-6d93-40bc-9576-520d8da917b1\" (UID: \"2cf42f94-6d93-40bc-9576-520d8da917b1\") " Jul 06 01:52:24.391797 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:52:24.391648 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2cf42f94-6d93-40bc-9576-520d8da917b1-model-cache\") pod \"2cf42f94-6d93-40bc-9576-520d8da917b1\" (UID: \"2cf42f94-6d93-40bc-9576-520d8da917b1\") " Jul 06 01:52:24.391797 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:52:24.391665 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2cf42f94-6d93-40bc-9576-520d8da917b1-home\") pod \"2cf42f94-6d93-40bc-9576-520d8da917b1\" (UID: \"2cf42f94-6d93-40bc-9576-520d8da917b1\") " Jul 06 01:52:24.391797 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:52:24.391681 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x9ffw\" (UniqueName: \"kubernetes.io/projected/2cf42f94-6d93-40bc-9576-520d8da917b1-kube-api-access-x9ffw\") pod \"2cf42f94-6d93-40bc-9576-520d8da917b1\" (UID: \"2cf42f94-6d93-40bc-9576-520d8da917b1\") " Jul 06 01:52:24.391797 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:52:24.391698 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2cf42f94-6d93-40bc-9576-520d8da917b1-dshm\") pod \"2cf42f94-6d93-40bc-9576-520d8da917b1\" (UID: \"2cf42f94-6d93-40bc-9576-520d8da917b1\") " Jul 06 01:52:24.391797 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:52:24.391713 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2cf42f94-6d93-40bc-9576-520d8da917b1-tmp-dir\") pod \"2cf42f94-6d93-40bc-9576-520d8da917b1\" (UID: \"2cf42f94-6d93-40bc-9576-520d8da917b1\") " Jul 06 01:52:24.392054 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:52:24.391954 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cf42f94-6d93-40bc-9576-520d8da917b1-home" (OuterVolumeSpecName: "home") pod "2cf42f94-6d93-40bc-9576-520d8da917b1" (UID: "2cf42f94-6d93-40bc-9576-520d8da917b1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:52:24.392054 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:52:24.391965 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cf42f94-6d93-40bc-9576-520d8da917b1-model-cache" (OuterVolumeSpecName: "model-cache") pod "2cf42f94-6d93-40bc-9576-520d8da917b1" (UID: "2cf42f94-6d93-40bc-9576-520d8da917b1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:52:24.392155 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:52:24.392134 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cf42f94-6d93-40bc-9576-520d8da917b1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2cf42f94-6d93-40bc-9576-520d8da917b1" (UID: "2cf42f94-6d93-40bc-9576-520d8da917b1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:52:24.392155 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:52:24.392140 2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2cf42f94-6d93-40bc-9576-520d8da917b1-model-cache\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:52:24.392251 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:52:24.392161 2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2cf42f94-6d93-40bc-9576-520d8da917b1-home\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:52:24.394006 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:52:24.393974 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cf42f94-6d93-40bc-9576-520d8da917b1-dshm" (OuterVolumeSpecName: "dshm") pod "2cf42f94-6d93-40bc-9576-520d8da917b1" (UID: "2cf42f94-6d93-40bc-9576-520d8da917b1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:52:24.394117 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:52:24.394008 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2cf42f94-6d93-40bc-9576-520d8da917b1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2cf42f94-6d93-40bc-9576-520d8da917b1" (UID: "2cf42f94-6d93-40bc-9576-520d8da917b1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 01:52:24.394320 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:52:24.394292 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cf42f94-6d93-40bc-9576-520d8da917b1-kube-api-access-x9ffw" (OuterVolumeSpecName: "kube-api-access-x9ffw") pod "2cf42f94-6d93-40bc-9576-520d8da917b1" (UID: "2cf42f94-6d93-40bc-9576-520d8da917b1"). InnerVolumeSpecName "kube-api-access-x9ffw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:52:24.493216 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:52:24.493184 2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2cf42f94-6d93-40bc-9576-520d8da917b1-tls-certs\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:52:24.493216 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:52:24.493213 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9ffw\" (UniqueName: \"kubernetes.io/projected/2cf42f94-6d93-40bc-9576-520d8da917b1-kube-api-access-x9ffw\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:52:24.493216 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:52:24.493224 2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2cf42f94-6d93-40bc-9576-520d8da917b1-dshm\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:52:24.493482 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:52:24.493258 2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2cf42f94-6d93-40bc-9576-520d8da917b1-tmp-dir\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:52:25.031791 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:52:25.031756 2565 generic.go:356] "Generic (PLEG): container finished" podID="2cf42f94-6d93-40bc-9576-520d8da917b1" containerID="d78b0be6a5dd9edb8b8d30affa21a7e313ebe3ab3ba8eee224f58690cad03df2" exitCode=0 Jul 06 01:52:25.031963 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:52:25.031801 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7c01b8/test-model-7c01b8-kserve-576fdc687f-m7tfd" Jul 06 01:52:25.031963 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:52:25.031843 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7c01b8/test-model-7c01b8-kserve-576fdc687f-m7tfd" event={"ID":"2cf42f94-6d93-40bc-9576-520d8da917b1","Type":"ContainerDied","Data":"d78b0be6a5dd9edb8b8d30affa21a7e313ebe3ab3ba8eee224f58690cad03df2"} Jul 06 01:52:25.031963 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:52:25.031888 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7c01b8/test-model-7c01b8-kserve-576fdc687f-m7tfd" event={"ID":"2cf42f94-6d93-40bc-9576-520d8da917b1","Type":"ContainerDied","Data":"4537575ed198f7ce556250206b15486f3190c2b2acd1b5567a1d8f7a9e2d9f4d"} Jul 06 01:52:25.031963 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:52:25.031907 2565 scope.go:117] "RemoveContainer" containerID="d78b0be6a5dd9edb8b8d30affa21a7e313ebe3ab3ba8eee224f58690cad03df2" Jul 06 01:52:25.041135 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:52:25.041117 2565 scope.go:117] "RemoveContainer" containerID="d78b0be6a5dd9edb8b8d30affa21a7e313ebe3ab3ba8eee224f58690cad03df2" Jul 06 01:52:25.041430 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:52:25.041411 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d78b0be6a5dd9edb8b8d30affa21a7e313ebe3ab3ba8eee224f58690cad03df2\": container with ID starting with d78b0be6a5dd9edb8b8d30affa21a7e313ebe3ab3ba8eee224f58690cad03df2 not found: ID does not exist" containerID="d78b0be6a5dd9edb8b8d30affa21a7e313ebe3ab3ba8eee224f58690cad03df2" Jul 06 01:52:25.041498 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:52:25.041438 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d78b0be6a5dd9edb8b8d30affa21a7e313ebe3ab3ba8eee224f58690cad03df2"} err="failed to get container status \"d78b0be6a5dd9edb8b8d30affa21a7e313ebe3ab3ba8eee224f58690cad03df2\": rpc error: code = NotFound desc = could not find container \"d78b0be6a5dd9edb8b8d30affa21a7e313ebe3ab3ba8eee224f58690cad03df2\": container with ID starting with d78b0be6a5dd9edb8b8d30affa21a7e313ebe3ab3ba8eee224f58690cad03df2 not found: ID does not exist" Jul 06 01:52:25.054463 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:52:25.054441 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-7c01b8/test-model-7c01b8-kserve-576fdc687f-m7tfd"] Jul 06 01:52:25.059260 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:52:25.059237 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-7c01b8/test-model-7c01b8-kserve-576fdc687f-m7tfd"] Jul 06 01:52:26.323921 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:52:26.323885 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2cf42f94-6d93-40bc-9576-520d8da917b1" path="/var/lib/kubelet/pods/2cf42f94-6d93-40bc-9576-520d8da917b1/volumes" Jul 06 01:54:39.271212 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:54:39.271169 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-ed9e8f/auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh"] Jul 06 01:54:39.271828 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:54:39.271773 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cf42f94-6d93-40bc-9576-520d8da917b1" containerName="main" Jul 06 01:54:39.271828 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:54:39.271791 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cf42f94-6d93-40bc-9576-520d8da917b1" containerName="main" Jul 06 01:54:39.271958 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:54:39.271854 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cf42f94-6d93-40bc-9576-520d8da917b1" containerName="main" Jul 06 01:54:39.274848 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:54:39.274832 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ed9e8f/auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh" Jul 06 01:54:39.278717 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:54:39.278650 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ed9e8f\"/\"kube-root-ca.crt\"" Jul 06 01:54:39.278717 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:54:39.278650 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ed9e8f\"/\"auth-test-model-ed9e8f-kserve-self-signed-certs\"" Jul 06 01:54:39.278948 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:54:39.278611 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ed9e8f\"/\"openshift-service-ca.crt\"" Jul 06 01:54:39.278948 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:54:39.278660 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ed9e8f\"/\"default-dockercfg-rxhjd\"" Jul 06 01:54:39.290509 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:54:39.287298 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ed9e8f/auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh"] Jul 06 01:54:39.366004 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:54:39.365965 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f224bccf-fbe5-4782-bc98-85ea7b7aea32-tmp-dir\") pod \"auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh\" (UID: \"f224bccf-fbe5-4782-bc98-85ea7b7aea32\") " pod="ai-tenant-e2e-shared-b-ed9e8f/auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh" Jul 06 01:54:39.366224 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:54:39.366133 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlhcb\" (UniqueName: \"kubernetes.io/projected/f224bccf-fbe5-4782-bc98-85ea7b7aea32-kube-api-access-rlhcb\") pod \"auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh\" (UID: \"f224bccf-fbe5-4782-bc98-85ea7b7aea32\") " pod="ai-tenant-e2e-shared-b-ed9e8f/auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh" Jul 06 01:54:39.366224 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:54:39.366167 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f224bccf-fbe5-4782-bc98-85ea7b7aea32-tls-certs\") pod \"auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh\" (UID: \"f224bccf-fbe5-4782-bc98-85ea7b7aea32\") " pod="ai-tenant-e2e-shared-b-ed9e8f/auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh" Jul 06 01:54:39.366224 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:54:39.366222 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f224bccf-fbe5-4782-bc98-85ea7b7aea32-model-cache\") pod \"auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh\" (UID: \"f224bccf-fbe5-4782-bc98-85ea7b7aea32\") " pod="ai-tenant-e2e-shared-b-ed9e8f/auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh" Jul 06 01:54:39.366426 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:54:39.366276 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f224bccf-fbe5-4782-bc98-85ea7b7aea32-home\") pod \"auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh\" (UID: \"f224bccf-fbe5-4782-bc98-85ea7b7aea32\") " pod="ai-tenant-e2e-shared-b-ed9e8f/auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh" Jul 06 01:54:39.366426 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:54:39.366324 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f224bccf-fbe5-4782-bc98-85ea7b7aea32-dshm\") pod \"auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh\" (UID: \"f224bccf-fbe5-4782-bc98-85ea7b7aea32\") " pod="ai-tenant-e2e-shared-b-ed9e8f/auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh" Jul 06 01:54:39.467857 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:54:39.467812 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f224bccf-fbe5-4782-bc98-85ea7b7aea32-tmp-dir\") pod \"auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh\" (UID: \"f224bccf-fbe5-4782-bc98-85ea7b7aea32\") " pod="ai-tenant-e2e-shared-b-ed9e8f/auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh" Jul 06 01:54:39.468079 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:54:39.467895 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlhcb\" (UniqueName: \"kubernetes.io/projected/f224bccf-fbe5-4782-bc98-85ea7b7aea32-kube-api-access-rlhcb\") pod \"auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh\" (UID: \"f224bccf-fbe5-4782-bc98-85ea7b7aea32\") " pod="ai-tenant-e2e-shared-b-ed9e8f/auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh" Jul 06 01:54:39.468079 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:54:39.467922 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f224bccf-fbe5-4782-bc98-85ea7b7aea32-tls-certs\") pod \"auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh\" (UID: \"f224bccf-fbe5-4782-bc98-85ea7b7aea32\") " pod="ai-tenant-e2e-shared-b-ed9e8f/auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh" Jul 06 01:54:39.468079 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:54:39.467951 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f224bccf-fbe5-4782-bc98-85ea7b7aea32-model-cache\") pod \"auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh\" (UID: \"f224bccf-fbe5-4782-bc98-85ea7b7aea32\") " pod="ai-tenant-e2e-shared-b-ed9e8f/auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh" Jul 06 01:54:39.468079 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:54:39.467984 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f224bccf-fbe5-4782-bc98-85ea7b7aea32-home\") pod \"auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh\" (UID: \"f224bccf-fbe5-4782-bc98-85ea7b7aea32\") " pod="ai-tenant-e2e-shared-b-ed9e8f/auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh" Jul 06 01:54:39.468309 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:54:39.468132 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f224bccf-fbe5-4782-bc98-85ea7b7aea32-dshm\") pod \"auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh\" (UID: \"f224bccf-fbe5-4782-bc98-85ea7b7aea32\") " pod="ai-tenant-e2e-shared-b-ed9e8f/auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh" Jul 06 01:54:39.468364 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:54:39.468305 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f224bccf-fbe5-4782-bc98-85ea7b7aea32-tmp-dir\") pod \"auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh\" (UID: \"f224bccf-fbe5-4782-bc98-85ea7b7aea32\") " pod="ai-tenant-e2e-shared-b-ed9e8f/auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh" Jul 06 01:54:39.468364 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:54:39.468346 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f224bccf-fbe5-4782-bc98-85ea7b7aea32-home\") pod \"auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh\" (UID: \"f224bccf-fbe5-4782-bc98-85ea7b7aea32\") " pod="ai-tenant-e2e-shared-b-ed9e8f/auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh" Jul 06 01:54:39.468498 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:54:39.468422 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f224bccf-fbe5-4782-bc98-85ea7b7aea32-model-cache\") pod \"auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh\" (UID: \"f224bccf-fbe5-4782-bc98-85ea7b7aea32\") " pod="ai-tenant-e2e-shared-b-ed9e8f/auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh" Jul 06 01:54:39.470444 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:54:39.470423 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f224bccf-fbe5-4782-bc98-85ea7b7aea32-dshm\") pod \"auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh\" (UID: \"f224bccf-fbe5-4782-bc98-85ea7b7aea32\") " pod="ai-tenant-e2e-shared-b-ed9e8f/auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh" Jul 06 01:54:39.470589 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:54:39.470572 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f224bccf-fbe5-4782-bc98-85ea7b7aea32-tls-certs\") pod \"auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh\" (UID: \"f224bccf-fbe5-4782-bc98-85ea7b7aea32\") " pod="ai-tenant-e2e-shared-b-ed9e8f/auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh" Jul 06 01:54:39.476420 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:54:39.476396 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlhcb\" (UniqueName: \"kubernetes.io/projected/f224bccf-fbe5-4782-bc98-85ea7b7aea32-kube-api-access-rlhcb\") pod \"auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh\" (UID: \"f224bccf-fbe5-4782-bc98-85ea7b7aea32\") " pod="ai-tenant-e2e-shared-b-ed9e8f/auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh" Jul 06 01:54:39.589507 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:54:39.589420 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ed9e8f/auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh" Jul 06 01:54:39.728871 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:54:39.728843 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ed9e8f/auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh"] Jul 06 01:54:39.730789 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:54:39.730764 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf224bccf_fbe5_4782_bc98_85ea7b7aea32.slice/crio-16c9b2ef4954563f94d835850f14db11bd68198744f506c6017349fe9c911db0 WatchSource:0}: Error finding container 16c9b2ef4954563f94d835850f14db11bd68198744f506c6017349fe9c911db0: Status 404 returned error can't find the container with id 16c9b2ef4954563f94d835850f14db11bd68198744f506c6017349fe9c911db0 Jul 06 01:54:40.722685 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:54:40.722648 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ed9e8f/auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh" event={"ID":"f224bccf-fbe5-4782-bc98-85ea7b7aea32","Type":"ContainerStarted","Data":"cab4609f2557bf88456b7f29f01d2ce7f92e353a4cba181d1a6a5261d6a33df6"} Jul 06 01:54:40.723098 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:54:40.722696 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ed9e8f/auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh" event={"ID":"f224bccf-fbe5-4782-bc98-85ea7b7aea32","Type":"ContainerStarted","Data":"16c9b2ef4954563f94d835850f14db11bd68198744f506c6017349fe9c911db0"} Jul 06 01:54:40.738764 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:54:40.738710 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-ed9e8f/auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh" podStartSLOduration=1.738693963 podStartE2EDuration="1.738693963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:54:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:54:40.737557945 +0000 UTC m=+2052.879116729" watchObservedRunningTime="2026-07-06 01:54:40.738693963 +0000 UTC m=+2052.880252748" Jul 06 01:54:49.590076 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:54:49.589982 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-ed9e8f/auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh" Jul 06 01:54:49.590076 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:54:49.590051 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-ed9e8f/auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh" Jul 06 01:54:49.602838 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:54:49.602813 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-ed9e8f/auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh" Jul 06 01:54:49.778727 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:54:49.778695 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-ed9e8f/auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh" Jul 06 01:56:38.986551 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:56:38.986471 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ed9e8f/auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh"] Jul 06 01:56:54.007669 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:56:54.007621 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-ed9e8f/auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh" podUID="f224bccf-fbe5-4782-bc98-85ea7b7aea32" containerName="main" containerID="cri-o://cab4609f2557bf88456b7f29f01d2ce7f92e353a4cba181d1a6a5261d6a33df6" gracePeriod=45 Jul 06 01:56:54.262879 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:56:54.262814 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ed9e8f/auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh" Jul 06 01:56:54.264767 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:56:54.264747 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f224bccf-fbe5-4782-bc98-85ea7b7aea32-dshm\") pod \"f224bccf-fbe5-4782-bc98-85ea7b7aea32\" (UID: \"f224bccf-fbe5-4782-bc98-85ea7b7aea32\") " Jul 06 01:56:54.265175 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:56:54.265154 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f224bccf-fbe5-4782-bc98-85ea7b7aea32-tmp-dir\") pod \"f224bccf-fbe5-4782-bc98-85ea7b7aea32\" (UID: \"f224bccf-fbe5-4782-bc98-85ea7b7aea32\") " Jul 06 01:56:54.265465 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:56:54.265446 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rlhcb\" (UniqueName: \"kubernetes.io/projected/f224bccf-fbe5-4782-bc98-85ea7b7aea32-kube-api-access-rlhcb\") pod \"f224bccf-fbe5-4782-bc98-85ea7b7aea32\" (UID: \"f224bccf-fbe5-4782-bc98-85ea7b7aea32\") " Jul 06 01:56:54.265595 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:56:54.265581 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f224bccf-fbe5-4782-bc98-85ea7b7aea32-tls-certs\") pod \"f224bccf-fbe5-4782-bc98-85ea7b7aea32\" (UID: \"f224bccf-fbe5-4782-bc98-85ea7b7aea32\") " Jul 06 01:56:54.265777 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:56:54.265763 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f224bccf-fbe5-4782-bc98-85ea7b7aea32-home\") pod \"f224bccf-fbe5-4782-bc98-85ea7b7aea32\" (UID: \"f224bccf-fbe5-4782-bc98-85ea7b7aea32\") " Jul 06 01:56:54.265888 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:56:54.265876 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f224bccf-fbe5-4782-bc98-85ea7b7aea32-model-cache\") pod \"f224bccf-fbe5-4782-bc98-85ea7b7aea32\" (UID: \"f224bccf-fbe5-4782-bc98-85ea7b7aea32\") " Jul 06 01:56:54.266690 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:56:54.266659 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f224bccf-fbe5-4782-bc98-85ea7b7aea32-model-cache" (OuterVolumeSpecName: "model-cache") pod "f224bccf-fbe5-4782-bc98-85ea7b7aea32" (UID: "f224bccf-fbe5-4782-bc98-85ea7b7aea32"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:56:54.266969 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:56:54.266952 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f224bccf-fbe5-4782-bc98-85ea7b7aea32-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f224bccf-fbe5-4782-bc98-85ea7b7aea32" (UID: "f224bccf-fbe5-4782-bc98-85ea7b7aea32"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:56:54.268230 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:56:54.268204 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f224bccf-fbe5-4782-bc98-85ea7b7aea32-home" (OuterVolumeSpecName: "home") pod "f224bccf-fbe5-4782-bc98-85ea7b7aea32" (UID: "f224bccf-fbe5-4782-bc98-85ea7b7aea32"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:56:54.269574 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:56:54.269548 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f224bccf-fbe5-4782-bc98-85ea7b7aea32-dshm" (OuterVolumeSpecName: "dshm") pod "f224bccf-fbe5-4782-bc98-85ea7b7aea32" (UID: "f224bccf-fbe5-4782-bc98-85ea7b7aea32"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:56:54.269959 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:56:54.269926 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f224bccf-fbe5-4782-bc98-85ea7b7aea32-kube-api-access-rlhcb" (OuterVolumeSpecName: "kube-api-access-rlhcb") pod "f224bccf-fbe5-4782-bc98-85ea7b7aea32" (UID: "f224bccf-fbe5-4782-bc98-85ea7b7aea32"). InnerVolumeSpecName "kube-api-access-rlhcb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:56:54.275023 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:56:54.274999 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f224bccf-fbe5-4782-bc98-85ea7b7aea32-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f224bccf-fbe5-4782-bc98-85ea7b7aea32" (UID: "f224bccf-fbe5-4782-bc98-85ea7b7aea32"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 01:56:54.367665 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:56:54.367636 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rlhcb\" (UniqueName: \"kubernetes.io/projected/f224bccf-fbe5-4782-bc98-85ea7b7aea32-kube-api-access-rlhcb\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:56:54.367665 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:56:54.367660 2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f224bccf-fbe5-4782-bc98-85ea7b7aea32-tls-certs\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:56:54.367665 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:56:54.367668 2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f224bccf-fbe5-4782-bc98-85ea7b7aea32-home\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:56:54.367862 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:56:54.367676 2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f224bccf-fbe5-4782-bc98-85ea7b7aea32-model-cache\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:56:54.367862 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:56:54.367684 2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f224bccf-fbe5-4782-bc98-85ea7b7aea32-dshm\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:56:54.367862 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:56:54.367691 2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f224bccf-fbe5-4782-bc98-85ea7b7aea32-tmp-dir\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:56:54.414300 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:56:54.414265 2565 generic.go:356] "Generic (PLEG): container finished" podID="f224bccf-fbe5-4782-bc98-85ea7b7aea32" containerID="cab4609f2557bf88456b7f29f01d2ce7f92e353a4cba181d1a6a5261d6a33df6" exitCode=0 Jul 06 01:56:54.414478 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:56:54.414328 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ed9e8f/auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh" Jul 06 01:56:54.414478 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:56:54.414353 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ed9e8f/auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh" event={"ID":"f224bccf-fbe5-4782-bc98-85ea7b7aea32","Type":"ContainerDied","Data":"cab4609f2557bf88456b7f29f01d2ce7f92e353a4cba181d1a6a5261d6a33df6"} Jul 06 01:56:54.414478 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:56:54.414421 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ed9e8f/auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh" event={"ID":"f224bccf-fbe5-4782-bc98-85ea7b7aea32","Type":"ContainerDied","Data":"16c9b2ef4954563f94d835850f14db11bd68198744f506c6017349fe9c911db0"} Jul 06 01:56:54.414478 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:56:54.414445 2565 scope.go:117] "RemoveContainer" containerID="cab4609f2557bf88456b7f29f01d2ce7f92e353a4cba181d1a6a5261d6a33df6" Jul 06 01:56:54.424002 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:56:54.423986 2565 scope.go:117] "RemoveContainer" containerID="cab4609f2557bf88456b7f29f01d2ce7f92e353a4cba181d1a6a5261d6a33df6" Jul 06 01:56:54.424254 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:56:54.424234 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cab4609f2557bf88456b7f29f01d2ce7f92e353a4cba181d1a6a5261d6a33df6\": container with ID starting with cab4609f2557bf88456b7f29f01d2ce7f92e353a4cba181d1a6a5261d6a33df6 not found: ID does not exist" containerID="cab4609f2557bf88456b7f29f01d2ce7f92e353a4cba181d1a6a5261d6a33df6" Jul 06 01:56:54.424318 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:56:54.424261 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cab4609f2557bf88456b7f29f01d2ce7f92e353a4cba181d1a6a5261d6a33df6"} err="failed to get container status \"cab4609f2557bf88456b7f29f01d2ce7f92e353a4cba181d1a6a5261d6a33df6\": rpc error: code = NotFound desc = could not find container \"cab4609f2557bf88456b7f29f01d2ce7f92e353a4cba181d1a6a5261d6a33df6\": container with ID starting with cab4609f2557bf88456b7f29f01d2ce7f92e353a4cba181d1a6a5261d6a33df6 not found: ID does not exist" Jul 06 01:56:54.431914 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:56:54.431889 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ed9e8f/auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh"] Jul 06 01:56:54.440121 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:56:54.440102 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-ed9e8f/auth-test-model-ed9e8f-kserve-6c7f556b7-qphnh"] Jul 06 01:56:56.323956 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:56:56.323911 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f224bccf-fbe5-4782-bc98-85ea7b7aea32" path="/var/lib/kubelet/pods/f224bccf-fbe5-4782-bc98-85ea7b7aea32/volumes" Jul 06 01:58:05.348575 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:58:05.348539 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-5afb9c/rate-test-model-5afb9c-kserve-945956779-fnwkn"] Jul 06 01:58:05.349165 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:58:05.349137 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f224bccf-fbe5-4782-bc98-85ea7b7aea32" containerName="main" Jul 06 01:58:05.349281 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:58:05.349163 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f224bccf-fbe5-4782-bc98-85ea7b7aea32" containerName="main" Jul 06 01:58:05.349361 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:58:05.349344 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f224bccf-fbe5-4782-bc98-85ea7b7aea32" containerName="main" Jul 06 01:58:05.352620 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:58:05.352604 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5afb9c/rate-test-model-5afb9c-kserve-945956779-fnwkn" Jul 06 01:58:05.355139 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:58:05.355120 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5afb9c\"/\"kube-root-ca.crt\"" Jul 06 01:58:05.356329 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:58:05.356310 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5afb9c\"/\"default-dockercfg-fbqxk\"" Jul 06 01:58:05.356329 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:58:05.356320 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5afb9c\"/\"openshift-service-ca.crt\"" Jul 06 01:58:05.356492 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:58:05.356320 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5afb9c\"/\"rate-test-model-5afb9c-kserve-self-signed-certs\"" Jul 06 01:58:05.362798 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:58:05.362778 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5afb9c/rate-test-model-5afb9c-kserve-945956779-fnwkn"] Jul 06 01:58:05.526777 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:58:05.526743 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/364de976-3b02-4e3c-9227-5f55bd3bed70-tls-certs\") pod \"rate-test-model-5afb9c-kserve-945956779-fnwkn\" (UID: \"364de976-3b02-4e3c-9227-5f55bd3bed70\") " pod="ai-tenant-e2e-shared-a-5afb9c/rate-test-model-5afb9c-kserve-945956779-fnwkn" Jul 06 01:58:05.526954 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:58:05.526791 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/364de976-3b02-4e3c-9227-5f55bd3bed70-home\") pod \"rate-test-model-5afb9c-kserve-945956779-fnwkn\" (UID: \"364de976-3b02-4e3c-9227-5f55bd3bed70\") " pod="ai-tenant-e2e-shared-a-5afb9c/rate-test-model-5afb9c-kserve-945956779-fnwkn" Jul 06 01:58:05.526954 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:58:05.526821 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/364de976-3b02-4e3c-9227-5f55bd3bed70-dshm\") pod \"rate-test-model-5afb9c-kserve-945956779-fnwkn\" (UID: \"364de976-3b02-4e3c-9227-5f55bd3bed70\") " pod="ai-tenant-e2e-shared-a-5afb9c/rate-test-model-5afb9c-kserve-945956779-fnwkn" Jul 06 01:58:05.526954 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:58:05.526889 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/364de976-3b02-4e3c-9227-5f55bd3bed70-model-cache\") pod \"rate-test-model-5afb9c-kserve-945956779-fnwkn\" (UID: \"364de976-3b02-4e3c-9227-5f55bd3bed70\") " pod="ai-tenant-e2e-shared-a-5afb9c/rate-test-model-5afb9c-kserve-945956779-fnwkn" Jul 06 01:58:05.526954 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:58:05.526931 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2rkg\" (UniqueName: \"kubernetes.io/projected/364de976-3b02-4e3c-9227-5f55bd3bed70-kube-api-access-p2rkg\") pod \"rate-test-model-5afb9c-kserve-945956779-fnwkn\" (UID: \"364de976-3b02-4e3c-9227-5f55bd3bed70\") " pod="ai-tenant-e2e-shared-a-5afb9c/rate-test-model-5afb9c-kserve-945956779-fnwkn" Jul 06 01:58:05.527094 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:58:05.527029 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/364de976-3b02-4e3c-9227-5f55bd3bed70-tmp-dir\") pod \"rate-test-model-5afb9c-kserve-945956779-fnwkn\" (UID: \"364de976-3b02-4e3c-9227-5f55bd3bed70\") " pod="ai-tenant-e2e-shared-a-5afb9c/rate-test-model-5afb9c-kserve-945956779-fnwkn" Jul 06 01:58:05.628046 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:58:05.627950 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/364de976-3b02-4e3c-9227-5f55bd3bed70-home\") pod \"rate-test-model-5afb9c-kserve-945956779-fnwkn\" (UID: \"364de976-3b02-4e3c-9227-5f55bd3bed70\") " pod="ai-tenant-e2e-shared-a-5afb9c/rate-test-model-5afb9c-kserve-945956779-fnwkn" Jul 06 01:58:05.628046 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:58:05.628009 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/364de976-3b02-4e3c-9227-5f55bd3bed70-dshm\") pod \"rate-test-model-5afb9c-kserve-945956779-fnwkn\" (UID: \"364de976-3b02-4e3c-9227-5f55bd3bed70\") " pod="ai-tenant-e2e-shared-a-5afb9c/rate-test-model-5afb9c-kserve-945956779-fnwkn" Jul 06 01:58:05.628267 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:58:05.628057 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/364de976-3b02-4e3c-9227-5f55bd3bed70-model-cache\") pod \"rate-test-model-5afb9c-kserve-945956779-fnwkn\" (UID: \"364de976-3b02-4e3c-9227-5f55bd3bed70\") " pod="ai-tenant-e2e-shared-a-5afb9c/rate-test-model-5afb9c-kserve-945956779-fnwkn" Jul 06 01:58:05.628267 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:58:05.628080 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2rkg\" (UniqueName: \"kubernetes.io/projected/364de976-3b02-4e3c-9227-5f55bd3bed70-kube-api-access-p2rkg\") pod \"rate-test-model-5afb9c-kserve-945956779-fnwkn\" (UID: \"364de976-3b02-4e3c-9227-5f55bd3bed70\") " pod="ai-tenant-e2e-shared-a-5afb9c/rate-test-model-5afb9c-kserve-945956779-fnwkn" Jul 06 01:58:05.628267 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:58:05.628144 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/364de976-3b02-4e3c-9227-5f55bd3bed70-tmp-dir\") pod \"rate-test-model-5afb9c-kserve-945956779-fnwkn\" (UID: \"364de976-3b02-4e3c-9227-5f55bd3bed70\") " pod="ai-tenant-e2e-shared-a-5afb9c/rate-test-model-5afb9c-kserve-945956779-fnwkn" Jul 06 01:58:05.628267 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:58:05.628208 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/364de976-3b02-4e3c-9227-5f55bd3bed70-tls-certs\") pod \"rate-test-model-5afb9c-kserve-945956779-fnwkn\" (UID: \"364de976-3b02-4e3c-9227-5f55bd3bed70\") " pod="ai-tenant-e2e-shared-a-5afb9c/rate-test-model-5afb9c-kserve-945956779-fnwkn" Jul 06 01:58:05.628504 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:58:05.628350 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/364de976-3b02-4e3c-9227-5f55bd3bed70-home\") pod \"rate-test-model-5afb9c-kserve-945956779-fnwkn\" (UID: \"364de976-3b02-4e3c-9227-5f55bd3bed70\") " pod="ai-tenant-e2e-shared-a-5afb9c/rate-test-model-5afb9c-kserve-945956779-fnwkn" Jul 06 01:58:05.628504 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:58:05.628427 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/364de976-3b02-4e3c-9227-5f55bd3bed70-model-cache\") pod \"rate-test-model-5afb9c-kserve-945956779-fnwkn\" (UID: \"364de976-3b02-4e3c-9227-5f55bd3bed70\") " pod="ai-tenant-e2e-shared-a-5afb9c/rate-test-model-5afb9c-kserve-945956779-fnwkn" Jul 06 01:58:05.628615 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:58:05.628546 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/364de976-3b02-4e3c-9227-5f55bd3bed70-tmp-dir\") pod \"rate-test-model-5afb9c-kserve-945956779-fnwkn\" (UID: \"364de976-3b02-4e3c-9227-5f55bd3bed70\") " pod="ai-tenant-e2e-shared-a-5afb9c/rate-test-model-5afb9c-kserve-945956779-fnwkn" Jul 06 01:58:05.630489 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:58:05.630452 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/364de976-3b02-4e3c-9227-5f55bd3bed70-dshm\") pod \"rate-test-model-5afb9c-kserve-945956779-fnwkn\" (UID: \"364de976-3b02-4e3c-9227-5f55bd3bed70\") " pod="ai-tenant-e2e-shared-a-5afb9c/rate-test-model-5afb9c-kserve-945956779-fnwkn" Jul 06 01:58:05.630654 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:58:05.630639 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/364de976-3b02-4e3c-9227-5f55bd3bed70-tls-certs\") pod \"rate-test-model-5afb9c-kserve-945956779-fnwkn\" (UID: \"364de976-3b02-4e3c-9227-5f55bd3bed70\") " pod="ai-tenant-e2e-shared-a-5afb9c/rate-test-model-5afb9c-kserve-945956779-fnwkn" Jul 06 01:58:05.635854 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:58:05.635836 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2rkg\" (UniqueName: \"kubernetes.io/projected/364de976-3b02-4e3c-9227-5f55bd3bed70-kube-api-access-p2rkg\") pod \"rate-test-model-5afb9c-kserve-945956779-fnwkn\" (UID: \"364de976-3b02-4e3c-9227-5f55bd3bed70\") " pod="ai-tenant-e2e-shared-a-5afb9c/rate-test-model-5afb9c-kserve-945956779-fnwkn" Jul 06 01:58:05.662865 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:58:05.662841 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5afb9c/rate-test-model-5afb9c-kserve-945956779-fnwkn" Jul 06 01:58:05.800357 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:58:05.800325 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5afb9c/rate-test-model-5afb9c-kserve-945956779-fnwkn"] Jul 06 01:58:05.802413 ip-10-0-143-159 kubenswrapper[2565]: W0706 01:58:05.802366 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod364de976_3b02_4e3c_9227_5f55bd3bed70.slice/crio-1ff1876386063797cdd4b7e8312164316b0c30528bef86b373afde5ea0fb31e5 WatchSource:0}: Error finding container 1ff1876386063797cdd4b7e8312164316b0c30528bef86b373afde5ea0fb31e5: Status 404 returned error can't find the container with id 1ff1876386063797cdd4b7e8312164316b0c30528bef86b373afde5ea0fb31e5 Jul 06 01:58:06.779574 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:58:06.779535 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5afb9c/rate-test-model-5afb9c-kserve-945956779-fnwkn" event={"ID":"364de976-3b02-4e3c-9227-5f55bd3bed70","Type":"ContainerStarted","Data":"fe10baea1d81d02185bf349302e8a1aa75d8c368e4f2844fcec0f893b1e7c1c9"} Jul 06 01:58:06.779574 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:58:06.779576 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5afb9c/rate-test-model-5afb9c-kserve-945956779-fnwkn" event={"ID":"364de976-3b02-4e3c-9227-5f55bd3bed70","Type":"ContainerStarted","Data":"1ff1876386063797cdd4b7e8312164316b0c30528bef86b373afde5ea0fb31e5"} Jul 06 01:58:06.795975 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:58:06.795925 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-5afb9c/rate-test-model-5afb9c-kserve-945956779-fnwkn" podStartSLOduration=1.7959102919999999 podStartE2EDuration="1.795910292s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:58:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:58:06.795146938 +0000 UTC m=+2258.936705737" watchObservedRunningTime="2026-07-06 01:58:06.795910292 +0000 UTC m=+2258.937469076" Jul 06 01:58:15.663457 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:58:15.663404 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-5afb9c/rate-test-model-5afb9c-kserve-945956779-fnwkn" Jul 06 01:58:15.663457 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:58:15.663465 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-5afb9c/rate-test-model-5afb9c-kserve-945956779-fnwkn" Jul 06 01:58:15.675719 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:58:15.675697 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-5afb9c/rate-test-model-5afb9c-kserve-945956779-fnwkn" Jul 06 01:58:15.837237 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:58:15.837211 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-5afb9c/rate-test-model-5afb9c-kserve-945956779-fnwkn" Jul 06 01:59:09.845281 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:59:09.845246 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5afb9c/rate-test-model-5afb9c-kserve-945956779-fnwkn"] Jul 06 01:59:24.867134 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:59:24.867018 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-5afb9c/rate-test-model-5afb9c-kserve-945956779-fnwkn" podUID="364de976-3b02-4e3c-9227-5f55bd3bed70" containerName="main" containerID="cri-o://fe10baea1d81d02185bf349302e8a1aa75d8c368e4f2844fcec0f893b1e7c1c9" gracePeriod=45 Jul 06 01:59:25.113207 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:59:25.113173 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5afb9c/rate-test-model-5afb9c-kserve-945956779-fnwkn" Jul 06 01:59:25.180201 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:59:25.180109 2565 generic.go:356] "Generic (PLEG): container finished" podID="364de976-3b02-4e3c-9227-5f55bd3bed70" containerID="fe10baea1d81d02185bf349302e8a1aa75d8c368e4f2844fcec0f893b1e7c1c9" exitCode=0 Jul 06 01:59:25.180201 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:59:25.180167 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5afb9c/rate-test-model-5afb9c-kserve-945956779-fnwkn" Jul 06 01:59:25.180469 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:59:25.180202 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5afb9c/rate-test-model-5afb9c-kserve-945956779-fnwkn" event={"ID":"364de976-3b02-4e3c-9227-5f55bd3bed70","Type":"ContainerDied","Data":"fe10baea1d81d02185bf349302e8a1aa75d8c368e4f2844fcec0f893b1e7c1c9"} Jul 06 01:59:25.180469 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:59:25.180247 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5afb9c/rate-test-model-5afb9c-kserve-945956779-fnwkn" event={"ID":"364de976-3b02-4e3c-9227-5f55bd3bed70","Type":"ContainerDied","Data":"1ff1876386063797cdd4b7e8312164316b0c30528bef86b373afde5ea0fb31e5"} Jul 06 01:59:25.180469 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:59:25.180264 2565 scope.go:117] "RemoveContainer" containerID="fe10baea1d81d02185bf349302e8a1aa75d8c368e4f2844fcec0f893b1e7c1c9" Jul 06 01:59:25.188838 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:59:25.188818 2565 scope.go:117] "RemoveContainer" containerID="fe10baea1d81d02185bf349302e8a1aa75d8c368e4f2844fcec0f893b1e7c1c9" Jul 06 01:59:25.191748 ip-10-0-143-159 kubenswrapper[2565]: E0706 01:59:25.191719 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe10baea1d81d02185bf349302e8a1aa75d8c368e4f2844fcec0f893b1e7c1c9\": container with ID starting with fe10baea1d81d02185bf349302e8a1aa75d8c368e4f2844fcec0f893b1e7c1c9 not found: ID does not exist" containerID="fe10baea1d81d02185bf349302e8a1aa75d8c368e4f2844fcec0f893b1e7c1c9" Jul 06 01:59:25.191845 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:59:25.191755 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe10baea1d81d02185bf349302e8a1aa75d8c368e4f2844fcec0f893b1e7c1c9"} err="failed to get container status \"fe10baea1d81d02185bf349302e8a1aa75d8c368e4f2844fcec0f893b1e7c1c9\": rpc error: code = NotFound desc = could not find container \"fe10baea1d81d02185bf349302e8a1aa75d8c368e4f2844fcec0f893b1e7c1c9\": container with ID starting with fe10baea1d81d02185bf349302e8a1aa75d8c368e4f2844fcec0f893b1e7c1c9 not found: ID does not exist" Jul 06 01:59:25.251518 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:59:25.251492 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p2rkg\" (UniqueName: \"kubernetes.io/projected/364de976-3b02-4e3c-9227-5f55bd3bed70-kube-api-access-p2rkg\") pod \"364de976-3b02-4e3c-9227-5f55bd3bed70\" (UID: \"364de976-3b02-4e3c-9227-5f55bd3bed70\") " Jul 06 01:59:25.251667 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:59:25.251536 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/364de976-3b02-4e3c-9227-5f55bd3bed70-tls-certs\") pod \"364de976-3b02-4e3c-9227-5f55bd3bed70\" (UID: \"364de976-3b02-4e3c-9227-5f55bd3bed70\") " Jul 06 01:59:25.251667 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:59:25.251555 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/364de976-3b02-4e3c-9227-5f55bd3bed70-model-cache\") pod \"364de976-3b02-4e3c-9227-5f55bd3bed70\" (UID: \"364de976-3b02-4e3c-9227-5f55bd3bed70\") " Jul 06 01:59:25.251667 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:59:25.251584 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/364de976-3b02-4e3c-9227-5f55bd3bed70-home\") pod \"364de976-3b02-4e3c-9227-5f55bd3bed70\" (UID: \"364de976-3b02-4e3c-9227-5f55bd3bed70\") " Jul 06 01:59:25.251835 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:59:25.251732 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/364de976-3b02-4e3c-9227-5f55bd3bed70-dshm\") pod \"364de976-3b02-4e3c-9227-5f55bd3bed70\" (UID: \"364de976-3b02-4e3c-9227-5f55bd3bed70\") " Jul 06 01:59:25.251835 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:59:25.251793 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/364de976-3b02-4e3c-9227-5f55bd3bed70-home" (OuterVolumeSpecName: "home") pod "364de976-3b02-4e3c-9227-5f55bd3bed70" (UID: "364de976-3b02-4e3c-9227-5f55bd3bed70"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:59:25.251942 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:59:25.251845 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/364de976-3b02-4e3c-9227-5f55bd3bed70-tmp-dir\") pod \"364de976-3b02-4e3c-9227-5f55bd3bed70\" (UID: \"364de976-3b02-4e3c-9227-5f55bd3bed70\") " Jul 06 01:59:25.251942 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:59:25.251862 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/364de976-3b02-4e3c-9227-5f55bd3bed70-model-cache" (OuterVolumeSpecName: "model-cache") pod "364de976-3b02-4e3c-9227-5f55bd3bed70" (UID: "364de976-3b02-4e3c-9227-5f55bd3bed70"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:59:25.252115 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:59:25.252089 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/364de976-3b02-4e3c-9227-5f55bd3bed70-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "364de976-3b02-4e3c-9227-5f55bd3bed70" (UID: "364de976-3b02-4e3c-9227-5f55bd3bed70"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:59:25.252207 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:59:25.252191 2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/364de976-3b02-4e3c-9227-5f55bd3bed70-tmp-dir\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:59:25.252268 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:59:25.252209 2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/364de976-3b02-4e3c-9227-5f55bd3bed70-model-cache\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:59:25.252268 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:59:25.252216 2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/364de976-3b02-4e3c-9227-5f55bd3bed70-home\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:59:25.253831 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:59:25.253803 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/364de976-3b02-4e3c-9227-5f55bd3bed70-dshm" (OuterVolumeSpecName: "dshm") pod "364de976-3b02-4e3c-9227-5f55bd3bed70" (UID: "364de976-3b02-4e3c-9227-5f55bd3bed70"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:59:25.253939 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:59:25.253900 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/364de976-3b02-4e3c-9227-5f55bd3bed70-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "364de976-3b02-4e3c-9227-5f55bd3bed70" (UID: "364de976-3b02-4e3c-9227-5f55bd3bed70"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 01:59:25.253939 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:59:25.253922 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/364de976-3b02-4e3c-9227-5f55bd3bed70-kube-api-access-p2rkg" (OuterVolumeSpecName: "kube-api-access-p2rkg") pod "364de976-3b02-4e3c-9227-5f55bd3bed70" (UID: "364de976-3b02-4e3c-9227-5f55bd3bed70"). InnerVolumeSpecName "kube-api-access-p2rkg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:59:25.352811 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:59:25.352767 2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/364de976-3b02-4e3c-9227-5f55bd3bed70-tls-certs\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:59:25.352811 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:59:25.352807 2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/364de976-3b02-4e3c-9227-5f55bd3bed70-dshm\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:59:25.352811 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:59:25.352829 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2rkg\" (UniqueName: \"kubernetes.io/projected/364de976-3b02-4e3c-9227-5f55bd3bed70-kube-api-access-p2rkg\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 06 01:59:25.503540 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:59:25.503499 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5afb9c/rate-test-model-5afb9c-kserve-945956779-fnwkn"] Jul 06 01:59:25.510004 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:59:25.509974 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-5afb9c/rate-test-model-5afb9c-kserve-945956779-fnwkn"] Jul 06 01:59:26.324794 ip-10-0-143-159 kubenswrapper[2565]: I0706 01:59:26.324760 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="364de976-3b02-4e3c-9227-5f55bd3bed70" path="/var/lib/kubelet/pods/364de976-3b02-4e3c-9227-5f55bd3bed70/volumes" Jul 06 02:00:07.434292 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:07.434260 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-6787fdf5f-8fjs9_59b58f6a-6c28-47b0-b99b-dd462f59d915/maas-api/0.log" Jul 06 02:00:07.892728 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:07.892697 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-797b48b58c-45b4g_2d9be716-b0b9-4983-a9eb-b0912f4d854f/manager/0.log" Jul 06 02:00:08.499368 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:08.499337 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d8fbf9755-s7rjg_787b65e1-3d80-4453-85e3-f4489d16b595/copy-manifests/0.log" Jul 06 02:00:08.506747 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:08.506727 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d8fbf9755-s7rjg_787b65e1-3d80-4453-85e3-f4489d16b595/manager/0.log" Jul 06 02:00:08.614989 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:08.614959 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-jk4pl_1138dd2e-074c-4dc2-a681-f02a1ea002e7/postgres/0.log" Jul 06 02:00:10.453309 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:10.453280 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-8mgg7_24550d84-e84e-4d58-9918-cbe0b6808cba/manager/0.log" Jul 06 02:00:11.254100 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:11.254072 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-799868c9c7-h75ft_979609f9-f877-47b2-a3b7-17d491a53d23/kube-auth-proxy/0.log" Jul 06 02:00:11.698462 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:11.698361 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-6prj5_767bc03c-400a-4f1b-a4e5-f5c50c3a8ad1/payload-processing/0.log" Jul 06 02:00:12.353517 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:12.353490 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-vdpkn_bf969efe-8d72-4116-a5ce-dd1c4598986c/main/0.log" Jul 06 02:00:12.577111 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:12.577080 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-qdpfl_14f71f8c-1a5d-4048-8df9-0bd19bf39eed/main/0.log" Jul 06 02:00:19.661315 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:19.661283 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-pm8wq_4c0a1cea-8dca-4e9d-afdb-81f874754505/global-pull-secret-syncer/0.log" Jul 06 02:00:19.818672 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:19.818644 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-k46tr_9b90df9d-7dc5-41c1-a529-7cfdc8a41d09/konnectivity-agent/0.log" Jul 06 02:00:19.890862 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:19.890831 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-159.ec2.internal_008efb678643724ebfdbed72c5411e31/haproxy/0.log" Jul 06 02:00:24.733742 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:24.733709 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-8mgg7_24550d84-e84e-4d58-9918-cbe0b6808cba/manager/0.log" Jul 06 02:00:26.567642 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:26.567609 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_14ae4c05-d30f-413c-88b5-bd77d3fbeda6/alertmanager/0.log" Jul 06 02:00:26.594810 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:26.594789 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_14ae4c05-d30f-413c-88b5-bd77d3fbeda6/config-reloader/0.log" Jul 06 02:00:26.617207 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:26.617175 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_14ae4c05-d30f-413c-88b5-bd77d3fbeda6/kube-rbac-proxy-web/0.log" Jul 06 02:00:26.637268 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:26.637245 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_14ae4c05-d30f-413c-88b5-bd77d3fbeda6/kube-rbac-proxy/0.log" Jul 06 02:00:26.659745 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:26.659726 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_14ae4c05-d30f-413c-88b5-bd77d3fbeda6/kube-rbac-proxy-metric/0.log" Jul 06 02:00:26.680421 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:26.680402 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_14ae4c05-d30f-413c-88b5-bd77d3fbeda6/prom-label-proxy/0.log" Jul 06 02:00:26.699762 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:26.699743 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_14ae4c05-d30f-413c-88b5-bd77d3fbeda6/init-config-reloader/0.log" Jul 06 02:00:26.763890 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:26.763868 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-ksnx7_a582740b-395e-485b-b214-81c6f58a7747/kube-state-metrics/0.log" Jul 06 02:00:26.784171 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:26.784147 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-ksnx7_a582740b-395e-485b-b214-81c6f58a7747/kube-rbac-proxy-main/0.log" Jul 06 02:00:26.804651 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:26.804635 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-ksnx7_a582740b-395e-485b-b214-81c6f58a7747/kube-rbac-proxy-self/0.log" Jul 06 02:00:26.835140 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:26.835077 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-88b699c9-dc7x2_e8e13654-1d6d-436b-a582-40074d4bdde2/metrics-server/0.log" Jul 06 02:00:26.894062 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:26.894040 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cjx7b_ca215253-2060-46f3-94c8-2b48a3fc81a2/node-exporter/0.log" Jul 06 02:00:26.915536 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:26.915517 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cjx7b_ca215253-2060-46f3-94c8-2b48a3fc81a2/kube-rbac-proxy/0.log" Jul 06 02:00:26.945786 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:26.945769 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cjx7b_ca215253-2060-46f3-94c8-2b48a3fc81a2/init-textfile/0.log" Jul 06 02:00:27.156331 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:27.156273 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-448pq_50574349-d208-4de6-aba3-06e535105c6a/kube-rbac-proxy-main/0.log" Jul 06 02:00:27.173824 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:27.173803 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-448pq_50574349-d208-4de6-aba3-06e535105c6a/kube-rbac-proxy-self/0.log" Jul 06 02:00:27.197147 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:27.197124 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-448pq_50574349-d208-4de6-aba3-06e535105c6a/openshift-state-metrics/0.log" Jul 06 02:00:27.233541 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:27.233520 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0b46c140-08b2-4581-8f42-3ddc1dcb8f6c/prometheus/0.log" Jul 06 02:00:27.255004 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:27.254980 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0b46c140-08b2-4581-8f42-3ddc1dcb8f6c/config-reloader/0.log" Jul 06 02:00:27.278897 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:27.278880 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0b46c140-08b2-4581-8f42-3ddc1dcb8f6c/thanos-sidecar/0.log" Jul 06 02:00:27.298307 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:27.298292 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0b46c140-08b2-4581-8f42-3ddc1dcb8f6c/kube-rbac-proxy-web/0.log" Jul 06 02:00:27.323959 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:27.323943 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0b46c140-08b2-4581-8f42-3ddc1dcb8f6c/kube-rbac-proxy/0.log" Jul 06 02:00:27.344726 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:27.344709 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0b46c140-08b2-4581-8f42-3ddc1dcb8f6c/kube-rbac-proxy-thanos/0.log" Jul 06 02:00:27.366729 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:27.366666 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0b46c140-08b2-4581-8f42-3ddc1dcb8f6c/init-config-reloader/0.log" Jul 06 02:00:27.395880 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:27.395858 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-q6scb_6e241b85-89f3-4b81-9fdf-5af2a97b9033/prometheus-operator/0.log" Jul 06 02:00:27.414164 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:27.414145 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-q6scb_6e241b85-89f3-4b81-9fdf-5af2a97b9033/kube-rbac-proxy/0.log" Jul 06 02:00:27.441013 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:27.440991 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-fnb5n_6d50abeb-044b-4ff4-9f61-408e4363dbe6/prometheus-operator-admission-webhook/0.log" Jul 06 02:00:27.573541 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:27.573514 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-857dc8f57d-rtklb_2b5d11a6-bb09-4c32-9e61-db7732341d0c/thanos-query/0.log" Jul 06 02:00:27.597480 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:27.597461 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-857dc8f57d-rtklb_2b5d11a6-bb09-4c32-9e61-db7732341d0c/kube-rbac-proxy-web/0.log" Jul 06 02:00:27.614171 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:27.614147 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-857dc8f57d-rtklb_2b5d11a6-bb09-4c32-9e61-db7732341d0c/kube-rbac-proxy/0.log" Jul 06 02:00:27.638125 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:27.638071 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-857dc8f57d-rtklb_2b5d11a6-bb09-4c32-9e61-db7732341d0c/prom-label-proxy/0.log" Jul 06 02:00:27.658627 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:27.658610 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-857dc8f57d-rtklb_2b5d11a6-bb09-4c32-9e61-db7732341d0c/kube-rbac-proxy-rules/0.log" Jul 06 02:00:27.677852 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:27.677831 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-857dc8f57d-rtklb_2b5d11a6-bb09-4c32-9e61-db7732341d0c/kube-rbac-proxy-metrics/0.log" Jul 06 02:00:30.183255 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:30.183230 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6c7d44bb8f-jc6tc_35f08148-08b1-4267-908c-e24d039e73d6/console/0.log" Jul 06 02:00:30.215052 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:30.215030 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-x9sxx_6122d230-b346-4d52-9e50-762d328a2651/download-server/0.log" Jul 06 02:00:30.458755 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:30.458635 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hqwmc/perf-node-gather-daemonset-vg8kg"] Jul 06 02:00:30.459186 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:30.459167 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="364de976-3b02-4e3c-9227-5f55bd3bed70" containerName="main" Jul 06 02:00:30.459186 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:30.459188 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="364de976-3b02-4e3c-9227-5f55bd3bed70" containerName="main" Jul 06 02:00:30.459357 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:30.459307 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="364de976-3b02-4e3c-9227-5f55bd3bed70" containerName="main" Jul 06 02:00:30.462816 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:30.462795 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-vg8kg" Jul 06 02:00:30.465441 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:30.465421 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hqwmc\"/\"openshift-service-ca.crt\"" Jul 06 02:00:30.466321 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:30.466303 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-hqwmc\"/\"default-dockercfg-5sss8\"" Jul 06 02:00:30.466421 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:30.466332 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hqwmc\"/\"kube-root-ca.crt\"" Jul 06 02:00:30.470591 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:30.469987 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hqwmc/perf-node-gather-daemonset-vg8kg"] Jul 06 02:00:30.559024 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:30.558987 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b4f38ce3-9fcb-4091-987c-0639914d0f04-proc\") pod \"perf-node-gather-daemonset-vg8kg\" (UID: \"b4f38ce3-9fcb-4091-987c-0639914d0f04\") " pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-vg8kg" Jul 06 02:00:30.559189 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:30.559051 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tcn9\" (UniqueName: \"kubernetes.io/projected/b4f38ce3-9fcb-4091-987c-0639914d0f04-kube-api-access-6tcn9\") pod \"perf-node-gather-daemonset-vg8kg\" (UID: \"b4f38ce3-9fcb-4091-987c-0639914d0f04\") " pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-vg8kg" Jul 06 02:00:30.559189 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:30.559111 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b4f38ce3-9fcb-4091-987c-0639914d0f04-podres\") pod \"perf-node-gather-daemonset-vg8kg\" (UID: \"b4f38ce3-9fcb-4091-987c-0639914d0f04\") " pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-vg8kg" Jul 06 02:00:30.559189 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:30.559164 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b4f38ce3-9fcb-4091-987c-0639914d0f04-lib-modules\") pod \"perf-node-gather-daemonset-vg8kg\" (UID: \"b4f38ce3-9fcb-4091-987c-0639914d0f04\") " pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-vg8kg" Jul 06 02:00:30.559189 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:30.559187 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b4f38ce3-9fcb-4091-987c-0639914d0f04-sys\") pod \"perf-node-gather-daemonset-vg8kg\" (UID: \"b4f38ce3-9fcb-4091-987c-0639914d0f04\") " pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-vg8kg" Jul 06 02:00:30.660441 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:30.660410 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b4f38ce3-9fcb-4091-987c-0639914d0f04-podres\") pod \"perf-node-gather-daemonset-vg8kg\" (UID: \"b4f38ce3-9fcb-4091-987c-0639914d0f04\") " pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-vg8kg" Jul 06 02:00:30.660614 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:30.660462 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b4f38ce3-9fcb-4091-987c-0639914d0f04-lib-modules\") pod \"perf-node-gather-daemonset-vg8kg\" (UID: \"b4f38ce3-9fcb-4091-987c-0639914d0f04\") " pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-vg8kg" Jul 06 02:00:30.660614 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:30.660484 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b4f38ce3-9fcb-4091-987c-0639914d0f04-sys\") pod \"perf-node-gather-daemonset-vg8kg\" (UID: \"b4f38ce3-9fcb-4091-987c-0639914d0f04\") " pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-vg8kg" Jul 06 02:00:30.660614 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:30.660508 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b4f38ce3-9fcb-4091-987c-0639914d0f04-proc\") pod \"perf-node-gather-daemonset-vg8kg\" (UID: \"b4f38ce3-9fcb-4091-987c-0639914d0f04\") " pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-vg8kg" Jul 06 02:00:30.660614 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:30.660549 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tcn9\" (UniqueName: \"kubernetes.io/projected/b4f38ce3-9fcb-4091-987c-0639914d0f04-kube-api-access-6tcn9\") pod \"perf-node-gather-daemonset-vg8kg\" (UID: \"b4f38ce3-9fcb-4091-987c-0639914d0f04\") " pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-vg8kg" Jul 06 02:00:30.660614 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:30.660577 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b4f38ce3-9fcb-4091-987c-0639914d0f04-podres\") pod \"perf-node-gather-daemonset-vg8kg\" (UID: \"b4f38ce3-9fcb-4091-987c-0639914d0f04\") " pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-vg8kg" Jul 06 02:00:30.660614 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:30.660604 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b4f38ce3-9fcb-4091-987c-0639914d0f04-sys\") pod \"perf-node-gather-daemonset-vg8kg\" (UID: \"b4f38ce3-9fcb-4091-987c-0639914d0f04\") " pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-vg8kg" Jul 06 02:00:30.660845 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:30.660617 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b4f38ce3-9fcb-4091-987c-0639914d0f04-proc\") pod \"perf-node-gather-daemonset-vg8kg\" (UID: \"b4f38ce3-9fcb-4091-987c-0639914d0f04\") " pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-vg8kg" Jul 06 02:00:30.660845 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:30.660638 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b4f38ce3-9fcb-4091-987c-0639914d0f04-lib-modules\") pod \"perf-node-gather-daemonset-vg8kg\" (UID: \"b4f38ce3-9fcb-4091-987c-0639914d0f04\") " pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-vg8kg" Jul 06 02:00:30.668839 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:30.668812 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tcn9\" (UniqueName: \"kubernetes.io/projected/b4f38ce3-9fcb-4091-987c-0639914d0f04-kube-api-access-6tcn9\") pod \"perf-node-gather-daemonset-vg8kg\" (UID: \"b4f38ce3-9fcb-4091-987c-0639914d0f04\") " pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-vg8kg" Jul 06 02:00:30.774704 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:30.774675 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-vg8kg" Jul 06 02:00:30.908419 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:30.908393 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hqwmc/perf-node-gather-daemonset-vg8kg"] Jul 06 02:00:30.910210 ip-10-0-143-159 kubenswrapper[2565]: W0706 02:00:30.910181 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb4f38ce3_9fcb_4091_987c_0639914d0f04.slice/crio-e8456a22deeb6bf702541ddfe5deb80febb893f5730e967d33de90f8325896a4 WatchSource:0}: Error finding container e8456a22deeb6bf702541ddfe5deb80febb893f5730e967d33de90f8325896a4: Status 404 returned error can't find the container with id e8456a22deeb6bf702541ddfe5deb80febb893f5730e967d33de90f8325896a4 Jul 06 02:00:31.505559 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:31.505519 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-vg8kg" event={"ID":"b4f38ce3-9fcb-4091-987c-0639914d0f04","Type":"ContainerStarted","Data":"aa8e1737904370605469204f5ac369b3fe9efecddbd1f55efe1741bd6808f295"} Jul 06 02:00:31.505931 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:31.505585 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-vg8kg" event={"ID":"b4f38ce3-9fcb-4091-987c-0639914d0f04","Type":"ContainerStarted","Data":"e8456a22deeb6bf702541ddfe5deb80febb893f5730e967d33de90f8325896a4"} Jul 06 02:00:31.505931 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:31.505726 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-vg8kg" Jul 06 02:00:31.523628 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:31.523587 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-vg8kg" podStartSLOduration=1.523572445 podStartE2EDuration="1.523572445s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 02:00:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:00:31.520890442 +0000 UTC m=+2403.662449240" watchObservedRunningTime="2026-07-06 02:00:31.523572445 +0000 UTC m=+2403.665131230" Jul 06 02:00:32.037446 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:32.037415 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tfzf9_e4384477-41b5-40c5-83e5-23f7c541dfa0/dns/0.log" Jul 06 02:00:32.056989 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:32.056965 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tfzf9_e4384477-41b5-40c5-83e5-23f7c541dfa0/kube-rbac-proxy/0.log" Jul 06 02:00:32.125111 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:32.125091 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2s7xk_28f968d2-c730-436e-8bb9-413d0f88cf10/dns-node-resolver/0.log" Jul 06 02:00:32.734843 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:32.734808 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5nr92_ec5262e8-19a7-4dce-bf06-4d792eb2b877/node-ca/0.log" Jul 06 02:00:33.784511 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:33.784483 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-799868c9c7-h75ft_979609f9-f877-47b2-a3b7-17d491a53d23/kube-auth-proxy/0.log" Jul 06 02:00:33.943855 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:33.943826 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-6prj5_767bc03c-400a-4f1b-a4e5-f5c50c3a8ad1/payload-processing/0.log" Jul 06 02:00:34.606537 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:34.606512 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5br85_889f2b80-8fba-40de-a6f6-314a17b310df/serve-healthcheck-canary/0.log" Jul 06 02:00:35.114190 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:35.114162 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9vxks_117cb15c-e26a-4b08-9dc2-57f776b22adc/kube-rbac-proxy/0.log" Jul 06 02:00:35.134152 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:35.134125 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9vxks_117cb15c-e26a-4b08-9dc2-57f776b22adc/exporter/0.log" Jul 06 02:00:35.154736 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:35.154703 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9vxks_117cb15c-e26a-4b08-9dc2-57f776b22adc/extractor/0.log" Jul 06 02:00:37.520718 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:37.520690 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-hqwmc/perf-node-gather-daemonset-vg8kg" Jul 06 02:00:37.600936 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:37.600908 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-6787fdf5f-8fjs9_59b58f6a-6c28-47b0-b99b-dd462f59d915/maas-api/0.log" Jul 06 02:00:37.746195 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:37.746163 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-797b48b58c-45b4g_2d9be716-b0b9-4983-a9eb-b0912f4d854f/manager/0.log" Jul 06 02:00:38.080969 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:38.080932 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d8fbf9755-s7rjg_787b65e1-3d80-4453-85e3-f4489d16b595/manager/0.log" Jul 06 02:00:38.099688 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:38.099666 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d8fbf9755-s7rjg_787b65e1-3d80-4453-85e3-f4489d16b595/copy-manifests/0.log" Jul 06 02:00:38.121888 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:38.121870 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-jk4pl_1138dd2e-074c-4dc2-a681-f02a1ea002e7/postgres/0.log" Jul 06 02:00:39.367134 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:39.367106 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-76dc9d4b5b-wnpn9_6849fc29-d12d-4792-95d9-1dca00e22b72/manager/0.log" Jul 06 02:00:39.392930 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:39.392903 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-cqkv2_d3ad6ae1-7d23-43fc-9617-e7e084891b69/openshift-lws-operator/0.log" Jul 06 02:00:40.206986 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:40.206957 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-799868c9c7-h75ft_979609f9-f877-47b2-a3b7-17d491a53d23/kube-auth-proxy/0.log" Jul 06 02:00:40.330526 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:40.330500 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-6prj5_767bc03c-400a-4f1b-a4e5-f5c50c3a8ad1/payload-processing/0.log" Jul 06 02:00:41.826510 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:41.826481 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-799868c9c7-h75ft_979609f9-f877-47b2-a3b7-17d491a53d23/kube-auth-proxy/0.log" Jul 06 02:00:41.984349 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:41.980426 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-6prj5_767bc03c-400a-4f1b-a4e5-f5c50c3a8ad1/payload-processing/0.log" Jul 06 02:00:44.426905 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:44.426879 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-7wptq_0af069fa-e1e2-4bc0-bc70-762b072c0cbb/migrator/0.log" Jul 06 02:00:44.446019 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:44.445996 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-7wptq_0af069fa-e1e2-4bc0-bc70-762b072c0cbb/graceful-termination/0.log" Jul 06 02:00:45.847078 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:45.847049 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-87qvv_483f8b7d-8c78-4edb-b992-d448f8e1f2ca/kube-multus/0.log" Jul 06 02:00:45.868111 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:45.868086 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5nrjj_b771700e-b5fb-4e58-b526-570e6d21c9d7/kube-multus-additional-cni-plugins/0.log" Jul 06 02:00:45.887662 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:45.887642 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5nrjj_b771700e-b5fb-4e58-b526-570e6d21c9d7/egress-router-binary-copy/0.log" Jul 06 02:00:45.907152 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:45.907122 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5nrjj_b771700e-b5fb-4e58-b526-570e6d21c9d7/cni-plugins/0.log" Jul 06 02:00:45.926543 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:45.926513 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5nrjj_b771700e-b5fb-4e58-b526-570e6d21c9d7/bond-cni-plugin/0.log" Jul 06 02:00:45.945806 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:45.945787 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5nrjj_b771700e-b5fb-4e58-b526-570e6d21c9d7/routeoverride-cni/0.log" Jul 06 02:00:45.964697 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:45.964673 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5nrjj_b771700e-b5fb-4e58-b526-570e6d21c9d7/whereabouts-cni-bincopy/0.log" Jul 06 02:00:45.983663 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:45.983643 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5nrjj_b771700e-b5fb-4e58-b526-570e6d21c9d7/whereabouts-cni/0.log" Jul 06 02:00:46.365440 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:46.365411 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-drnhm_fbe62245-715f-4565-bfa8-af7c3ab5d36a/network-metrics-daemon/0.log" Jul 06 02:00:46.383532 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:46.383489 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-drnhm_fbe62245-715f-4565-bfa8-af7c3ab5d36a/kube-rbac-proxy/0.log" Jul 06 02:00:47.270802 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:47.270771 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b9krk_68d84c01-13c1-4832-bcbf-41ad082024f6/ovn-controller/0.log" Jul 06 02:00:47.287813 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:47.287792 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b9krk_68d84c01-13c1-4832-bcbf-41ad082024f6/ovn-acl-logging/0.log" Jul 06 02:00:47.308860 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:47.308819 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b9krk_68d84c01-13c1-4832-bcbf-41ad082024f6/kube-rbac-proxy-node/0.log" Jul 06 02:00:47.327583 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:47.327567 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b9krk_68d84c01-13c1-4832-bcbf-41ad082024f6/kube-rbac-proxy-ovn-metrics/0.log" Jul 06 02:00:47.346026 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:47.346004 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b9krk_68d84c01-13c1-4832-bcbf-41ad082024f6/northd/0.log" Jul 06 02:00:47.365014 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:47.364996 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b9krk_68d84c01-13c1-4832-bcbf-41ad082024f6/nbdb/0.log" Jul 06 02:00:47.384512 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:47.384493 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b9krk_68d84c01-13c1-4832-bcbf-41ad082024f6/sbdb/0.log" Jul 06 02:00:47.433855 ip-10-0-143-159 kubenswrapper[2565]: I0706 02:00:47.433832 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b9krk_68d84c01-13c1-4832-bcbf-41ad082024f6/ovnkube-controller/0.log"