Jul 28 01:23:53.013184 ip-10-0-143-186 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 28 01:23:53.013199 ip-10-0-143-186 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 28 01:23:53.013209 ip-10-0-143-186 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 28 01:23:53.013557 ip-10-0-143-186 systemd[1]: Failed to start Kubernetes Kubelet. Jul 28 01:24:04.330200 ip-10-0-143-186 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 28 01:24:04.330220 ip-10-0-143-186 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot c28711efa190455caa3e0ffb8613c81c -- Jul 28 01:26:01.331862 ip-10-0-143-186 systemd[1]: Starting Kubernetes Kubelet... Jul 28 01:26:01.713709 ip-10-0-143-186 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 01:26:01.713709 ip-10-0-143-186 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 28 01:26:01.713709 ip-10-0-143-186 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 01:26:01.713709 ip-10-0-143-186 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 28 01:26:01.713709 ip-10-0-143-186 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 01:26:01.715680 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.715591 2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 28 01:26:01.722158 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722136 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 01:26:01.722158 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722155 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 01:26:01.722158 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722159 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 01:26:01.722158 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722163 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 01:26:01.722158 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722166 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 01:26:01.722355 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722169 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 01:26:01.722355 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722172 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 01:26:01.722355 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722175 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 01:26:01.722355 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722178 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 01:26:01.722355 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722180 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 01:26:01.722355 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722183 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 01:26:01.722355 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722185 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 01:26:01.722355 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722189 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 01:26:01.722355 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722191 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 01:26:01.722355 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722194 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 01:26:01.722355 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722197 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 01:26:01.722355 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722200 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 01:26:01.722355 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722203 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 01:26:01.722355 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722206 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 01:26:01.722355 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722208 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 01:26:01.722355 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722211 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 01:26:01.722355 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722213 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 01:26:01.722355 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722216 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 01:26:01.722355 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722218 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 01:26:01.722355 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722221 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 01:26:01.722848 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722224 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 01:26:01.722848 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722226 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 01:26:01.722848 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722229 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 01:26:01.722848 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722231 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 01:26:01.722848 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722234 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 01:26:01.722848 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722236 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 01:26:01.722848 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722239 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 01:26:01.722848 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722244 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 01:26:01.722848 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722247 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 01:26:01.722848 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722249 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 01:26:01.722848 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722252 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 01:26:01.722848 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722254 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 01:26:01.722848 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722258 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 01:26:01.722848 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722262 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 01:26:01.722848 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722264 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 01:26:01.722848 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722267 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 01:26:01.722848 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722269 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 01:26:01.722848 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722272 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 01:26:01.722848 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722275 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 01:26:01.723368 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722278 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 01:26:01.723368 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722280 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 01:26:01.723368 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722283 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 01:26:01.723368 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722285 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 01:26:01.723368 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722288 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 01:26:01.723368 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722291 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 01:26:01.723368 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722294 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 01:26:01.723368 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722296 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 01:26:01.723368 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722299 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 01:26:01.723368 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722302 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 01:26:01.723368 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722304 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 01:26:01.723368 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722309 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 01:26:01.723368 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722313 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 01:26:01.723368 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722316 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 01:26:01.723368 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722319 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 01:26:01.723368 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722322 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 01:26:01.723368 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722324 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 01:26:01.723368 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722327 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 01:26:01.723368 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722330 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 01:26:01.723368 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722332 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 01:26:01.723859 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722335 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 01:26:01.723859 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722338 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 01:26:01.723859 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722342 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 01:26:01.723859 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722345 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 01:26:01.723859 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722348 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 01:26:01.723859 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722352 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 01:26:01.723859 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722355 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 01:26:01.723859 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722359 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 01:26:01.723859 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722362 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 01:26:01.723859 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722365 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 01:26:01.723859 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722368 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 01:26:01.723859 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722371 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 01:26:01.723859 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722373 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 01:26:01.723859 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722376 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 01:26:01.723859 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722380 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 01:26:01.723859 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722383 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 01:26:01.723859 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722386 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 01:26:01.723859 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722388 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 01:26:01.723859 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722391 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 01:26:01.724312 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722399 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 01:26:01.724312 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722402 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 01:26:01.724312 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722405 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 28 01:26:01.724312 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722408 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 01:26:01.724312 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722410 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 01:26:01.724312 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722414 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 01:26:01.724312 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722418 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 01:26:01.724312 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722421 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 01:26:01.724312 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722423 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 01:26:01.724312 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722426 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 01:26:01.724312 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722429 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 01:26:01.724312 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722432 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 01:26:01.724312 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722434 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 01:26:01.724312 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722437 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 01:26:01.724312 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722440 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 01:26:01.724312 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.722443 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 01:26:01.724312 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723170 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 01:26:01.724312 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723177 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 01:26:01.724312 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723180 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 01:26:01.724778 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723185 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 01:26:01.724778 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723187 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 01:26:01.724778 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723190 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 01:26:01.724778 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723193 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 01:26:01.724778 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723195 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 01:26:01.724778 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723198 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 01:26:01.724778 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723200 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 01:26:01.724778 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723205 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 01:26:01.724778 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723208 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 01:26:01.724778 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723211 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 01:26:01.724778 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723214 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 01:26:01.724778 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723217 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 01:26:01.724778 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723220 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 01:26:01.724778 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723223 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 01:26:01.724778 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723226 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 01:26:01.724778 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723229 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 01:26:01.724778 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723232 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 01:26:01.724778 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723235 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 01:26:01.724778 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723237 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 01:26:01.725247 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723240 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 01:26:01.725247 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723242 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 01:26:01.725247 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723245 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 01:26:01.725247 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723247 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 01:26:01.725247 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723250 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 01:26:01.725247 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723253 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 01:26:01.725247 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723255 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 01:26:01.725247 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723258 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 01:26:01.725247 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723262 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 01:26:01.725247 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723264 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 01:26:01.725247 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723267 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 01:26:01.725247 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723270 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 01:26:01.725247 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723273 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 01:26:01.725247 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723276 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 01:26:01.725247 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723279 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 01:26:01.725247 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723281 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 01:26:01.725247 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723284 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 01:26:01.725247 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723287 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 01:26:01.725247 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723289 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 01:26:01.725740 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723292 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 01:26:01.725740 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723295 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 01:26:01.725740 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723298 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 01:26:01.725740 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723302 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 01:26:01.725740 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723305 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 01:26:01.725740 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723307 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 01:26:01.725740 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723310 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 01:26:01.725740 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723313 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 01:26:01.725740 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723316 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 01:26:01.725740 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723319 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 01:26:01.725740 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723321 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 01:26:01.725740 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723324 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 01:26:01.725740 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723326 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 01:26:01.725740 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723329 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 01:26:01.725740 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723331 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 01:26:01.725740 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723334 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 01:26:01.725740 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723336 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 01:26:01.725740 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723339 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 01:26:01.725740 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723341 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 01:26:01.726228 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723344 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 01:26:01.726228 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723346 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 01:26:01.726228 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723349 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 01:26:01.726228 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723351 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 01:26:01.726228 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723355 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 01:26:01.726228 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723358 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 01:26:01.726228 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723362 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 01:26:01.726228 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723365 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 01:26:01.726228 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723369 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 01:26:01.726228 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723372 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 01:26:01.726228 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723375 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 01:26:01.726228 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723377 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 01:26:01.726228 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723381 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 01:26:01.726228 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723384 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 01:26:01.726228 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723386 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 01:26:01.726228 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723389 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 01:26:01.726228 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723392 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 01:26:01.726228 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723394 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 01:26:01.726228 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723397 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 01:26:01.726713 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723399 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 01:26:01.726713 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723402 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 01:26:01.726713 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723404 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 01:26:01.726713 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723407 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 28 01:26:01.726713 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723413 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 01:26:01.726713 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723415 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 01:26:01.726713 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723418 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 01:26:01.726713 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723421 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 01:26:01.726713 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723423 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 01:26:01.726713 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723426 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 01:26:01.726713 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723428 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 01:26:01.726713 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723431 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 01:26:01.726713 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723433 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 01:26:01.726713 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723436 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 01:26:01.726713 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723438 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 01:26:01.726713 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723441 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 01:26:01.726713 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723443 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 01:26:01.726713 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723446 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 01:26:01.726713 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723448 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 01:26:01.726713 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723451 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 01:26:01.727196 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723853 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 01:26:01.727196 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723859 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 01:26:01.727196 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723862 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 01:26:01.727196 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723864 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 01:26:01.727196 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723867 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 01:26:01.727196 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723870 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 01:26:01.727196 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723873 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 01:26:01.727196 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723875 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 01:26:01.727196 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723878 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 01:26:01.727196 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723881 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 01:26:01.727196 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723883 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 01:26:01.727196 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723886 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 01:26:01.727196 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723888 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 01:26:01.727196 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723891 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 01:26:01.727196 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723894 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 01:26:01.727196 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723897 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 01:26:01.727196 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723903 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 01:26:01.727196 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723906 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 01:26:01.727196 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723909 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 01:26:01.727687 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723913 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 01:26:01.727687 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723916 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 01:26:01.727687 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723919 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 01:26:01.727687 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723922 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 01:26:01.727687 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723925 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 01:26:01.727687 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723927 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 01:26:01.727687 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723930 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 01:26:01.727687 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723932 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 01:26:01.727687 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723935 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 01:26:01.727687 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723938 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 01:26:01.727687 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723940 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 01:26:01.727687 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723943 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 01:26:01.727687 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723947 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 01:26:01.727687 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723951 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 01:26:01.727687 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723954 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 01:26:01.727687 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723957 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 01:26:01.727687 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723960 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 01:26:01.727687 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723963 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 01:26:01.727687 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723966 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 01:26:01.727687 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723969 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 01:26:01.728189 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723971 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 01:26:01.728189 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723974 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 01:26:01.728189 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723976 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 01:26:01.728189 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723979 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 01:26:01.728189 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723982 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 01:26:01.728189 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723984 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 01:26:01.728189 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723987 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 01:26:01.728189 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723989 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 01:26:01.728189 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723992 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 01:26:01.728189 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.723997 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 01:26:01.728189 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724000 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 01:26:01.728189 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724004 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 01:26:01.728189 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724008 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 01:26:01.728189 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724011 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 01:26:01.728189 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724014 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 01:26:01.728189 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724017 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 01:26:01.728189 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724020 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 01:26:01.728189 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724022 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 01:26:01.728189 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724026 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 01:26:01.728673 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724030 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 01:26:01.728673 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724034 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 01:26:01.728673 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724037 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 01:26:01.728673 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724040 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 01:26:01.728673 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724043 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 01:26:01.728673 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724046 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 28 01:26:01.728673 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724048 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 01:26:01.728673 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724051 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 01:26:01.728673 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724053 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 01:26:01.728673 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724056 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 01:26:01.728673 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724058 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 01:26:01.728673 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724061 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 01:26:01.728673 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724063 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 01:26:01.728673 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724066 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 01:26:01.728673 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724069 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 01:26:01.728673 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724071 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 01:26:01.728673 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724073 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 01:26:01.728673 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724076 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 01:26:01.728673 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724079 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 01:26:01.728673 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724081 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 01:26:01.729167 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724084 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 01:26:01.729167 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724086 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 01:26:01.729167 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724090 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 01:26:01.729167 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724093 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 01:26:01.729167 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724096 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 01:26:01.729167 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724098 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 01:26:01.729167 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724101 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 01:26:01.729167 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724104 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 01:26:01.729167 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724107 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 01:26:01.729167 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724110 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 01:26:01.729167 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724112 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 01:26:01.729167 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724115 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 01:26:01.729167 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724117 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 01:26:01.729167 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724120 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 01:26:01.729167 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724123 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 01:26:01.729167 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724125 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 01:26:01.729167 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724128 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 01:26:01.729167 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724131 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 01:26:01.729167 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724133 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 01:26:01.729632 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724135 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 01:26:01.729632 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724138 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 01:26:01.729632 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724215 2578 flags.go:64] FLAG: --address="0.0.0.0" Jul 28 01:26:01.729632 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724226 2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 28 01:26:01.729632 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724235 2578 flags.go:64] FLAG: --anonymous-auth="true" Jul 28 01:26:01.729632 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724241 2578 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 28 01:26:01.729632 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724247 2578 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 28 01:26:01.729632 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724252 2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 28 01:26:01.729632 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724258 2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 28 01:26:01.729632 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724262 2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 28 01:26:01.729632 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724266 2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 28 01:26:01.729632 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724270 2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 28 01:26:01.729632 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724274 2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 28 01:26:01.729632 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724278 2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 28 01:26:01.729632 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724281 2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 28 01:26:01.729632 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724285 2578 flags.go:64] FLAG: --cgroup-root="" Jul 28 01:26:01.729632 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724288 2578 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 28 01:26:01.729632 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724292 2578 flags.go:64] FLAG: --client-ca-file="" Jul 28 01:26:01.729632 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724299 2578 flags.go:64] FLAG: --cloud-provider="external" Jul 28 01:26:01.729632 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724302 2578 flags.go:64] FLAG: --cluster-dns="[]" Jul 28 01:26:01.729632 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724306 2578 flags.go:64] FLAG: --cluster-domain="" Jul 28 01:26:01.729632 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724309 2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 28 01:26:01.729632 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724312 2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 28 01:26:01.730195 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724315 2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 28 01:26:01.730195 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724319 2578 flags.go:64] FLAG: --container-log-max-files="5" Jul 28 01:26:01.730195 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724323 2578 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 28 01:26:01.730195 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724326 2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 28 01:26:01.730195 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724329 2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 28 01:26:01.730195 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724332 2578 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 28 01:26:01.730195 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724335 2578 flags.go:64] FLAG: --contention-profiling="false" Jul 28 01:26:01.730195 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724339 2578 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 28 01:26:01.730195 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724342 2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 28 01:26:01.730195 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724345 2578 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 28 01:26:01.730195 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724348 2578 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 28 01:26:01.730195 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724352 2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 28 01:26:01.730195 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724355 2578 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 28 01:26:01.730195 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724358 2578 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 28 01:26:01.730195 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724361 2578 flags.go:64] FLAG: --enable-load-reader="false" Jul 28 01:26:01.730195 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724364 2578 flags.go:64] FLAG: --enable-server="true" Jul 28 01:26:01.730195 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724366 2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 28 01:26:01.730195 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724372 2578 flags.go:64] FLAG: --event-burst="100" Jul 28 01:26:01.730195 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724375 2578 flags.go:64] FLAG: --event-qps="50" Jul 28 01:26:01.730195 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724377 2578 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 28 01:26:01.730195 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724382 2578 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 28 01:26:01.730195 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724385 2578 flags.go:64] FLAG: --eviction-hard="" Jul 28 01:26:01.730195 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724389 2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 28 01:26:01.730195 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724392 2578 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 28 01:26:01.730790 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724395 2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 28 01:26:01.730790 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724398 2578 flags.go:64] FLAG: --eviction-soft="" Jul 28 01:26:01.730790 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724401 2578 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 28 01:26:01.730790 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724404 2578 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 28 01:26:01.730790 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724408 2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 28 01:26:01.730790 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724411 2578 flags.go:64] FLAG: --experimental-mounter-path="" Jul 28 01:26:01.730790 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724414 2578 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 28 01:26:01.730790 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724417 2578 flags.go:64] FLAG: --fail-swap-on="true" Jul 28 01:26:01.730790 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724420 2578 flags.go:64] FLAG: --feature-gates="" Jul 28 01:26:01.730790 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724424 2578 flags.go:64] FLAG: --file-check-frequency="20s" Jul 28 01:26:01.730790 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724427 2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 28 01:26:01.730790 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724430 2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 28 01:26:01.730790 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724434 2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 28 01:26:01.730790 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724437 2578 flags.go:64] FLAG: --healthz-port="10248" Jul 28 01:26:01.730790 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724440 2578 flags.go:64] FLAG: --help="false" Jul 28 01:26:01.730790 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724443 2578 flags.go:64] FLAG: --hostname-override="ip-10-0-143-186.ec2.internal" Jul 28 01:26:01.730790 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724446 2578 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 28 01:26:01.730790 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724449 2578 flags.go:64] FLAG: --http-check-frequency="20s" Jul 28 01:26:01.730790 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724452 2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 28 01:26:01.730790 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724455 2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 28 01:26:01.730790 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724458 2578 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 28 01:26:01.730790 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724461 2578 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 28 01:26:01.730790 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724464 2578 flags.go:64] FLAG: --image-service-endpoint="" Jul 28 01:26:01.731441 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724467 2578 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 28 01:26:01.731441 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724470 2578 flags.go:64] FLAG: --kube-api-burst="100" Jul 28 01:26:01.731441 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724473 2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 28 01:26:01.731441 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724476 2578 flags.go:64] FLAG: --kube-api-qps="50" Jul 28 01:26:01.731441 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724479 2578 flags.go:64] FLAG: --kube-reserved="" Jul 28 01:26:01.731441 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724483 2578 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 28 01:26:01.731441 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724486 2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 28 01:26:01.731441 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724489 2578 flags.go:64] FLAG: --kubelet-cgroups="" Jul 28 01:26:01.731441 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724492 2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 28 01:26:01.731441 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724495 2578 flags.go:64] FLAG: --lock-file="" Jul 28 01:26:01.731441 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724497 2578 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 28 01:26:01.731441 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724500 2578 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 28 01:26:01.731441 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724504 2578 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 28 01:26:01.731441 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724509 2578 flags.go:64] FLAG: --log-json-split-stream="false" Jul 28 01:26:01.731441 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724512 2578 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 28 01:26:01.731441 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724515 2578 flags.go:64] FLAG: --log-text-split-stream="false" Jul 28 01:26:01.731441 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724518 2578 flags.go:64] FLAG: --logging-format="text" Jul 28 01:26:01.731441 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724521 2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 28 01:26:01.731441 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724524 2578 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 28 01:26:01.731441 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724527 2578 flags.go:64] FLAG: --manifest-url="" Jul 28 01:26:01.731441 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724530 2578 flags.go:64] FLAG: --manifest-url-header="" Jul 28 01:26:01.731441 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724534 2578 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 28 01:26:01.731441 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724538 2578 flags.go:64] FLAG: --max-open-files="1000000" Jul 28 01:26:01.731441 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724554 2578 flags.go:64] FLAG: --max-pods="110" Jul 28 01:26:01.731441 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724558 2578 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 28 01:26:01.732112 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724561 2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 28 01:26:01.732112 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724564 2578 flags.go:64] FLAG: --memory-manager-policy="None" Jul 28 01:26:01.732112 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724567 2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 28 01:26:01.732112 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724570 2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 28 01:26:01.732112 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724573 2578 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 28 01:26:01.732112 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724576 2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 28 01:26:01.732112 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724585 2578 flags.go:64] FLAG: --node-status-max-images="50" Jul 28 01:26:01.732112 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724588 2578 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 28 01:26:01.732112 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724592 2578 flags.go:64] FLAG: --oom-score-adj="-999" Jul 28 01:26:01.732112 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724595 2578 flags.go:64] FLAG: --pod-cidr="" Jul 28 01:26:01.732112 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724598 2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 28 01:26:01.732112 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724603 2578 flags.go:64] FLAG: --pod-manifest-path="" Jul 28 01:26:01.732112 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724606 2578 flags.go:64] FLAG: --pod-max-pids="-1" Jul 28 01:26:01.732112 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724610 2578 flags.go:64] FLAG: --pods-per-core="0" Jul 28 01:26:01.732112 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724613 2578 flags.go:64] FLAG: --port="10250" Jul 28 01:26:01.732112 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724616 2578 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 28 01:26:01.732112 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724619 2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f481bb7f1151a809" Jul 28 01:26:01.732112 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724622 2578 flags.go:64] FLAG: --qos-reserved="" Jul 28 01:26:01.732112 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724625 2578 flags.go:64] FLAG: --read-only-port="10255" Jul 28 01:26:01.732112 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724628 2578 flags.go:64] FLAG: --register-node="true" Jul 28 01:26:01.732112 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724631 2578 flags.go:64] FLAG: --register-with-taints="" Jul 28 01:26:01.732112 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724639 2578 flags.go:64] FLAG: --registry-burst="10" Jul 28 01:26:01.732112 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724643 2578 flags.go:64] FLAG: --registry-qps="5" Jul 28 01:26:01.732112 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724646 2578 flags.go:64] FLAG: --reserved-cpus="" Jul 28 01:26:01.732710 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724649 2578 flags.go:64] FLAG: --reserved-memory="" Jul 28 01:26:01.732710 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724652 2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 28 01:26:01.732710 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724655 2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 28 01:26:01.732710 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724658 2578 flags.go:64] FLAG: --rotate-certificates="false" Jul 28 01:26:01.732710 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724661 2578 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 28 01:26:01.732710 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724664 2578 flags.go:64] FLAG: --runonce="false" Jul 28 01:26:01.732710 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724667 2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 28 01:26:01.732710 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724670 2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 28 01:26:01.732710 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724673 2578 flags.go:64] FLAG: --seccomp-default="false" Jul 28 01:26:01.732710 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724676 2578 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 28 01:26:01.732710 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724679 2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 28 01:26:01.732710 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724681 2578 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 28 01:26:01.732710 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724685 2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 28 01:26:01.732710 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724688 2578 flags.go:64] FLAG: --storage-driver-password="root" Jul 28 01:26:01.732710 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724691 2578 flags.go:64] FLAG: --storage-driver-secure="false" Jul 28 01:26:01.732710 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724694 2578 flags.go:64] FLAG: --storage-driver-table="stats" Jul 28 01:26:01.732710 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724697 2578 flags.go:64] FLAG: --storage-driver-user="root" Jul 28 01:26:01.732710 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724700 2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 28 01:26:01.732710 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724702 2578 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 28 01:26:01.732710 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724705 2578 flags.go:64] FLAG: --system-cgroups="" Jul 28 01:26:01.732710 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724708 2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 28 01:26:01.732710 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724714 2578 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 28 01:26:01.732710 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724717 2578 flags.go:64] FLAG: --tls-cert-file="" Jul 28 01:26:01.732710 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724720 2578 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 28 01:26:01.732710 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724724 2578 flags.go:64] FLAG: --tls-min-version="" Jul 28 01:26:01.733316 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724727 2578 flags.go:64] FLAG: --tls-private-key-file="" Jul 28 01:26:01.733316 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724730 2578 flags.go:64] FLAG: --topology-manager-policy="none" Jul 28 01:26:01.733316 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724732 2578 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 28 01:26:01.733316 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724735 2578 flags.go:64] FLAG: --topology-manager-scope="container" Jul 28 01:26:01.733316 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724738 2578 flags.go:64] FLAG: --v="2" Jul 28 01:26:01.733316 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724742 2578 flags.go:64] FLAG: --version="false" Jul 28 01:26:01.733316 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724746 2578 flags.go:64] FLAG: --vmodule="" Jul 28 01:26:01.733316 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724750 2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 28 01:26:01.733316 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.724753 2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 28 01:26:01.733316 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724870 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 01:26:01.733316 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724876 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 01:26:01.733316 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724880 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 01:26:01.733316 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724883 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 01:26:01.733316 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724886 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 01:26:01.733316 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724889 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 01:26:01.733316 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724891 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 01:26:01.733316 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724894 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 01:26:01.733316 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724897 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 01:26:01.733316 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724899 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 01:26:01.733316 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724902 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 01:26:01.733316 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724905 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 01:26:01.733897 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724907 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 01:26:01.733897 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724912 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 01:26:01.733897 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724915 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 01:26:01.733897 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724918 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 01:26:01.733897 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724921 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 01:26:01.733897 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724923 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 28 01:26:01.733897 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724926 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 01:26:01.733897 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724930 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 01:26:01.733897 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724934 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 01:26:01.733897 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724938 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 01:26:01.733897 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724942 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 01:26:01.733897 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724945 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 01:26:01.733897 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724948 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 01:26:01.733897 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724951 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 01:26:01.733897 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724954 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 01:26:01.733897 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724957 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 01:26:01.733897 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724960 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 01:26:01.733897 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724963 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 01:26:01.733897 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724971 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 01:26:01.734376 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724974 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 01:26:01.734376 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724977 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 01:26:01.734376 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724980 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 01:26:01.734376 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724982 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 01:26:01.734376 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724985 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 01:26:01.734376 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724987 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 01:26:01.734376 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724990 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 01:26:01.734376 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724993 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 01:26:01.734376 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724995 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 01:26:01.734376 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.724998 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 01:26:01.734376 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725001 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 01:26:01.734376 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725003 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 01:26:01.734376 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725006 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 01:26:01.734376 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725009 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 01:26:01.734376 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725013 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 01:26:01.734376 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725015 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 01:26:01.734376 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725018 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 01:26:01.734376 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725021 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 01:26:01.734376 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725024 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 01:26:01.734376 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725027 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 01:26:01.734876 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725030 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 01:26:01.734876 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725033 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 01:26:01.734876 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725036 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 01:26:01.734876 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725039 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 01:26:01.734876 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725041 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 01:26:01.734876 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725044 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 01:26:01.734876 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725046 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 01:26:01.734876 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725048 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 01:26:01.734876 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725051 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 01:26:01.734876 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725054 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 01:26:01.734876 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725056 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 01:26:01.734876 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725059 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 01:26:01.734876 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725062 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 01:26:01.734876 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725064 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 01:26:01.734876 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725067 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 01:26:01.734876 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725070 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 01:26:01.734876 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725072 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 01:26:01.734876 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725075 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 01:26:01.734876 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725077 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 01:26:01.734876 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725080 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 01:26:01.735372 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725082 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 01:26:01.735372 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725085 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 01:26:01.735372 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725087 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 01:26:01.735372 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725090 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 01:26:01.735372 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725093 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 01:26:01.735372 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725095 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 01:26:01.735372 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725099 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 01:26:01.735372 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725102 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 01:26:01.735372 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725104 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 01:26:01.735372 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725107 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 01:26:01.735372 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725109 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 01:26:01.735372 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725112 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 01:26:01.735372 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725115 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 01:26:01.735372 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725118 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 01:26:01.735372 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725121 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 01:26:01.735372 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725123 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 01:26:01.735372 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725126 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 01:26:01.735372 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725128 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 01:26:01.735372 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725131 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 01:26:01.735848 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725134 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 01:26:01.735848 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725136 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 01:26:01.735848 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725139 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 01:26:01.735848 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725141 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 01:26:01.735848 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725144 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 01:26:01.735848 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725147 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 01:26:01.735848 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725150 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 01:26:01.735848 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725153 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 01:26:01.735848 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.725155 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 01:26:01.735848 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.725718 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 01:26:01.735848 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.732268 2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 28 01:26:01.735848 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.732282 2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 28 01:26:01.735848 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.733011 2578 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 28 01:26:01.735848 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.733019 2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 28 01:26:01.735848 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733079 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 01:26:01.736218 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733085 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 01:26:01.736218 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733090 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 01:26:01.736218 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733093 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 01:26:01.736218 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733097 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 01:26:01.736218 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733100 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 01:26:01.736218 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733102 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 01:26:01.736218 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733105 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 01:26:01.736218 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733107 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 01:26:01.736218 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733110 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 01:26:01.736218 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733113 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 01:26:01.736218 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733116 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 01:26:01.736218 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733118 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 01:26:01.736218 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733121 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 01:26:01.736218 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733123 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 01:26:01.736218 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733126 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 01:26:01.736218 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733129 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 01:26:01.736218 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733131 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 01:26:01.736218 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733134 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 01:26:01.736218 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733137 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 01:26:01.736218 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733139 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 01:26:01.736714 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733142 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 01:26:01.736714 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733144 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 01:26:01.736714 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733147 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 01:26:01.736714 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733150 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 01:26:01.736714 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733153 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 01:26:01.736714 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733156 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 01:26:01.736714 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733159 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 01:26:01.736714 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733162 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 01:26:01.736714 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733166 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 01:26:01.736714 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733169 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 01:26:01.736714 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733171 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 01:26:01.736714 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733174 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 01:26:01.736714 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733177 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 01:26:01.736714 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733179 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 01:26:01.736714 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733182 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 01:26:01.736714 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733185 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 01:26:01.736714 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733187 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 01:26:01.736714 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733190 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 01:26:01.736714 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733193 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 01:26:01.736714 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733195 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 01:26:01.736714 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733198 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 01:26:01.737235 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733201 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 01:26:01.737235 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733205 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 01:26:01.737235 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733208 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 01:26:01.737235 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733212 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 01:26:01.737235 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733215 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 01:26:01.737235 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733218 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 01:26:01.737235 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733222 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 01:26:01.737235 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733224 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 01:26:01.737235 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733227 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 01:26:01.737235 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733230 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 01:26:01.737235 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733232 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 01:26:01.737235 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733235 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 01:26:01.737235 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733238 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 01:26:01.737235 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733240 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 01:26:01.737235 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733243 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 01:26:01.737235 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733246 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 01:26:01.737235 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733249 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 01:26:01.737235 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733251 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 01:26:01.737235 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733254 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 01:26:01.737797 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733257 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 01:26:01.737797 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733260 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 01:26:01.737797 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733263 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 01:26:01.737797 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733266 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 01:26:01.737797 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733268 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 01:26:01.737797 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733271 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 01:26:01.737797 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733273 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 01:26:01.737797 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733276 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 01:26:01.737797 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733278 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 01:26:01.737797 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733281 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 01:26:01.737797 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733283 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 01:26:01.737797 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733286 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 28 01:26:01.737797 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733289 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 01:26:01.737797 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733291 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 01:26:01.737797 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733294 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 01:26:01.737797 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733296 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 01:26:01.737797 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733299 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 01:26:01.737797 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733301 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 01:26:01.737797 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733304 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 01:26:01.737797 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733306 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 01:26:01.738278 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733309 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 01:26:01.738278 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733312 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 01:26:01.738278 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733315 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 01:26:01.738278 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733319 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 01:26:01.738278 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733322 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 01:26:01.738278 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733326 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 01:26:01.738278 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733329 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 01:26:01.738278 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733332 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 01:26:01.738278 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733334 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 01:26:01.738278 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733337 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 01:26:01.738278 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733340 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 01:26:01.738278 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733343 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 01:26:01.738278 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733345 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 01:26:01.738278 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733348 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 01:26:01.738278 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733351 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 01:26:01.738278 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733354 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 01:26:01.738278 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733356 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 01:26:01.738278 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733359 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 01:26:01.738725 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.733364 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 01:26:01.738725 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733841 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 01:26:01.738725 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733847 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 01:26:01.738725 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733851 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 01:26:01.738725 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733854 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 01:26:01.738725 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733858 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 01:26:01.738725 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733860 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 01:26:01.738725 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733864 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 01:26:01.738725 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733868 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 01:26:01.738725 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733871 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 01:26:01.738725 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733874 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 01:26:01.738725 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733876 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 01:26:01.738725 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733879 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 01:26:01.738725 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733881 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 01:26:01.738725 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733884 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 01:26:01.739086 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733887 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 01:26:01.739086 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733889 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 01:26:01.739086 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733892 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 01:26:01.739086 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733894 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 01:26:01.739086 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733897 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 01:26:01.739086 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733899 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 01:26:01.739086 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733902 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 01:26:01.739086 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733905 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 01:26:01.739086 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733908 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 01:26:01.739086 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733911 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 01:26:01.739086 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733913 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 01:26:01.739086 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733916 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 01:26:01.739086 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733919 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 01:26:01.739086 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733921 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 01:26:01.739086 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733924 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 01:26:01.739086 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733927 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 01:26:01.739086 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733930 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 01:26:01.739086 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733933 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 01:26:01.739086 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733935 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 01:26:01.739086 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733938 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 01:26:01.739582 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733941 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 01:26:01.739582 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733943 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 01:26:01.739582 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733946 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 01:26:01.739582 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733949 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 01:26:01.739582 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733951 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 01:26:01.739582 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733954 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 01:26:01.739582 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733957 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 01:26:01.739582 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733959 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 01:26:01.739582 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733962 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 01:26:01.739582 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733964 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 01:26:01.739582 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733967 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 01:26:01.739582 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733969 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 01:26:01.739582 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733972 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 01:26:01.739582 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733974 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 01:26:01.739582 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733977 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 01:26:01.739582 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733979 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 01:26:01.739582 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733981 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 01:26:01.739582 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733985 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 01:26:01.739582 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733989 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 01:26:01.739582 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733991 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 01:26:01.740068 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733994 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 01:26:01.740068 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733997 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 01:26:01.740068 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.733999 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 01:26:01.740068 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.734002 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 01:26:01.740068 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.734005 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 01:26:01.740068 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.734007 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 01:26:01.740068 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.734010 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 01:26:01.740068 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.734013 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 01:26:01.740068 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.734016 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 01:26:01.740068 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.734019 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 28 01:26:01.740068 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.734021 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 01:26:01.740068 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.734024 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 01:26:01.740068 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.734026 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 01:26:01.740068 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.734029 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 01:26:01.740068 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.734032 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 01:26:01.740068 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.734035 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 01:26:01.740068 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.734037 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 01:26:01.740068 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.734039 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 01:26:01.740068 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.734042 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 01:26:01.740068 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.734045 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 01:26:01.740579 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.734047 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 01:26:01.740579 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.734050 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 01:26:01.740579 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.734052 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 01:26:01.740579 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.734055 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 01:26:01.740579 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.734057 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 01:26:01.740579 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.734060 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 01:26:01.740579 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.734062 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 01:26:01.740579 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.734065 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 01:26:01.740579 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.734067 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 01:26:01.740579 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.734070 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 01:26:01.740579 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.734072 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 01:26:01.740579 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.734074 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 01:26:01.740579 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.734077 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 01:26:01.740579 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.734079 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 01:26:01.740579 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.734082 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 01:26:01.740579 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.734085 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 01:26:01.740579 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.734087 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 01:26:01.740579 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.734090 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 01:26:01.740579 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.734093 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 01:26:01.740579 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.734095 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 01:26:01.741172 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.734098 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 01:26:01.741172 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.734100 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 01:26:01.741172 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.734104 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 01:26:01.741172 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.734107 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 01:26:01.741172 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:01.734110 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 01:26:01.741172 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.734115 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 01:26:01.741172 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.734219 2578 server.go:962] "Client rotation is on, will bootstrap in background" Jul 28 01:26:01.741172 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.736062 2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 28 01:26:01.741172 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.737403 2578 server.go:1019] "Starting client certificate rotation" Jul 28 01:26:01.741172 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.737502 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 28 01:26:01.741172 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.737537 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 28 01:26:01.741172 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.739557 2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 28 01:26:01.744184 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.744161 2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 28 01:26:01.755156 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.755137 2578 log.go:25] "Validated CRI v1 runtime API" Jul 28 01:26:01.757159 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.757145 2578 log.go:25] "Validated CRI v1 image API" Jul 28 01:26:01.757945 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.757930 2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 28 01:26:01.760268 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.760244 2578 fs.go:135] Filesystem UUIDs: map[17b1c9f5-6157-43f2-b288-3f10108a9664:/dev/nvme0n1p3 667e41a2-1eec-49ef-aac4-53967bb91de8:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 28 01:26:01.760320 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.760269 2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 28 01:26:01.765573 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.765446 2578 manager.go:216] Machine: {Timestamp:2026-07-28 01:26:01.764120934 +0000 UTC m=+0.318245724 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101932 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2bb54977629195987998841ee9a88c SystemUUID:ec2bb549-7762-9195-9879-98841ee9a88c BootID:c28711ef-a190-455c-aa3e-0ffb8613c81c Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:5b:51:4d:bc:73 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:5b:51:4d:bc:73 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:b2:b9:71:ac:bf:47 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 28 01:26:01.765573 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.765568 2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 28 01:26:01.765697 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.765644 2578 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 28 01:26:01.767212 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.767197 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 28 01:26:01.768232 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.768208 2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 28 01:26:01.768382 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.768237 2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-186.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 28 01:26:01.768997 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.768985 2578 topology_manager.go:138] "Creating topology manager with none policy" Jul 28 01:26:01.769029 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.769001 2578 container_manager_linux.go:309] "Creating device plugin manager" Jul 28 01:26:01.769029 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.769009 2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 28 01:26:01.769648 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.769639 2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 28 01:26:01.770353 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.770344 2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 28 01:26:01.772031 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.772020 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 28 01:26:01.772155 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.772145 2578 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 28 01:26:01.772193 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.772175 2578 kubelet.go:486] "Attempting to sync node with API server" Jul 28 01:26:01.772193 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.772185 2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 28 01:26:01.772261 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.772196 2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 28 01:26:01.772261 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.772205 2578 kubelet.go:398] "Adding apiserver pod source" Jul 28 01:26:01.772261 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.772213 2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 28 01:26:01.772601 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.772559 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 28 01:26:01.772601 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.772585 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 28 01:26:01.774638 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.774619 2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 28 01:26:01.776134 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.776121 2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 28 01:26:01.776180 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.776152 2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 28 01:26:01.777679 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.777667 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 28 01:26:01.777747 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.777684 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 28 01:26:01.777747 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.777690 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 28 01:26:01.777747 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.777696 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 28 01:26:01.777747 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.777706 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 28 01:26:01.777747 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.777714 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 28 01:26:01.777747 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.777727 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 28 01:26:01.777747 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.777733 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 28 01:26:01.777747 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.777740 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 28 01:26:01.777747 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.777745 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 28 01:26:01.777971 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.777754 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 28 01:26:01.777971 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.777759 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 28 01:26:01.778586 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.778576 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 28 01:26:01.778586 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.778585 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 28 01:26:01.781949 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.781933 2578 server.go:1268] "Started kubelet" Jul 28 01:26:01.782039 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.782002 2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 28 01:26:01.782101 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.782014 2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 28 01:26:01.782149 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.782124 2578 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 28 01:26:01.782833 ip-10-0-143-186 systemd[1]: Started Kubernetes Kubelet. Jul 28 01:26:01.786101 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.786079 2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 28 01:26:01.786712 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:01.786681 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 28 01:26:01.786930 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.786432 2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-186.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 28 01:26:01.787020 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:01.786923 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-186.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 28 01:26:01.788197 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.788012 2578 server.go:310] "Adding debug handlers to kubelet server" Jul 28 01:26:01.788391 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:01.788345 2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 28 01:26:01.792779 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:01.791818 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-186.ec2.internal.18c64fb9d05739a0 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-186.ec2.internal,UID:ip-10-0-143-186.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-186.ec2.internal,},FirstTimestamp:2026-07-28 01:26:01.781901728 +0000 UTC m=+0.336026517,LastTimestamp:2026-07-28 01:26:01.781901728 +0000 UTC m=+0.336026517,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-186.ec2.internal,}" Jul 28 01:26:01.792882 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.792833 2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 28 01:26:01.792882 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.792849 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 28 01:26:01.792996 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:01.792900 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-186.ec2.internal\" not found" Jul 28 01:26:01.793358 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.793336 2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 28 01:26:01.793455 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.793367 2578 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 28 01:26:01.793455 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.793383 2578 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 28 01:26:01.793577 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.793501 2578 reconstruct.go:97] "Volume reconstruction finished" Jul 28 01:26:01.793577 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.793512 2578 reconciler.go:29] "Reconciler: start to sync state" Jul 28 01:26:01.793779 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.793763 2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 28 01:26:01.793779 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.793780 2578 factory.go:55] Registering systemd factory Jul 28 01:26:01.793884 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.793788 2578 factory.go:223] Registration of the systemd container factory successfully Jul 28 01:26:01.794041 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.793996 2578 factory.go:153] Registering CRI-O factory Jul 28 01:26:01.794041 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.794013 2578 factory.go:223] Registration of the crio container factory successfully Jul 28 01:26:01.794041 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.794036 2578 factory.go:103] Registering Raw factory Jul 28 01:26:01.794187 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.794048 2578 manager.go:1196] Started watching for new ooms in manager Jul 28 01:26:01.794412 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.794402 2578 manager.go:349] Starting recovery of all containers Jul 28 01:26:01.799342 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:01.799161 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 28 01:26:01.799342 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:01.799161 2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-186.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 28 01:26:01.803971 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.803953 2578 manager.go:354] Recovery completed Jul 28 01:26:01.808318 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.808306 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 01:26:01.810162 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.810140 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-186.ec2.internal" event="NodeHasSufficientMemory" Jul 28 01:26:01.810233 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.810176 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-186.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 01:26:01.810233 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.810187 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-186.ec2.internal" event="NodeHasSufficientPID" Jul 28 01:26:01.810743 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.810731 2578 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 28 01:26:01.810743 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.810743 2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 28 01:26:01.810820 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.810758 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 28 01:26:01.810820 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.810733 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 01:26:01.811538 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.811525 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-186.ec2.internal" event="NodeHasSufficientMemory" Jul 28 01:26:01.811671 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.811580 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-186.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 01:26:01.811671 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.811598 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-186.ec2.internal" event="NodeHasSufficientPID" Jul 28 01:26:01.812317 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.812304 2578 policy_none.go:49] "None policy: Start" Jul 28 01:26:01.812364 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.812324 2578 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 28 01:26:01.812364 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.812342 2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 28 01:26:01.812778 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:01.812417 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-186.ec2.internal.18c64fb9d2067421 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-186.ec2.internal,UID:ip-10-0-143-186.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-186.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-186.ec2.internal,},FirstTimestamp:2026-07-28 01:26:01.810162721 +0000 UTC m=+0.364287510,LastTimestamp:2026-07-28 01:26:01.810162721 +0000 UTC m=+0.364287510,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-186.ec2.internal,}" Jul 28 01:26:01.813720 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.813698 2578 policy_none.go:47] "Start" Jul 28 01:26:01.823877 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:01.823811 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-186.ec2.internal.18c64fb9d206bbfd default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-186.ec2.internal,UID:ip-10-0-143-186.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-186.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-186.ec2.internal,},FirstTimestamp:2026-07-28 01:26:01.810181117 +0000 UTC m=+0.364305907,LastTimestamp:2026-07-28 01:26:01.810181117 +0000 UTC m=+0.364305907,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-186.ec2.internal,}" Jul 28 01:26:01.835018 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:01.834929 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-186.ec2.internal.18c64fb9d206e216 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-186.ec2.internal,UID:ip-10-0-143-186.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-143-186.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-143-186.ec2.internal,},FirstTimestamp:2026-07-28 01:26:01.81019087 +0000 UTC m=+0.364315659,LastTimestamp:2026-07-28 01:26:01.81019087 +0000 UTC m=+0.364315659,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-186.ec2.internal,}" Jul 28 01:26:01.844931 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.844912 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7v29j" Jul 28 01:26:01.847049 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:01.846961 2578 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-186.ec2.internal.18c64fb9d2067421\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-186.ec2.internal.18c64fb9d2067421 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-186.ec2.internal,UID:ip-10-0-143-186.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-186.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-186.ec2.internal,},FirstTimestamp:2026-07-28 01:26:01.810162721 +0000 UTC m=+0.364287510,LastTimestamp:2026-07-28 01:26:01.811538106 +0000 UTC m=+0.365662896,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-186.ec2.internal,}" Jul 28 01:26:01.851199 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.851185 2578 manager.go:337] "Starting Device Plugin manager" Jul 28 01:26:01.851259 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:01.851218 2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 28 01:26:01.851259 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.851229 2578 server.go:87] "Starting device plugin registration server" Jul 28 01:26:01.851515 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.851501 2578 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 28 01:26:01.851576 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.851516 2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 28 01:26:01.851675 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.851658 2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 28 01:26:01.851899 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.851820 2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 28 01:26:01.851899 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.851835 2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 28 01:26:01.852428 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:01.852410 2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 28 01:26:01.852502 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:01.852466 2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-186.ec2.internal\" not found" Jul 28 01:26:01.854088 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.854071 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7v29j" Jul 28 01:26:01.857635 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:01.857551 2578 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-186.ec2.internal.18c64fb9d206bbfd\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-186.ec2.internal.18c64fb9d206bbfd default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-186.ec2.internal,UID:ip-10-0-143-186.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-186.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-186.ec2.internal,},FirstTimestamp:2026-07-28 01:26:01.810181117 +0000 UTC m=+0.364305907,LastTimestamp:2026-07-28 01:26:01.811588448 +0000 UTC m=+0.365713239,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-186.ec2.internal,}" Jul 28 01:26:01.951973 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.951926 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 01:26:01.953138 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.953009 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-186.ec2.internal" event="NodeHasSufficientMemory" Jul 28 01:26:01.953138 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.953044 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-186.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 01:26:01.953138 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.953054 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-186.ec2.internal" event="NodeHasSufficientPID" Jul 28 01:26:01.953138 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.953085 2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-186.ec2.internal" Jul 28 01:26:01.962052 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:01.962034 2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-186.ec2.internal" Jul 28 01:26:01.962141 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:01.962057 2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-186.ec2.internal\": node \"ip-10-0-143-186.ec2.internal\" not found" Jul 28 01:26:02.002514 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:02.002441 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-186.ec2.internal\" not found" Jul 28 01:26:02.002677 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.002655 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 28 01:26:02.003919 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.003901 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 28 01:26:02.003995 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.003946 2578 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 28 01:26:02.003995 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.003974 2578 kubelet.go:2448] "Starting kubelet main sync loop" Jul 28 01:26:02.004092 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:02.004021 2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 28 01:26:02.006510 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.006489 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 01:26:02.102667 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:02.102627 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-186.ec2.internal\" not found" Jul 28 01:26:02.104964 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.104941 2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-186.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-186.ec2.internal"] Jul 28 01:26:02.105050 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.105031 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 01:26:02.106773 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.106753 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-186.ec2.internal" event="NodeHasSufficientMemory" Jul 28 01:26:02.106860 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.106787 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-186.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 01:26:02.106860 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.106800 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-186.ec2.internal" event="NodeHasSufficientPID" Jul 28 01:26:02.108677 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.108664 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 01:26:02.108801 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.108786 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-186.ec2.internal" Jul 28 01:26:02.108851 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.108821 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 01:26:02.112210 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.112192 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-186.ec2.internal" event="NodeHasSufficientMemory" Jul 28 01:26:02.112312 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.112229 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-186.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 01:26:02.112312 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.112244 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-186.ec2.internal" event="NodeHasSufficientPID" Jul 28 01:26:02.112312 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.112196 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-186.ec2.internal" event="NodeHasSufficientMemory" Jul 28 01:26:02.112312 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.112300 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-186.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 01:26:02.112312 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.112311 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-186.ec2.internal" event="NodeHasSufficientPID" Jul 28 01:26:02.114543 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.114526 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-186.ec2.internal" Jul 28 01:26:02.114655 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.114580 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 01:26:02.115316 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.115301 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-186.ec2.internal" event="NodeHasSufficientMemory" Jul 28 01:26:02.115402 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.115331 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-186.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 01:26:02.115402 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.115347 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-186.ec2.internal" event="NodeHasSufficientPID" Jul 28 01:26:02.140460 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:02.140438 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-186.ec2.internal\" not found" node="ip-10-0-143-186.ec2.internal" Jul 28 01:26:02.145433 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:02.145418 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-186.ec2.internal\" not found" node="ip-10-0-143-186.ec2.internal" Jul 28 01:26:02.195083 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.195059 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9fe4fb4d5152991d383fb6f3c8471009-config\") pod \"kube-apiserver-proxy-ip-10-0-143-186.ec2.internal\" (UID: \"9fe4fb4d5152991d383fb6f3c8471009\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-186.ec2.internal" Jul 28 01:26:02.195083 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.195085 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/45f7ea7075c86f8ee5fac1e99899f141-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-186.ec2.internal\" (UID: \"45f7ea7075c86f8ee5fac1e99899f141\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-186.ec2.internal" Jul 28 01:26:02.195252 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.195105 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/45f7ea7075c86f8ee5fac1e99899f141-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-186.ec2.internal\" (UID: \"45f7ea7075c86f8ee5fac1e99899f141\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-186.ec2.internal" Jul 28 01:26:02.202739 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:02.202716 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-186.ec2.internal\" not found" Jul 28 01:26:02.295838 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.295751 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/45f7ea7075c86f8ee5fac1e99899f141-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-186.ec2.internal\" (UID: \"45f7ea7075c86f8ee5fac1e99899f141\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-186.ec2.internal" Jul 28 01:26:02.295838 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.295785 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/45f7ea7075c86f8ee5fac1e99899f141-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-186.ec2.internal\" (UID: \"45f7ea7075c86f8ee5fac1e99899f141\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-186.ec2.internal" Jul 28 01:26:02.295838 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.295804 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9fe4fb4d5152991d383fb6f3c8471009-config\") pod \"kube-apiserver-proxy-ip-10-0-143-186.ec2.internal\" (UID: \"9fe4fb4d5152991d383fb6f3c8471009\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-186.ec2.internal" Jul 28 01:26:02.295838 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.295833 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/45f7ea7075c86f8ee5fac1e99899f141-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-186.ec2.internal\" (UID: \"45f7ea7075c86f8ee5fac1e99899f141\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-186.ec2.internal" Jul 28 01:26:02.296132 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.295876 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/45f7ea7075c86f8ee5fac1e99899f141-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-186.ec2.internal\" (UID: \"45f7ea7075c86f8ee5fac1e99899f141\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-186.ec2.internal" Jul 28 01:26:02.296132 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.295908 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9fe4fb4d5152991d383fb6f3c8471009-config\") pod \"kube-apiserver-proxy-ip-10-0-143-186.ec2.internal\" (UID: \"9fe4fb4d5152991d383fb6f3c8471009\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-186.ec2.internal" Jul 28 01:26:02.302797 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:02.302774 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-186.ec2.internal\" not found" Jul 28 01:26:02.403886 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:02.403847 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-186.ec2.internal\" not found" Jul 28 01:26:02.445176 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.445140 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-186.ec2.internal" Jul 28 01:26:02.448006 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.447985 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-186.ec2.internal" Jul 28 01:26:02.504629 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:02.504593 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-186.ec2.internal\" not found" Jul 28 01:26:02.605207 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:02.605126 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-186.ec2.internal\" not found" Jul 28 01:26:02.706038 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:02.705993 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-186.ec2.internal\" not found" Jul 28 01:26:02.738225 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.738192 2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 28 01:26:02.738769 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.738414 2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 28 01:26:02.806124 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:02.806092 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-186.ec2.internal\" not found" Jul 28 01:26:02.812499 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.812479 2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 01:26:02.858124 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.858027 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 01:21:02 +0000 UTC" deadline="2027-12-22 02:02:12.674350589 +0000 UTC" Jul 28 01:26:02.858124 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.858075 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12288h36m9.816280495s" Jul 28 01:26:02.907186 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:02.907143 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-186.ec2.internal\" not found" Jul 28 01:26:02.913551 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.913530 2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 01:26:02.925955 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.925932 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 01:26:02.926948 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.926928 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-186.ec2.internal" event="NodeHasSufficientMemory" Jul 28 01:26:02.927050 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.926966 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-186.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 01:26:02.927050 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.926978 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-186.ec2.internal" event="NodeHasSufficientPID" Jul 28 01:26:02.930965 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.930935 2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 01:26:02.994167 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:02.994113 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-186.ec2.internal" Jul 28 01:26:03.009338 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.009312 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 28 01:26:03.010672 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.010656 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-186.ec2.internal" Jul 28 01:26:03.019302 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.019281 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 28 01:26:03.466090 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:03.466053 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45f7ea7075c86f8ee5fac1e99899f141.slice/crio-7d790dac6d9c6ca3bc2fd1ef0c31fc2c9463612c0245e90944677a3f62434419 WatchSource:0}: Error finding container 7d790dac6d9c6ca3bc2fd1ef0c31fc2c9463612c0245e90944677a3f62434419: Status 404 returned error can't find the container with id 7d790dac6d9c6ca3bc2fd1ef0c31fc2c9463612c0245e90944677a3f62434419 Jul 28 01:26:03.466312 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:03.466291 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fe4fb4d5152991d383fb6f3c8471009.slice/crio-8a4959c32b1492301cfc0c7ad213af4865da3593aa88e4a833596dab0aa79ba8 WatchSource:0}: Error finding container 8a4959c32b1492301cfc0c7ad213af4865da3593aa88e4a833596dab0aa79ba8: Status 404 returned error can't find the container with id 8a4959c32b1492301cfc0c7ad213af4865da3593aa88e4a833596dab0aa79ba8 Jul 28 01:26:03.471842 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.471825 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 01:26:03.729190 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.729103 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 01:26:03.774786 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.774758 2578 apiserver.go:52] "Watching apiserver" Jul 28 01:26:03.783188 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.783161 2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 28 01:26:03.784405 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.784383 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-7gfx9","kube-system/konnectivity-agent-hnpn2","kube-system/kube-apiserver-proxy-ip-10-0-143-186.ec2.internal","openshift-dns/node-resolver-2vjkb","openshift-multus/multus-x598j","openshift-network-diagnostics/network-check-target-2znt8","openshift-network-operator/iptables-alerter-qs68r","openshift-ovn-kubernetes/ovnkube-node-c6m66","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ww5wn","openshift-cluster-node-tuning-operator/tuned-kmk8x","openshift-image-registry/node-ca-bgvmj","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-186.ec2.internal","openshift-multus/multus-additional-cni-plugins-wgdbs","openshift-multus/network-metrics-daemon-brftp"] Jul 28 01:26:03.786810 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.786789 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qs68r" Jul 28 01:26:03.787935 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.787913 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wgdbs" Jul 28 01:26:03.788931 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.788906 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brftp" Jul 28 01:26:03.789050 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:03.789025 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-brftp" podUID="005aafcd-65c4-45cb-aa21-484a0c94167e" Jul 28 01:26:03.789979 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.789962 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 28 01:26:03.790279 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.789986 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 28 01:26:03.790279 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.789964 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 28 01:26:03.790279 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.790038 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-5t7gd\"" Jul 28 01:26:03.790279 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.790055 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2vjkb" Jul 28 01:26:03.791406 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.791193 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-x598j" Jul 28 01:26:03.791406 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.791214 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 28 01:26:03.791406 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.791224 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 28 01:26:03.791406 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.791254 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qg8n9\"" Jul 28 01:26:03.791406 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.791324 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 28 01:26:03.791406 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.791220 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 28 01:26:03.791702 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.791668 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 28 01:26:03.792369 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.792352 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2znt8" Jul 28 01:26:03.792441 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:03.792418 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2znt8" podUID="5060905d-b8ac-4bbd-94cb-b1a8a8952a9a" Jul 28 01:26:03.792669 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.792654 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 28 01:26:03.792746 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.792693 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 28 01:26:03.793361 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.793301 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-gmbq2\"" Jul 28 01:26:03.793361 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.793306 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 28 01:26:03.793726 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.793704 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ww5wn" Jul 28 01:26:03.794128 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.794108 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rvgf2\"" Jul 28 01:26:03.794220 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.794160 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 28 01:26:03.795340 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.795321 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gfx9" Jul 28 01:26:03.795452 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:03.795428 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gfx9" podUID="1bf2b1e4-0123-4a0b-9a30-b2ea3794732c" Jul 28 01:26:03.796608 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.796584 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-v85qb\"" Jul 28 01:26:03.796719 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.796625 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hnpn2" Jul 28 01:26:03.796902 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.796878 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 28 01:26:03.796975 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.796892 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 28 01:26:03.797228 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.797207 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 28 01:26:03.798893 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.798873 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:03.799182 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.799159 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 28 01:26:03.799734 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.799713 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-gw7ts\"" Jul 28 01:26:03.799823 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.799811 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 28 01:26:03.799884 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.799856 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bgvmj" Jul 28 01:26:03.801243 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.801224 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:03.801333 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.801263 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zbmt2\"" Jul 28 01:26:03.801512 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.801266 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 28 01:26:03.801794 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.801778 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 28 01:26:03.802380 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.802361 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 28 01:26:03.802875 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.802846 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 28 01:26:03.802958 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.802848 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-gqnqs\"" Jul 28 01:26:03.802958 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.802848 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 28 01:26:03.803792 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.803775 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 28 01:26:03.805457 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.805268 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qgtl7\"" Jul 28 01:26:03.805457 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.805303 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 28 01:26:03.805457 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.805307 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 28 01:26:03.805457 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.805323 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 28 01:26:03.805457 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.805345 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 28 01:26:03.805457 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.805274 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 28 01:26:03.806355 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.806335 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3394f81c-6c57-451b-8d8e-2b6504f34ed3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wgdbs\" (UID: \"3394f81c-6c57-451b-8d8e-2b6504f34ed3\") " pod="openshift-multus/multus-additional-cni-plugins-wgdbs" Jul 28 01:26:03.806416 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.806361 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xl8vf\" (UniqueName: \"kubernetes.io/projected/3394f81c-6c57-451b-8d8e-2b6504f34ed3-kube-api-access-xl8vf\") pod \"multus-additional-cni-plugins-wgdbs\" (UID: \"3394f81c-6c57-451b-8d8e-2b6504f34ed3\") " pod="openshift-multus/multus-additional-cni-plugins-wgdbs" Jul 28 01:26:03.806416 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.806377 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7af7909b-e421-437d-b0a8-895ad38876c7-etc-kubernetes\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.806416 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.806393 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cfb7\" (UniqueName: \"kubernetes.io/projected/5060905d-b8ac-4bbd-94cb-b1a8a8952a9a-kube-api-access-8cfb7\") pod \"network-check-target-2znt8\" (UID: \"5060905d-b8ac-4bbd-94cb-b1a8a8952a9a\") " pod="openshift-network-diagnostics/network-check-target-2znt8" Jul 28 01:26:03.806416 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.806411 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f80c0089-ca61-4ddb-850c-6c805e916aaf-registration-dir\") pod \"aws-ebs-csi-driver-node-ww5wn\" (UID: \"f80c0089-ca61-4ddb-850c-6c805e916aaf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ww5wn" Jul 28 01:26:03.806681 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.806455 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f80c0089-ca61-4ddb-850c-6c805e916aaf-sys-fs\") pod \"aws-ebs-csi-driver-node-ww5wn\" (UID: \"f80c0089-ca61-4ddb-850c-6c805e916aaf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ww5wn" Jul 28 01:26:03.806681 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.806486 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwk2d\" (UniqueName: \"kubernetes.io/projected/f80c0089-ca61-4ddb-850c-6c805e916aaf-kube-api-access-zwk2d\") pod \"aws-ebs-csi-driver-node-ww5wn\" (UID: \"f80c0089-ca61-4ddb-850c-6c805e916aaf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ww5wn" Jul 28 01:26:03.806681 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.806504 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7af7909b-e421-437d-b0a8-895ad38876c7-os-release\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.806681 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.806521 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7af7909b-e421-437d-b0a8-895ad38876c7-multus-daemon-config\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.806681 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.806542 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1bf2b1e4-0123-4a0b-9a30-b2ea3794732c-dbus\") pod \"global-pull-secret-syncer-7gfx9\" (UID: \"1bf2b1e4-0123-4a0b-9a30-b2ea3794732c\") " pod="kube-system/global-pull-secret-syncer-7gfx9" Jul 28 01:26:03.806681 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.806561 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3f0ab34c-4530-44b3-9bea-e54cdbb50377-iptables-alerter-script\") pod \"iptables-alerter-qs68r\" (UID: \"3f0ab34c-4530-44b3-9bea-e54cdbb50377\") " pod="openshift-network-operator/iptables-alerter-qs68r" Jul 28 01:26:03.806681 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.806583 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3f0ab34c-4530-44b3-9bea-e54cdbb50377-host-slash\") pod \"iptables-alerter-qs68r\" (UID: \"3f0ab34c-4530-44b3-9bea-e54cdbb50377\") " pod="openshift-network-operator/iptables-alerter-qs68r" Jul 28 01:26:03.806681 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.806598 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/516e3ead-ffab-4201-9715-2684dcfc7aa6-hosts-file\") pod \"node-resolver-2vjkb\" (UID: \"516e3ead-ffab-4201-9715-2684dcfc7aa6\") " pod="openshift-dns/node-resolver-2vjkb" Jul 28 01:26:03.806681 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.806614 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75z85\" (UniqueName: \"kubernetes.io/projected/005aafcd-65c4-45cb-aa21-484a0c94167e-kube-api-access-75z85\") pod \"network-metrics-daemon-brftp\" (UID: \"005aafcd-65c4-45cb-aa21-484a0c94167e\") " pod="openshift-multus/network-metrics-daemon-brftp" Jul 28 01:26:03.806681 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.806649 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7af7909b-e421-437d-b0a8-895ad38876c7-multus-cni-dir\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.806681 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.806667 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7af7909b-e421-437d-b0a8-895ad38876c7-host-run-k8s-cni-cncf-io\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.806681 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.806682 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7af7909b-e421-437d-b0a8-895ad38876c7-host-run-multus-certs\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.807286 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.806698 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f80c0089-ca61-4ddb-850c-6c805e916aaf-etc-selinux\") pod \"aws-ebs-csi-driver-node-ww5wn\" (UID: \"f80c0089-ca61-4ddb-850c-6c805e916aaf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ww5wn" Jul 28 01:26:03.807286 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.806734 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4n4kz\" (UniqueName: \"kubernetes.io/projected/3f0ab34c-4530-44b3-9bea-e54cdbb50377-kube-api-access-4n4kz\") pod \"iptables-alerter-qs68r\" (UID: \"3f0ab34c-4530-44b3-9bea-e54cdbb50377\") " pod="openshift-network-operator/iptables-alerter-qs68r" Jul 28 01:26:03.807286 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.806750 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3394f81c-6c57-451b-8d8e-2b6504f34ed3-cnibin\") pod \"multus-additional-cni-plugins-wgdbs\" (UID: \"3394f81c-6c57-451b-8d8e-2b6504f34ed3\") " pod="openshift-multus/multus-additional-cni-plugins-wgdbs" Jul 28 01:26:03.807286 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.806768 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7af7909b-e421-437d-b0a8-895ad38876c7-multus-socket-dir-parent\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.807286 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.806792 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7af7909b-e421-437d-b0a8-895ad38876c7-multus-conf-dir\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.807286 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.806816 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1bf2b1e4-0123-4a0b-9a30-b2ea3794732c-kubelet-config\") pod \"global-pull-secret-syncer-7gfx9\" (UID: \"1bf2b1e4-0123-4a0b-9a30-b2ea3794732c\") " pod="kube-system/global-pull-secret-syncer-7gfx9" Jul 28 01:26:03.807286 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.806833 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f80c0089-ca61-4ddb-850c-6c805e916aaf-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ww5wn\" (UID: \"f80c0089-ca61-4ddb-850c-6c805e916aaf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ww5wn" Jul 28 01:26:03.807286 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.806847 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f80c0089-ca61-4ddb-850c-6c805e916aaf-socket-dir\") pod \"aws-ebs-csi-driver-node-ww5wn\" (UID: \"f80c0089-ca61-4ddb-850c-6c805e916aaf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ww5wn" Jul 28 01:26:03.807286 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.806873 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3394f81c-6c57-451b-8d8e-2b6504f34ed3-cni-binary-copy\") pod \"multus-additional-cni-plugins-wgdbs\" (UID: \"3394f81c-6c57-451b-8d8e-2b6504f34ed3\") " pod="openshift-multus/multus-additional-cni-plugins-wgdbs" Jul 28 01:26:03.807286 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.806925 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7af7909b-e421-437d-b0a8-895ad38876c7-host-var-lib-cni-multus\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.807286 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.806941 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1bf2b1e4-0123-4a0b-9a30-b2ea3794732c-original-pull-secret\") pod \"global-pull-secret-syncer-7gfx9\" (UID: \"1bf2b1e4-0123-4a0b-9a30-b2ea3794732c\") " pod="kube-system/global-pull-secret-syncer-7gfx9" Jul 28 01:26:03.807286 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.806967 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbjmb\" (UniqueName: \"kubernetes.io/projected/516e3ead-ffab-4201-9715-2684dcfc7aa6-kube-api-access-qbjmb\") pod \"node-resolver-2vjkb\" (UID: \"516e3ead-ffab-4201-9715-2684dcfc7aa6\") " pod="openshift-dns/node-resolver-2vjkb" Jul 28 01:26:03.807286 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.806983 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f80c0089-ca61-4ddb-850c-6c805e916aaf-device-dir\") pod \"aws-ebs-csi-driver-node-ww5wn\" (UID: \"f80c0089-ca61-4ddb-850c-6c805e916aaf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ww5wn" Jul 28 01:26:03.807286 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.806997 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3394f81c-6c57-451b-8d8e-2b6504f34ed3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wgdbs\" (UID: \"3394f81c-6c57-451b-8d8e-2b6504f34ed3\") " pod="openshift-multus/multus-additional-cni-plugins-wgdbs" Jul 28 01:26:03.807286 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.807012 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3394f81c-6c57-451b-8d8e-2b6504f34ed3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wgdbs\" (UID: \"3394f81c-6c57-451b-8d8e-2b6504f34ed3\") " pod="openshift-multus/multus-additional-cni-plugins-wgdbs" Jul 28 01:26:03.807286 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.807026 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7af7909b-e421-437d-b0a8-895ad38876c7-hostroot\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.808097 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.807059 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7af7909b-e421-437d-b0a8-895ad38876c7-system-cni-dir\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.808097 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.807094 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7af7909b-e421-437d-b0a8-895ad38876c7-cnibin\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.808097 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.807119 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7af7909b-e421-437d-b0a8-895ad38876c7-host-run-netns\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.808097 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.807139 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7af7909b-e421-437d-b0a8-895ad38876c7-host-var-lib-cni-bin\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.808097 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.807155 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6q8zh\" (UniqueName: \"kubernetes.io/projected/7af7909b-e421-437d-b0a8-895ad38876c7-kube-api-access-6q8zh\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.808097 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.807173 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3394f81c-6c57-451b-8d8e-2b6504f34ed3-system-cni-dir\") pod \"multus-additional-cni-plugins-wgdbs\" (UID: \"3394f81c-6c57-451b-8d8e-2b6504f34ed3\") " pod="openshift-multus/multus-additional-cni-plugins-wgdbs" Jul 28 01:26:03.808097 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.807197 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3394f81c-6c57-451b-8d8e-2b6504f34ed3-os-release\") pod \"multus-additional-cni-plugins-wgdbs\" (UID: \"3394f81c-6c57-451b-8d8e-2b6504f34ed3\") " pod="openshift-multus/multus-additional-cni-plugins-wgdbs" Jul 28 01:26:03.808097 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.807216 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/005aafcd-65c4-45cb-aa21-484a0c94167e-metrics-certs\") pod \"network-metrics-daemon-brftp\" (UID: \"005aafcd-65c4-45cb-aa21-484a0c94167e\") " pod="openshift-multus/network-metrics-daemon-brftp" Jul 28 01:26:03.808097 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.807231 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7af7909b-e421-437d-b0a8-895ad38876c7-cni-binary-copy\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.808097 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.807246 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7af7909b-e421-437d-b0a8-895ad38876c7-host-var-lib-kubelet\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.808097 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.807272 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/516e3ead-ffab-4201-9715-2684dcfc7aa6-tmp-dir\") pod \"node-resolver-2vjkb\" (UID: \"516e3ead-ffab-4201-9715-2684dcfc7aa6\") " pod="openshift-dns/node-resolver-2vjkb" Jul 28 01:26:03.808097 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.807806 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 28 01:26:03.837473 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.837449 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-mh588" Jul 28 01:26:03.848024 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.848000 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-mh588" Jul 28 01:26:03.894730 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.894679 2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 28 01:26:03.907910 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.907876 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7af7909b-e421-437d-b0a8-895ad38876c7-host-var-lib-cni-bin\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.908101 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.907924 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6q8zh\" (UniqueName: \"kubernetes.io/projected/7af7909b-e421-437d-b0a8-895ad38876c7-kube-api-access-6q8zh\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.908101 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.907955 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7af7909b-e421-437d-b0a8-895ad38876c7-host-var-lib-cni-bin\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.908101 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.907971 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-host-run-ovn-kubernetes\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:03.908101 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.908017 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/03b14c3f-225a-4d46-9f03-31f3c8da2916-etc-sysctl-conf\") pod \"tuned-kmk8x\" (UID: \"03b14c3f-225a-4d46-9f03-31f3c8da2916\") " pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:03.908286 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.908127 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9g57\" (UniqueName: \"kubernetes.io/projected/03b14c3f-225a-4d46-9f03-31f3c8da2916-kube-api-access-j9g57\") pod \"tuned-kmk8x\" (UID: \"03b14c3f-225a-4d46-9f03-31f3c8da2916\") " pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:03.908286 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.908167 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/005aafcd-65c4-45cb-aa21-484a0c94167e-metrics-certs\") pod \"network-metrics-daemon-brftp\" (UID: \"005aafcd-65c4-45cb-aa21-484a0c94167e\") " pod="openshift-multus/network-metrics-daemon-brftp" Jul 28 01:26:03.908286 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.908204 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/516e3ead-ffab-4201-9715-2684dcfc7aa6-tmp-dir\") pod \"node-resolver-2vjkb\" (UID: \"516e3ead-ffab-4201-9715-2684dcfc7aa6\") " pod="openshift-dns/node-resolver-2vjkb" Jul 28 01:26:03.908286 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.908228 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-host-slash\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:03.908528 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.908289 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3a77a2d8-54f1-4eae-b356-213cd00c7462-ovnkube-config\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:03.908528 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.908327 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/751f8878-a0ed-4756-93c1-97a66b2afd10-agent-certs\") pod \"konnectivity-agent-hnpn2\" (UID: \"751f8878-a0ed-4756-93c1-97a66b2afd10\") " pod="kube-system/konnectivity-agent-hnpn2" Jul 28 01:26:03.908528 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.908366 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e1808965-dc33-46a8-a355-b3ebbe849c65-host\") pod \"node-ca-bgvmj\" (UID: \"e1808965-dc33-46a8-a355-b3ebbe849c65\") " pod="openshift-image-registry/node-ca-bgvmj" Jul 28 01:26:03.908528 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.908393 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/03b14c3f-225a-4d46-9f03-31f3c8da2916-lib-modules\") pod \"tuned-kmk8x\" (UID: \"03b14c3f-225a-4d46-9f03-31f3c8da2916\") " pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:03.908528 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.908433 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7af7909b-e421-437d-b0a8-895ad38876c7-etc-kubernetes\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.908528 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.908461 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cfb7\" (UniqueName: \"kubernetes.io/projected/5060905d-b8ac-4bbd-94cb-b1a8a8952a9a-kube-api-access-8cfb7\") pod \"network-check-target-2znt8\" (UID: \"5060905d-b8ac-4bbd-94cb-b1a8a8952a9a\") " pod="openshift-network-diagnostics/network-check-target-2znt8" Jul 28 01:26:03.908846 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.908528 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7af7909b-e421-437d-b0a8-895ad38876c7-etc-kubernetes\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.908846 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.908545 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/516e3ead-ffab-4201-9715-2684dcfc7aa6-tmp-dir\") pod \"node-resolver-2vjkb\" (UID: \"516e3ead-ffab-4201-9715-2684dcfc7aa6\") " pod="openshift-dns/node-resolver-2vjkb" Jul 28 01:26:03.908846 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.908562 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-run-systemd\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:03.908846 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.908616 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03b14c3f-225a-4d46-9f03-31f3c8da2916-etc-kubernetes\") pod \"tuned-kmk8x\" (UID: \"03b14c3f-225a-4d46-9f03-31f3c8da2916\") " pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:03.908846 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.908651 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7af7909b-e421-437d-b0a8-895ad38876c7-os-release\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.908846 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.908680 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7af7909b-e421-437d-b0a8-895ad38876c7-multus-daemon-config\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.908846 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.908728 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1bf2b1e4-0123-4a0b-9a30-b2ea3794732c-dbus\") pod \"global-pull-secret-syncer-7gfx9\" (UID: \"1bf2b1e4-0123-4a0b-9a30-b2ea3794732c\") " pod="kube-system/global-pull-secret-syncer-7gfx9" Jul 28 01:26:03.908846 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:03.908754 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 01:26:03.908846 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.908768 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3f0ab34c-4530-44b3-9bea-e54cdbb50377-host-slash\") pod \"iptables-alerter-qs68r\" (UID: \"3f0ab34c-4530-44b3-9bea-e54cdbb50377\") " pod="openshift-network-operator/iptables-alerter-qs68r" Jul 28 01:26:03.908846 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.908833 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3f0ab34c-4530-44b3-9bea-e54cdbb50377-host-slash\") pod \"iptables-alerter-qs68r\" (UID: \"3f0ab34c-4530-44b3-9bea-e54cdbb50377\") " pod="openshift-network-operator/iptables-alerter-qs68r" Jul 28 01:26:03.909296 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.908871 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7af7909b-e421-437d-b0a8-895ad38876c7-os-release\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.909296 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:03.908913 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/005aafcd-65c4-45cb-aa21-484a0c94167e-metrics-certs podName:005aafcd-65c4-45cb-aa21-484a0c94167e nodeName:}" failed. No retries permitted until 2026-07-28 01:26:04.40889046 +0000 UTC m=+2.963015238 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/005aafcd-65c4-45cb-aa21-484a0c94167e-metrics-certs") pod "network-metrics-daemon-brftp" (UID: "005aafcd-65c4-45cb-aa21-484a0c94167e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 01:26:03.909296 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.909063 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-run-openvswitch\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:03.909296 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.909103 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e1808965-dc33-46a8-a355-b3ebbe849c65-serviceca\") pod \"node-ca-bgvmj\" (UID: \"e1808965-dc33-46a8-a355-b3ebbe849c65\") " pod="openshift-image-registry/node-ca-bgvmj" Jul 28 01:26:03.909296 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.909138 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/03b14c3f-225a-4d46-9f03-31f3c8da2916-etc-sysctl-d\") pod \"tuned-kmk8x\" (UID: \"03b14c3f-225a-4d46-9f03-31f3c8da2916\") " pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:03.909296 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.909135 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1bf2b1e4-0123-4a0b-9a30-b2ea3794732c-dbus\") pod \"global-pull-secret-syncer-7gfx9\" (UID: \"1bf2b1e4-0123-4a0b-9a30-b2ea3794732c\") " pod="kube-system/global-pull-secret-syncer-7gfx9" Jul 28 01:26:03.909296 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.909182 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-host-kubelet\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:03.909296 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.909210 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4n4kz\" (UniqueName: \"kubernetes.io/projected/3f0ab34c-4530-44b3-9bea-e54cdbb50377-kube-api-access-4n4kz\") pod \"iptables-alerter-qs68r\" (UID: \"3f0ab34c-4530-44b3-9bea-e54cdbb50377\") " pod="openshift-network-operator/iptables-alerter-qs68r" Jul 28 01:26:03.909296 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.909241 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3394f81c-6c57-451b-8d8e-2b6504f34ed3-cnibin\") pod \"multus-additional-cni-plugins-wgdbs\" (UID: \"3394f81c-6c57-451b-8d8e-2b6504f34ed3\") " pod="openshift-multus/multus-additional-cni-plugins-wgdbs" Jul 28 01:26:03.909296 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.909257 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-host-cni-bin\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:03.909296 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.909295 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:03.909891 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.909334 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/03b14c3f-225a-4d46-9f03-31f3c8da2916-etc-sysconfig\") pod \"tuned-kmk8x\" (UID: \"03b14c3f-225a-4d46-9f03-31f3c8da2916\") " pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:03.909891 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.909385 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3394f81c-6c57-451b-8d8e-2b6504f34ed3-cnibin\") pod \"multus-additional-cni-plugins-wgdbs\" (UID: \"3394f81c-6c57-451b-8d8e-2b6504f34ed3\") " pod="openshift-multus/multus-additional-cni-plugins-wgdbs" Jul 28 01:26:03.909891 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.909437 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7af7909b-e421-437d-b0a8-895ad38876c7-multus-socket-dir-parent\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.909891 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.909448 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7af7909b-e421-437d-b0a8-895ad38876c7-multus-daemon-config\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.909891 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.909495 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7af7909b-e421-437d-b0a8-895ad38876c7-multus-conf-dir\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.909891 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.909535 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7af7909b-e421-437d-b0a8-895ad38876c7-multus-socket-dir-parent\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.909891 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.909554 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1bf2b1e4-0123-4a0b-9a30-b2ea3794732c-kubelet-config\") pod \"global-pull-secret-syncer-7gfx9\" (UID: \"1bf2b1e4-0123-4a0b-9a30-b2ea3794732c\") " pod="kube-system/global-pull-secret-syncer-7gfx9" Jul 28 01:26:03.909891 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.909599 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7af7909b-e421-437d-b0a8-895ad38876c7-multus-conf-dir\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.909891 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.909662 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f80c0089-ca61-4ddb-850c-6c805e916aaf-socket-dir\") pod \"aws-ebs-csi-driver-node-ww5wn\" (UID: \"f80c0089-ca61-4ddb-850c-6c805e916aaf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ww5wn" Jul 28 01:26:03.909891 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.909728 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1bf2b1e4-0123-4a0b-9a30-b2ea3794732c-kubelet-config\") pod \"global-pull-secret-syncer-7gfx9\" (UID: \"1bf2b1e4-0123-4a0b-9a30-b2ea3794732c\") " pod="kube-system/global-pull-secret-syncer-7gfx9" Jul 28 01:26:03.909891 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.909793 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3394f81c-6c57-451b-8d8e-2b6504f34ed3-cni-binary-copy\") pod \"multus-additional-cni-plugins-wgdbs\" (UID: \"3394f81c-6c57-451b-8d8e-2b6504f34ed3\") " pod="openshift-multus/multus-additional-cni-plugins-wgdbs" Jul 28 01:26:03.909891 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.909849 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f80c0089-ca61-4ddb-850c-6c805e916aaf-socket-dir\") pod \"aws-ebs-csi-driver-node-ww5wn\" (UID: \"f80c0089-ca61-4ddb-850c-6c805e916aaf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ww5wn" Jul 28 01:26:03.909891 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.909868 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3394f81c-6c57-451b-8d8e-2b6504f34ed3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wgdbs\" (UID: \"3394f81c-6c57-451b-8d8e-2b6504f34ed3\") " pod="openshift-multus/multus-additional-cni-plugins-wgdbs" Jul 28 01:26:03.909891 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.909899 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-node-log\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:03.910600 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.909948 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3394f81c-6c57-451b-8d8e-2b6504f34ed3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wgdbs\" (UID: \"3394f81c-6c57-451b-8d8e-2b6504f34ed3\") " pod="openshift-multus/multus-additional-cni-plugins-wgdbs" Jul 28 01:26:03.910600 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.909993 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-host-run-netns\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:03.910600 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.910023 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3a77a2d8-54f1-4eae-b356-213cd00c7462-env-overrides\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:03.910600 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.910090 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/03b14c3f-225a-4d46-9f03-31f3c8da2916-etc-systemd\") pod \"tuned-kmk8x\" (UID: \"03b14c3f-225a-4d46-9f03-31f3c8da2916\") " pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:03.910600 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.910116 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/03b14c3f-225a-4d46-9f03-31f3c8da2916-run\") pod \"tuned-kmk8x\" (UID: \"03b14c3f-225a-4d46-9f03-31f3c8da2916\") " pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:03.910600 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.910143 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7af7909b-e421-437d-b0a8-895ad38876c7-hostroot\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.910600 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.910168 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-log-socket\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:03.910600 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.910194 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7af7909b-e421-437d-b0a8-895ad38876c7-system-cni-dir\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.910600 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.910255 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3394f81c-6c57-451b-8d8e-2b6504f34ed3-system-cni-dir\") pod \"multus-additional-cni-plugins-wgdbs\" (UID: \"3394f81c-6c57-451b-8d8e-2b6504f34ed3\") " pod="openshift-multus/multus-additional-cni-plugins-wgdbs" Jul 28 01:26:03.910600 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.910369 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3394f81c-6c57-451b-8d8e-2b6504f34ed3-os-release\") pod \"multus-additional-cni-plugins-wgdbs\" (UID: \"3394f81c-6c57-451b-8d8e-2b6504f34ed3\") " pod="openshift-multus/multus-additional-cni-plugins-wgdbs" Jul 28 01:26:03.910600 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.910404 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3a77a2d8-54f1-4eae-b356-213cd00c7462-ovn-node-metrics-cert\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:03.910600 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.910423 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7af7909b-e421-437d-b0a8-895ad38876c7-system-cni-dir\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.910600 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.910436 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6gj9\" (UniqueName: \"kubernetes.io/projected/3a77a2d8-54f1-4eae-b356-213cd00c7462-kube-api-access-x6gj9\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:03.910600 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.910464 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/03b14c3f-225a-4d46-9f03-31f3c8da2916-sys\") pod \"tuned-kmk8x\" (UID: \"03b14c3f-225a-4d46-9f03-31f3c8da2916\") " pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:03.910600 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.910477 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3394f81c-6c57-451b-8d8e-2b6504f34ed3-system-cni-dir\") pod \"multus-additional-cni-plugins-wgdbs\" (UID: \"3394f81c-6c57-451b-8d8e-2b6504f34ed3\") " pod="openshift-multus/multus-additional-cni-plugins-wgdbs" Jul 28 01:26:03.910600 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.910496 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7af7909b-e421-437d-b0a8-895ad38876c7-cni-binary-copy\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.910600 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.910559 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3394f81c-6c57-451b-8d8e-2b6504f34ed3-cni-binary-copy\") pod \"multus-additional-cni-plugins-wgdbs\" (UID: \"3394f81c-6c57-451b-8d8e-2b6504f34ed3\") " pod="openshift-multus/multus-additional-cni-plugins-wgdbs" Jul 28 01:26:03.911539 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.910678 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7af7909b-e421-437d-b0a8-895ad38876c7-host-var-lib-kubelet\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.911539 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.910690 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7af7909b-e421-437d-b0a8-895ad38876c7-hostroot\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.911539 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.910732 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3394f81c-6c57-451b-8d8e-2b6504f34ed3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wgdbs\" (UID: \"3394f81c-6c57-451b-8d8e-2b6504f34ed3\") " pod="openshift-multus/multus-additional-cni-plugins-wgdbs" Jul 28 01:26:03.911539 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.910789 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7af7909b-e421-437d-b0a8-895ad38876c7-host-var-lib-kubelet\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.911539 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.910821 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xl8vf\" (UniqueName: \"kubernetes.io/projected/3394f81c-6c57-451b-8d8e-2b6504f34ed3-kube-api-access-xl8vf\") pod \"multus-additional-cni-plugins-wgdbs\" (UID: \"3394f81c-6c57-451b-8d8e-2b6504f34ed3\") " pod="openshift-multus/multus-additional-cni-plugins-wgdbs" Jul 28 01:26:03.911539 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.910868 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/03b14c3f-225a-4d46-9f03-31f3c8da2916-host\") pod \"tuned-kmk8x\" (UID: \"03b14c3f-225a-4d46-9f03-31f3c8da2916\") " pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:03.911539 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.910923 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3394f81c-6c57-451b-8d8e-2b6504f34ed3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wgdbs\" (UID: \"3394f81c-6c57-451b-8d8e-2b6504f34ed3\") " pod="openshift-multus/multus-additional-cni-plugins-wgdbs" Jul 28 01:26:03.911539 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.910932 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f80c0089-ca61-4ddb-850c-6c805e916aaf-registration-dir\") pod \"aws-ebs-csi-driver-node-ww5wn\" (UID: \"f80c0089-ca61-4ddb-850c-6c805e916aaf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ww5wn" Jul 28 01:26:03.911539 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.910969 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f80c0089-ca61-4ddb-850c-6c805e916aaf-sys-fs\") pod \"aws-ebs-csi-driver-node-ww5wn\" (UID: \"f80c0089-ca61-4ddb-850c-6c805e916aaf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ww5wn" Jul 28 01:26:03.911539 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.910999 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwk2d\" (UniqueName: \"kubernetes.io/projected/f80c0089-ca61-4ddb-850c-6c805e916aaf-kube-api-access-zwk2d\") pod \"aws-ebs-csi-driver-node-ww5wn\" (UID: \"f80c0089-ca61-4ddb-850c-6c805e916aaf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ww5wn" Jul 28 01:26:03.911539 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.911044 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-var-lib-openvswitch\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:03.911539 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.911010 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3394f81c-6c57-451b-8d8e-2b6504f34ed3-os-release\") pod \"multus-additional-cni-plugins-wgdbs\" (UID: \"3394f81c-6c57-451b-8d8e-2b6504f34ed3\") " pod="openshift-multus/multus-additional-cni-plugins-wgdbs" Jul 28 01:26:03.911539 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.911066 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3394f81c-6c57-451b-8d8e-2b6504f34ed3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wgdbs\" (UID: \"3394f81c-6c57-451b-8d8e-2b6504f34ed3\") " pod="openshift-multus/multus-additional-cni-plugins-wgdbs" Jul 28 01:26:03.911539 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.911117 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f80c0089-ca61-4ddb-850c-6c805e916aaf-registration-dir\") pod \"aws-ebs-csi-driver-node-ww5wn\" (UID: \"f80c0089-ca61-4ddb-850c-6c805e916aaf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ww5wn" Jul 28 01:26:03.911539 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.911140 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f80c0089-ca61-4ddb-850c-6c805e916aaf-sys-fs\") pod \"aws-ebs-csi-driver-node-ww5wn\" (UID: \"f80c0089-ca61-4ddb-850c-6c805e916aaf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ww5wn" Jul 28 01:26:03.911539 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.911196 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3a77a2d8-54f1-4eae-b356-213cd00c7462-ovnkube-script-lib\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:03.912414 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.911278 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3394f81c-6c57-451b-8d8e-2b6504f34ed3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wgdbs\" (UID: \"3394f81c-6c57-451b-8d8e-2b6504f34ed3\") " pod="openshift-multus/multus-additional-cni-plugins-wgdbs" Jul 28 01:26:03.912414 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.911349 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03b14c3f-225a-4d46-9f03-31f3c8da2916-var-lib-kubelet\") pod \"tuned-kmk8x\" (UID: \"03b14c3f-225a-4d46-9f03-31f3c8da2916\") " pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:03.912414 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.911388 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3f0ab34c-4530-44b3-9bea-e54cdbb50377-iptables-alerter-script\") pod \"iptables-alerter-qs68r\" (UID: \"3f0ab34c-4530-44b3-9bea-e54cdbb50377\") " pod="openshift-network-operator/iptables-alerter-qs68r" Jul 28 01:26:03.912414 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.911476 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fz5v\" (UniqueName: \"kubernetes.io/projected/e1808965-dc33-46a8-a355-b3ebbe849c65-kube-api-access-9fz5v\") pod \"node-ca-bgvmj\" (UID: \"e1808965-dc33-46a8-a355-b3ebbe849c65\") " pod="openshift-image-registry/node-ca-bgvmj" Jul 28 01:26:03.912414 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.911510 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-systemd-units\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:03.912414 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.911102 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7af7909b-e421-437d-b0a8-895ad38876c7-cni-binary-copy\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.912414 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.911607 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-etc-openvswitch\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:03.912414 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.911675 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/516e3ead-ffab-4201-9715-2684dcfc7aa6-hosts-file\") pod \"node-resolver-2vjkb\" (UID: \"516e3ead-ffab-4201-9715-2684dcfc7aa6\") " pod="openshift-dns/node-resolver-2vjkb" Jul 28 01:26:03.912414 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.911726 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75z85\" (UniqueName: \"kubernetes.io/projected/005aafcd-65c4-45cb-aa21-484a0c94167e-kube-api-access-75z85\") pod \"network-metrics-daemon-brftp\" (UID: \"005aafcd-65c4-45cb-aa21-484a0c94167e\") " pod="openshift-multus/network-metrics-daemon-brftp" Jul 28 01:26:03.912414 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.911742 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/516e3ead-ffab-4201-9715-2684dcfc7aa6-hosts-file\") pod \"node-resolver-2vjkb\" (UID: \"516e3ead-ffab-4201-9715-2684dcfc7aa6\") " pod="openshift-dns/node-resolver-2vjkb" Jul 28 01:26:03.912414 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.911774 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7af7909b-e421-437d-b0a8-895ad38876c7-multus-cni-dir\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.912414 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.911827 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7af7909b-e421-437d-b0a8-895ad38876c7-host-run-k8s-cni-cncf-io\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.912414 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.911977 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7af7909b-e421-437d-b0a8-895ad38876c7-host-run-multus-certs\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.912414 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.912006 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f80c0089-ca61-4ddb-850c-6c805e916aaf-etc-selinux\") pod \"aws-ebs-csi-driver-node-ww5wn\" (UID: \"f80c0089-ca61-4ddb-850c-6c805e916aaf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ww5wn" Jul 28 01:26:03.912414 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.912020 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7af7909b-e421-437d-b0a8-895ad38876c7-host-run-k8s-cni-cncf-io\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.912414 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.912074 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7af7909b-e421-437d-b0a8-895ad38876c7-multus-cni-dir\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.912414 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.912092 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-run-ovn\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:03.913223 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.912103 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7af7909b-e421-437d-b0a8-895ad38876c7-host-run-multus-certs\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.913223 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.912120 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f80c0089-ca61-4ddb-850c-6c805e916aaf-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ww5wn\" (UID: \"f80c0089-ca61-4ddb-850c-6c805e916aaf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ww5wn" Jul 28 01:26:03.913223 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.912146 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-host-cni-netd\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:03.913223 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.912196 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f80c0089-ca61-4ddb-850c-6c805e916aaf-etc-selinux\") pod \"aws-ebs-csi-driver-node-ww5wn\" (UID: \"f80c0089-ca61-4ddb-850c-6c805e916aaf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ww5wn" Jul 28 01:26:03.913223 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.912194 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f80c0089-ca61-4ddb-850c-6c805e916aaf-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ww5wn\" (UID: \"f80c0089-ca61-4ddb-850c-6c805e916aaf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ww5wn" Jul 28 01:26:03.913223 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.912271 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/03b14c3f-225a-4d46-9f03-31f3c8da2916-etc-modprobe-d\") pod \"tuned-kmk8x\" (UID: \"03b14c3f-225a-4d46-9f03-31f3c8da2916\") " pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:03.913223 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.912310 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/03b14c3f-225a-4d46-9f03-31f3c8da2916-etc-tuned\") pod \"tuned-kmk8x\" (UID: \"03b14c3f-225a-4d46-9f03-31f3c8da2916\") " pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:03.913223 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.912347 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/03b14c3f-225a-4d46-9f03-31f3c8da2916-tmp\") pod \"tuned-kmk8x\" (UID: \"03b14c3f-225a-4d46-9f03-31f3c8da2916\") " pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:03.913223 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.912381 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7af7909b-e421-437d-b0a8-895ad38876c7-host-var-lib-cni-multus\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.913223 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.912427 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7af7909b-e421-437d-b0a8-895ad38876c7-host-var-lib-cni-multus\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.913223 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.912426 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1bf2b1e4-0123-4a0b-9a30-b2ea3794732c-original-pull-secret\") pod \"global-pull-secret-syncer-7gfx9\" (UID: \"1bf2b1e4-0123-4a0b-9a30-b2ea3794732c\") " pod="kube-system/global-pull-secret-syncer-7gfx9" Jul 28 01:26:03.913223 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.912467 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbjmb\" (UniqueName: \"kubernetes.io/projected/516e3ead-ffab-4201-9715-2684dcfc7aa6-kube-api-access-qbjmb\") pod \"node-resolver-2vjkb\" (UID: \"516e3ead-ffab-4201-9715-2684dcfc7aa6\") " pod="openshift-dns/node-resolver-2vjkb" Jul 28 01:26:03.913223 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.912495 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f80c0089-ca61-4ddb-850c-6c805e916aaf-device-dir\") pod \"aws-ebs-csi-driver-node-ww5wn\" (UID: \"f80c0089-ca61-4ddb-850c-6c805e916aaf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ww5wn" Jul 28 01:26:03.913223 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:03.912525 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 01:26:03.913223 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.912539 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/751f8878-a0ed-4756-93c1-97a66b2afd10-konnectivity-ca\") pod \"konnectivity-agent-hnpn2\" (UID: \"751f8878-a0ed-4756-93c1-97a66b2afd10\") " pod="kube-system/konnectivity-agent-hnpn2" Jul 28 01:26:03.913223 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.912631 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f80c0089-ca61-4ddb-850c-6c805e916aaf-device-dir\") pod \"aws-ebs-csi-driver-node-ww5wn\" (UID: \"f80c0089-ca61-4ddb-850c-6c805e916aaf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ww5wn" Jul 28 01:26:03.913223 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.912619 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3f0ab34c-4530-44b3-9bea-e54cdbb50377-iptables-alerter-script\") pod \"iptables-alerter-qs68r\" (UID: \"3f0ab34c-4530-44b3-9bea-e54cdbb50377\") " pod="openshift-network-operator/iptables-alerter-qs68r" Jul 28 01:26:03.914021 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:03.912694 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bf2b1e4-0123-4a0b-9a30-b2ea3794732c-original-pull-secret podName:1bf2b1e4-0123-4a0b-9a30-b2ea3794732c nodeName:}" failed. No retries permitted until 2026-07-28 01:26:04.412675962 +0000 UTC m=+2.966800754 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1bf2b1e4-0123-4a0b-9a30-b2ea3794732c-original-pull-secret") pod "global-pull-secret-syncer-7gfx9" (UID: "1bf2b1e4-0123-4a0b-9a30-b2ea3794732c") : object "kube-system"/"original-pull-secret" not registered Jul 28 01:26:03.914021 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.912716 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7af7909b-e421-437d-b0a8-895ad38876c7-cnibin\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.914021 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.912785 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7af7909b-e421-437d-b0a8-895ad38876c7-host-run-netns\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.914021 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.912814 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7af7909b-e421-437d-b0a8-895ad38876c7-cnibin\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.914021 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.912913 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7af7909b-e421-437d-b0a8-895ad38876c7-host-run-netns\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.926364 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.926319 2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 28 01:26:03.930459 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:03.930428 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 01:26:03.930459 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:03.930461 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 01:26:03.930739 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:03.930477 2578 projected.go:196] Error preparing data for projected volume kube-api-access-8cfb7 for pod openshift-network-diagnostics/network-check-target-2znt8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 01:26:03.930739 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:03.930544 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5060905d-b8ac-4bbd-94cb-b1a8a8952a9a-kube-api-access-8cfb7 podName:5060905d-b8ac-4bbd-94cb-b1a8a8952a9a nodeName:}" failed. No retries permitted until 2026-07-28 01:26:04.430524204 +0000 UTC m=+2.984648998 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-8cfb7" (UniqueName: "kubernetes.io/projected/5060905d-b8ac-4bbd-94cb-b1a8a8952a9a-kube-api-access-8cfb7") pod "network-check-target-2znt8" (UID: "5060905d-b8ac-4bbd-94cb-b1a8a8952a9a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 01:26:03.930903 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.930872 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6q8zh\" (UniqueName: \"kubernetes.io/projected/7af7909b-e421-437d-b0a8-895ad38876c7-kube-api-access-6q8zh\") pod \"multus-x598j\" (UID: \"7af7909b-e421-437d-b0a8-895ad38876c7\") " pod="openshift-multus/multus-x598j" Jul 28 01:26:03.933665 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.933587 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbjmb\" (UniqueName: \"kubernetes.io/projected/516e3ead-ffab-4201-9715-2684dcfc7aa6-kube-api-access-qbjmb\") pod \"node-resolver-2vjkb\" (UID: \"516e3ead-ffab-4201-9715-2684dcfc7aa6\") " pod="openshift-dns/node-resolver-2vjkb" Jul 28 01:26:03.934269 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.934241 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xl8vf\" (UniqueName: \"kubernetes.io/projected/3394f81c-6c57-451b-8d8e-2b6504f34ed3-kube-api-access-xl8vf\") pod \"multus-additional-cni-plugins-wgdbs\" (UID: \"3394f81c-6c57-451b-8d8e-2b6504f34ed3\") " pod="openshift-multus/multus-additional-cni-plugins-wgdbs" Jul 28 01:26:03.935073 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.935045 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75z85\" (UniqueName: \"kubernetes.io/projected/005aafcd-65c4-45cb-aa21-484a0c94167e-kube-api-access-75z85\") pod \"network-metrics-daemon-brftp\" (UID: \"005aafcd-65c4-45cb-aa21-484a0c94167e\") " pod="openshift-multus/network-metrics-daemon-brftp" Jul 28 01:26:03.935482 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.935460 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4n4kz\" (UniqueName: \"kubernetes.io/projected/3f0ab34c-4530-44b3-9bea-e54cdbb50377-kube-api-access-4n4kz\") pod \"iptables-alerter-qs68r\" (UID: \"3f0ab34c-4530-44b3-9bea-e54cdbb50377\") " pod="openshift-network-operator/iptables-alerter-qs68r" Jul 28 01:26:03.935988 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:03.935960 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwk2d\" (UniqueName: \"kubernetes.io/projected/f80c0089-ca61-4ddb-850c-6c805e916aaf-kube-api-access-zwk2d\") pod \"aws-ebs-csi-driver-node-ww5wn\" (UID: \"f80c0089-ca61-4ddb-850c-6c805e916aaf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ww5wn" Jul 28 01:26:04.010581 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.010465 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-186.ec2.internal" event={"ID":"45f7ea7075c86f8ee5fac1e99899f141","Type":"ContainerStarted","Data":"7d790dac6d9c6ca3bc2fd1ef0c31fc2c9463612c0245e90944677a3f62434419"} Jul 28 01:26:04.012143 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.012112 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-186.ec2.internal" event={"ID":"9fe4fb4d5152991d383fb6f3c8471009","Type":"ContainerStarted","Data":"8a4959c32b1492301cfc0c7ad213af4865da3593aa88e4a833596dab0aa79ba8"} Jul 28 01:26:04.013356 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.013322 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-host-run-ovn-kubernetes\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:04.013535 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.013363 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/03b14c3f-225a-4d46-9f03-31f3c8da2916-etc-sysctl-conf\") pod \"tuned-kmk8x\" (UID: \"03b14c3f-225a-4d46-9f03-31f3c8da2916\") " pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:04.013535 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.013389 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9g57\" (UniqueName: \"kubernetes.io/projected/03b14c3f-225a-4d46-9f03-31f3c8da2916-kube-api-access-j9g57\") pod \"tuned-kmk8x\" (UID: \"03b14c3f-225a-4d46-9f03-31f3c8da2916\") " pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:04.013535 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.013425 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-host-slash\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:04.013535 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.013450 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3a77a2d8-54f1-4eae-b356-213cd00c7462-ovnkube-config\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:04.013535 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.013474 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/751f8878-a0ed-4756-93c1-97a66b2afd10-agent-certs\") pod \"konnectivity-agent-hnpn2\" (UID: \"751f8878-a0ed-4756-93c1-97a66b2afd10\") " pod="kube-system/konnectivity-agent-hnpn2" Jul 28 01:26:04.013535 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.013489 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-host-run-ovn-kubernetes\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:04.013535 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.013497 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e1808965-dc33-46a8-a355-b3ebbe849c65-host\") pod \"node-ca-bgvmj\" (UID: \"e1808965-dc33-46a8-a355-b3ebbe849c65\") " pod="openshift-image-registry/node-ca-bgvmj" Jul 28 01:26:04.013535 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.013533 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e1808965-dc33-46a8-a355-b3ebbe849c65-host\") pod \"node-ca-bgvmj\" (UID: \"e1808965-dc33-46a8-a355-b3ebbe849c65\") " pod="openshift-image-registry/node-ca-bgvmj" Jul 28 01:26:04.013981 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.013554 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/03b14c3f-225a-4d46-9f03-31f3c8da2916-lib-modules\") pod \"tuned-kmk8x\" (UID: \"03b14c3f-225a-4d46-9f03-31f3c8da2916\") " pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:04.013981 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.013599 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-host-slash\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:04.013981 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.013643 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-run-systemd\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:04.013981 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.013768 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/03b14c3f-225a-4d46-9f03-31f3c8da2916-etc-sysctl-conf\") pod \"tuned-kmk8x\" (UID: \"03b14c3f-225a-4d46-9f03-31f3c8da2916\") " pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:04.013981 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.013945 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/03b14c3f-225a-4d46-9f03-31f3c8da2916-lib-modules\") pod \"tuned-kmk8x\" (UID: \"03b14c3f-225a-4d46-9f03-31f3c8da2916\") " pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:04.013981 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.013600 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-run-systemd\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:04.013981 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.013982 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03b14c3f-225a-4d46-9f03-31f3c8da2916-etc-kubernetes\") pod \"tuned-kmk8x\" (UID: \"03b14c3f-225a-4d46-9f03-31f3c8da2916\") " pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:04.014406 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.014006 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-run-openvswitch\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:04.014406 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.014028 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e1808965-dc33-46a8-a355-b3ebbe849c65-serviceca\") pod \"node-ca-bgvmj\" (UID: \"e1808965-dc33-46a8-a355-b3ebbe849c65\") " pod="openshift-image-registry/node-ca-bgvmj" Jul 28 01:26:04.014406 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.014051 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/03b14c3f-225a-4d46-9f03-31f3c8da2916-etc-sysctl-d\") pod \"tuned-kmk8x\" (UID: \"03b14c3f-225a-4d46-9f03-31f3c8da2916\") " pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:04.014406 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.014092 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-host-kubelet\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:04.014406 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.014116 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-host-cni-bin\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:04.014406 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.014166 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:04.014406 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.014191 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/03b14c3f-225a-4d46-9f03-31f3c8da2916-etc-sysconfig\") pod \"tuned-kmk8x\" (UID: \"03b14c3f-225a-4d46-9f03-31f3c8da2916\") " pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:04.014406 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.014218 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-node-log\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:04.014406 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.014241 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-host-run-netns\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:04.014406 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.014262 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3a77a2d8-54f1-4eae-b356-213cd00c7462-ovnkube-config\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:04.014406 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.014267 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3a77a2d8-54f1-4eae-b356-213cd00c7462-env-overrides\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:04.014406 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.014333 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/03b14c3f-225a-4d46-9f03-31f3c8da2916-etc-systemd\") pod \"tuned-kmk8x\" (UID: \"03b14c3f-225a-4d46-9f03-31f3c8da2916\") " pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:04.014406 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.014370 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/03b14c3f-225a-4d46-9f03-31f3c8da2916-run\") pod \"tuned-kmk8x\" (UID: \"03b14c3f-225a-4d46-9f03-31f3c8da2916\") " pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:04.014406 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.014398 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-log-socket\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:04.015267 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.014430 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3a77a2d8-54f1-4eae-b356-213cd00c7462-ovn-node-metrics-cert\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:04.015267 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.014475 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6gj9\" (UniqueName: \"kubernetes.io/projected/3a77a2d8-54f1-4eae-b356-213cd00c7462-kube-api-access-x6gj9\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:04.015267 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.014498 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/03b14c3f-225a-4d46-9f03-31f3c8da2916-sys\") pod \"tuned-kmk8x\" (UID: \"03b14c3f-225a-4d46-9f03-31f3c8da2916\") " pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:04.015267 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.014525 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/03b14c3f-225a-4d46-9f03-31f3c8da2916-host\") pod \"tuned-kmk8x\" (UID: \"03b14c3f-225a-4d46-9f03-31f3c8da2916\") " pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:04.015267 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.014552 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-var-lib-openvswitch\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:04.015267 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.014578 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3a77a2d8-54f1-4eae-b356-213cd00c7462-ovnkube-script-lib\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:04.015267 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.014603 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03b14c3f-225a-4d46-9f03-31f3c8da2916-var-lib-kubelet\") pod \"tuned-kmk8x\" (UID: \"03b14c3f-225a-4d46-9f03-31f3c8da2916\") " pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:04.015267 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.014644 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fz5v\" (UniqueName: \"kubernetes.io/projected/e1808965-dc33-46a8-a355-b3ebbe849c65-kube-api-access-9fz5v\") pod \"node-ca-bgvmj\" (UID: \"e1808965-dc33-46a8-a355-b3ebbe849c65\") " pod="openshift-image-registry/node-ca-bgvmj" Jul 28 01:26:04.015267 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.014672 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-systemd-units\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:04.015267 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.014699 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-etc-openvswitch\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:04.015267 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.014704 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3a77a2d8-54f1-4eae-b356-213cd00c7462-env-overrides\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:04.015267 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.014734 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-run-ovn\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:04.015267 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.014765 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-host-cni-netd\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:04.015267 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.014774 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03b14c3f-225a-4d46-9f03-31f3c8da2916-etc-kubernetes\") pod \"tuned-kmk8x\" (UID: \"03b14c3f-225a-4d46-9f03-31f3c8da2916\") " pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:04.015267 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.014803 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/03b14c3f-225a-4d46-9f03-31f3c8da2916-etc-modprobe-d\") pod \"tuned-kmk8x\" (UID: \"03b14c3f-225a-4d46-9f03-31f3c8da2916\") " pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:04.015267 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.014854 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/03b14c3f-225a-4d46-9f03-31f3c8da2916-etc-tuned\") pod \"tuned-kmk8x\" (UID: \"03b14c3f-225a-4d46-9f03-31f3c8da2916\") " pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:04.015267 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.014882 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/03b14c3f-225a-4d46-9f03-31f3c8da2916-tmp\") pod \"tuned-kmk8x\" (UID: \"03b14c3f-225a-4d46-9f03-31f3c8da2916\") " pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:04.016170 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.014942 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/751f8878-a0ed-4756-93c1-97a66b2afd10-konnectivity-ca\") pod \"konnectivity-agent-hnpn2\" (UID: \"751f8878-a0ed-4756-93c1-97a66b2afd10\") " pod="kube-system/konnectivity-agent-hnpn2" Jul 28 01:26:04.016170 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.015180 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:04.016170 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.015189 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e1808965-dc33-46a8-a355-b3ebbe849c65-serviceca\") pod \"node-ca-bgvmj\" (UID: \"e1808965-dc33-46a8-a355-b3ebbe849c65\") " pod="openshift-image-registry/node-ca-bgvmj" Jul 28 01:26:04.016170 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.015246 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/03b14c3f-225a-4d46-9f03-31f3c8da2916-etc-sysconfig\") pod \"tuned-kmk8x\" (UID: \"03b14c3f-225a-4d46-9f03-31f3c8da2916\") " pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:04.016170 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.015301 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-node-log\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:04.016170 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.015332 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/03b14c3f-225a-4d46-9f03-31f3c8da2916-etc-sysctl-d\") pod \"tuned-kmk8x\" (UID: \"03b14c3f-225a-4d46-9f03-31f3c8da2916\") " pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:04.016170 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.015344 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-host-run-netns\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:04.016170 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.015388 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-host-cni-bin\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:04.016170 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.015389 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-host-kubelet\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:04.016170 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.015438 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03b14c3f-225a-4d46-9f03-31f3c8da2916-var-lib-kubelet\") pod \"tuned-kmk8x\" (UID: \"03b14c3f-225a-4d46-9f03-31f3c8da2916\") " pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:04.016170 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.015446 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-host-cni-netd\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:04.016170 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.015490 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/03b14c3f-225a-4d46-9f03-31f3c8da2916-run\") pod \"tuned-kmk8x\" (UID: \"03b14c3f-225a-4d46-9f03-31f3c8da2916\") " pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:04.016170 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.015504 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-systemd-units\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:04.016170 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.015556 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-run-ovn\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:04.016170 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.015623 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/03b14c3f-225a-4d46-9f03-31f3c8da2916-host\") pod \"tuned-kmk8x\" (UID: \"03b14c3f-225a-4d46-9f03-31f3c8da2916\") " pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:04.016170 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.015661 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-log-socket\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:04.016170 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.015715 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-var-lib-openvswitch\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:04.016170 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.015815 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-run-openvswitch\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:04.017124 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.015818 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3a77a2d8-54f1-4eae-b356-213cd00c7462-etc-openvswitch\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:04.017124 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.015865 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/03b14c3f-225a-4d46-9f03-31f3c8da2916-etc-systemd\") pod \"tuned-kmk8x\" (UID: \"03b14c3f-225a-4d46-9f03-31f3c8da2916\") " pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:04.017124 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.015900 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/03b14c3f-225a-4d46-9f03-31f3c8da2916-sys\") pod \"tuned-kmk8x\" (UID: \"03b14c3f-225a-4d46-9f03-31f3c8da2916\") " pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:04.017124 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.016109 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3a77a2d8-54f1-4eae-b356-213cd00c7462-ovnkube-script-lib\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:04.017124 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.016222 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/03b14c3f-225a-4d46-9f03-31f3c8da2916-etc-modprobe-d\") pod \"tuned-kmk8x\" (UID: \"03b14c3f-225a-4d46-9f03-31f3c8da2916\") " pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:04.017124 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.016492 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/751f8878-a0ed-4756-93c1-97a66b2afd10-konnectivity-ca\") pod \"konnectivity-agent-hnpn2\" (UID: \"751f8878-a0ed-4756-93c1-97a66b2afd10\") " pod="kube-system/konnectivity-agent-hnpn2" Jul 28 01:26:04.017124 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.016915 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/751f8878-a0ed-4756-93c1-97a66b2afd10-agent-certs\") pod \"konnectivity-agent-hnpn2\" (UID: \"751f8878-a0ed-4756-93c1-97a66b2afd10\") " pod="kube-system/konnectivity-agent-hnpn2" Jul 28 01:26:04.019016 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.018992 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/03b14c3f-225a-4d46-9f03-31f3c8da2916-etc-tuned\") pod \"tuned-kmk8x\" (UID: \"03b14c3f-225a-4d46-9f03-31f3c8da2916\") " pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:04.019192 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.019166 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/03b14c3f-225a-4d46-9f03-31f3c8da2916-tmp\") pod \"tuned-kmk8x\" (UID: \"03b14c3f-225a-4d46-9f03-31f3c8da2916\") " pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:04.021387 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.021356 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3a77a2d8-54f1-4eae-b356-213cd00c7462-ovn-node-metrics-cert\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:04.026927 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.026879 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6gj9\" (UniqueName: \"kubernetes.io/projected/3a77a2d8-54f1-4eae-b356-213cd00c7462-kube-api-access-x6gj9\") pod \"ovnkube-node-c6m66\" (UID: \"3a77a2d8-54f1-4eae-b356-213cd00c7462\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:04.028567 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.028541 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fz5v\" (UniqueName: \"kubernetes.io/projected/e1808965-dc33-46a8-a355-b3ebbe849c65-kube-api-access-9fz5v\") pod \"node-ca-bgvmj\" (UID: \"e1808965-dc33-46a8-a355-b3ebbe849c65\") " pod="openshift-image-registry/node-ca-bgvmj" Jul 28 01:26:04.029485 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.029458 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9g57\" (UniqueName: \"kubernetes.io/projected/03b14c3f-225a-4d46-9f03-31f3c8da2916-kube-api-access-j9g57\") pod \"tuned-kmk8x\" (UID: \"03b14c3f-225a-4d46-9f03-31f3c8da2916\") " pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:04.099427 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.099387 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qs68r" Jul 28 01:26:04.106602 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.106578 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wgdbs" Jul 28 01:26:04.110139 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:04.110106 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f0ab34c_4530_44b3_9bea_e54cdbb50377.slice/crio-e3c96ef2127340db4e6b870ae701d314f75a6b95314cef870747c15362f3cc7f WatchSource:0}: Error finding container e3c96ef2127340db4e6b870ae701d314f75a6b95314cef870747c15362f3cc7f: Status 404 returned error can't find the container with id e3c96ef2127340db4e6b870ae701d314f75a6b95314cef870747c15362f3cc7f Jul 28 01:26:04.113658 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.113618 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2vjkb" Jul 28 01:26:04.118730 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.118703 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-x598j" Jul 28 01:26:04.119028 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:04.119000 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3394f81c_6c57_451b_8d8e_2b6504f34ed3.slice/crio-d0f4f57a57d04266a21b1be3020caacd76e385e38141510496c2241c20cd2027 WatchSource:0}: Error finding container d0f4f57a57d04266a21b1be3020caacd76e385e38141510496c2241c20cd2027: Status 404 returned error can't find the container with id d0f4f57a57d04266a21b1be3020caacd76e385e38141510496c2241c20cd2027 Jul 28 01:26:04.124227 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.124025 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ww5wn" Jul 28 01:26:04.130848 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.130828 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hnpn2" Jul 28 01:26:04.133131 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:04.133104 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod516e3ead_ffab_4201_9715_2684dcfc7aa6.slice/crio-ffb4aa851b6a8742e4d823dc1e10ea72fd50c1c66d8ad023e7488388a14c2472 WatchSource:0}: Error finding container ffb4aa851b6a8742e4d823dc1e10ea72fd50c1c66d8ad023e7488388a14c2472: Status 404 returned error can't find the container with id ffb4aa851b6a8742e4d823dc1e10ea72fd50c1c66d8ad023e7488388a14c2472 Jul 28 01:26:04.137822 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.137802 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" Jul 28 01:26:04.139195 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:04.139165 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7af7909b_e421_437d_b0a8_895ad38876c7.slice/crio-bd7bbc7d287c148668e50460dfc7c56a7f6aab87b91e40a735dd462385c8e5df WatchSource:0}: Error finding container bd7bbc7d287c148668e50460dfc7c56a7f6aab87b91e40a735dd462385c8e5df: Status 404 returned error can't find the container with id bd7bbc7d287c148668e50460dfc7c56a7f6aab87b91e40a735dd462385c8e5df Jul 28 01:26:04.144061 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.144041 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bgvmj" Jul 28 01:26:04.145725 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:04.145696 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf80c0089_ca61_4ddb_850c_6c805e916aaf.slice/crio-99bed1eaba611d5d5815094ac55564afd6bf0c5f7fa9c2c0c3358cc7c2c5eebc WatchSource:0}: Error finding container 99bed1eaba611d5d5815094ac55564afd6bf0c5f7fa9c2c0c3358cc7c2c5eebc: Status 404 returned error can't find the container with id 99bed1eaba611d5d5815094ac55564afd6bf0c5f7fa9c2c0c3358cc7c2c5eebc Jul 28 01:26:04.150284 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:04.150257 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod751f8878_a0ed_4756_93c1_97a66b2afd10.slice/crio-ec2aa87eb8fdbebba790033fa4d39149a923325cc7ff1ebbd0f524f9357d254d WatchSource:0}: Error finding container ec2aa87eb8fdbebba790033fa4d39149a923325cc7ff1ebbd0f524f9357d254d: Status 404 returned error can't find the container with id ec2aa87eb8fdbebba790033fa4d39149a923325cc7ff1ebbd0f524f9357d254d Jul 28 01:26:04.151951 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.151931 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:04.161593 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:04.161568 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1808965_dc33_46a8_a355_b3ebbe849c65.slice/crio-7d2b9848c030b1c4b734d48e38b65b4b70f7f00e269c8c3ec9d449facd640e65 WatchSource:0}: Error finding container 7d2b9848c030b1c4b734d48e38b65b4b70f7f00e269c8c3ec9d449facd640e65: Status 404 returned error can't find the container with id 7d2b9848c030b1c4b734d48e38b65b4b70f7f00e269c8c3ec9d449facd640e65 Jul 28 01:26:04.418428 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.418333 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1bf2b1e4-0123-4a0b-9a30-b2ea3794732c-original-pull-secret\") pod \"global-pull-secret-syncer-7gfx9\" (UID: \"1bf2b1e4-0123-4a0b-9a30-b2ea3794732c\") " pod="kube-system/global-pull-secret-syncer-7gfx9" Jul 28 01:26:04.418428 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.418396 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/005aafcd-65c4-45cb-aa21-484a0c94167e-metrics-certs\") pod \"network-metrics-daemon-brftp\" (UID: \"005aafcd-65c4-45cb-aa21-484a0c94167e\") " pod="openshift-multus/network-metrics-daemon-brftp" Jul 28 01:26:04.418654 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:04.418515 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 01:26:04.418654 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:04.418561 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 01:26:04.418654 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:04.418607 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bf2b1e4-0123-4a0b-9a30-b2ea3794732c-original-pull-secret podName:1bf2b1e4-0123-4a0b-9a30-b2ea3794732c nodeName:}" failed. No retries permitted until 2026-07-28 01:26:05.418585195 +0000 UTC m=+3.972709976 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1bf2b1e4-0123-4a0b-9a30-b2ea3794732c-original-pull-secret") pod "global-pull-secret-syncer-7gfx9" (UID: "1bf2b1e4-0123-4a0b-9a30-b2ea3794732c") : object "kube-system"/"original-pull-secret" not registered Jul 28 01:26:04.418814 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:04.418676 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/005aafcd-65c4-45cb-aa21-484a0c94167e-metrics-certs podName:005aafcd-65c4-45cb-aa21-484a0c94167e nodeName:}" failed. No retries permitted until 2026-07-28 01:26:05.418651774 +0000 UTC m=+3.972776568 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/005aafcd-65c4-45cb-aa21-484a0c94167e-metrics-certs") pod "network-metrics-daemon-brftp" (UID: "005aafcd-65c4-45cb-aa21-484a0c94167e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 01:26:04.519469 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.519429 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cfb7\" (UniqueName: \"kubernetes.io/projected/5060905d-b8ac-4bbd-94cb-b1a8a8952a9a-kube-api-access-8cfb7\") pod \"network-check-target-2znt8\" (UID: \"5060905d-b8ac-4bbd-94cb-b1a8a8952a9a\") " pod="openshift-network-diagnostics/network-check-target-2znt8" Jul 28 01:26:04.519713 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:04.519627 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 01:26:04.519713 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:04.519655 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 01:26:04.519713 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:04.519671 2578 projected.go:196] Error preparing data for projected volume kube-api-access-8cfb7 for pod openshift-network-diagnostics/network-check-target-2znt8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 01:26:04.519875 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:04.519735 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5060905d-b8ac-4bbd-94cb-b1a8a8952a9a-kube-api-access-8cfb7 podName:5060905d-b8ac-4bbd-94cb-b1a8a8952a9a nodeName:}" failed. No retries permitted until 2026-07-28 01:26:05.519719816 +0000 UTC m=+4.073844599 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-8cfb7" (UniqueName: "kubernetes.io/projected/5060905d-b8ac-4bbd-94cb-b1a8a8952a9a-kube-api-access-8cfb7") pod "network-check-target-2znt8" (UID: "5060905d-b8ac-4bbd-94cb-b1a8a8952a9a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 01:26:04.850740 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.850528 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 01:21:04 +0000 UTC" deadline="2028-02-21 01:03:45.572065472 +0000 UTC" Jul 28 01:26:04.850740 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:04.850608 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13751h37m40.721461881s" Jul 28 01:26:05.015738 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:05.015679 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" event={"ID":"3a77a2d8-54f1-4eae-b356-213cd00c7462","Type":"ContainerStarted","Data":"c65aa8b28a3d5c13f224aa84169281fd11caaefed0289491093a30e701ad6cdf"} Jul 28 01:26:05.017528 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:05.017493 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" event={"ID":"03b14c3f-225a-4d46-9f03-31f3c8da2916","Type":"ContainerStarted","Data":"600b310f777be1ff0a23f14229cc12689fd879f8804488a107f719e8c6b315a1"} Jul 28 01:26:05.019386 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:05.019355 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hnpn2" event={"ID":"751f8878-a0ed-4756-93c1-97a66b2afd10","Type":"ContainerStarted","Data":"ec2aa87eb8fdbebba790033fa4d39149a923325cc7ff1ebbd0f524f9357d254d"} Jul 28 01:26:05.020772 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:05.020750 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ww5wn" event={"ID":"f80c0089-ca61-4ddb-850c-6c805e916aaf","Type":"ContainerStarted","Data":"99bed1eaba611d5d5815094ac55564afd6bf0c5f7fa9c2c0c3358cc7c2c5eebc"} Jul 28 01:26:05.023741 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:05.022960 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-x598j" event={"ID":"7af7909b-e421-437d-b0a8-895ad38876c7","Type":"ContainerStarted","Data":"bd7bbc7d287c148668e50460dfc7c56a7f6aab87b91e40a735dd462385c8e5df"} Jul 28 01:26:05.024403 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:05.024346 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wgdbs" event={"ID":"3394f81c-6c57-451b-8d8e-2b6504f34ed3","Type":"ContainerStarted","Data":"d0f4f57a57d04266a21b1be3020caacd76e385e38141510496c2241c20cd2027"} Jul 28 01:26:05.025960 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:05.025936 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qs68r" event={"ID":"3f0ab34c-4530-44b3-9bea-e54cdbb50377","Type":"ContainerStarted","Data":"e3c96ef2127340db4e6b870ae701d314f75a6b95314cef870747c15362f3cc7f"} Jul 28 01:26:05.027564 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:05.027526 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bgvmj" event={"ID":"e1808965-dc33-46a8-a355-b3ebbe849c65","Type":"ContainerStarted","Data":"7d2b9848c030b1c4b734d48e38b65b4b70f7f00e269c8c3ec9d449facd640e65"} Jul 28 01:26:05.029455 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:05.029431 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2vjkb" event={"ID":"516e3ead-ffab-4201-9715-2684dcfc7aa6","Type":"ContainerStarted","Data":"ffb4aa851b6a8742e4d823dc1e10ea72fd50c1c66d8ad023e7488388a14c2472"} Jul 28 01:26:05.427337 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:05.426270 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1bf2b1e4-0123-4a0b-9a30-b2ea3794732c-original-pull-secret\") pod \"global-pull-secret-syncer-7gfx9\" (UID: \"1bf2b1e4-0123-4a0b-9a30-b2ea3794732c\") " pod="kube-system/global-pull-secret-syncer-7gfx9" Jul 28 01:26:05.427337 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:05.426328 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/005aafcd-65c4-45cb-aa21-484a0c94167e-metrics-certs\") pod \"network-metrics-daemon-brftp\" (UID: \"005aafcd-65c4-45cb-aa21-484a0c94167e\") " pod="openshift-multus/network-metrics-daemon-brftp" Jul 28 01:26:05.427337 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:05.426462 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 01:26:05.427337 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:05.426530 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/005aafcd-65c4-45cb-aa21-484a0c94167e-metrics-certs podName:005aafcd-65c4-45cb-aa21-484a0c94167e nodeName:}" failed. No retries permitted until 2026-07-28 01:26:07.426509488 +0000 UTC m=+5.980634270 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/005aafcd-65c4-45cb-aa21-484a0c94167e-metrics-certs") pod "network-metrics-daemon-brftp" (UID: "005aafcd-65c4-45cb-aa21-484a0c94167e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 01:26:05.427337 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:05.426619 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 01:26:05.427337 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:05.426657 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bf2b1e4-0123-4a0b-9a30-b2ea3794732c-original-pull-secret podName:1bf2b1e4-0123-4a0b-9a30-b2ea3794732c nodeName:}" failed. No retries permitted until 2026-07-28 01:26:07.426644706 +0000 UTC m=+5.980769486 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1bf2b1e4-0123-4a0b-9a30-b2ea3794732c-original-pull-secret") pod "global-pull-secret-syncer-7gfx9" (UID: "1bf2b1e4-0123-4a0b-9a30-b2ea3794732c") : object "kube-system"/"original-pull-secret" not registered Jul 28 01:26:05.528497 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:05.527273 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cfb7\" (UniqueName: \"kubernetes.io/projected/5060905d-b8ac-4bbd-94cb-b1a8a8952a9a-kube-api-access-8cfb7\") pod \"network-check-target-2znt8\" (UID: \"5060905d-b8ac-4bbd-94cb-b1a8a8952a9a\") " pod="openshift-network-diagnostics/network-check-target-2znt8" Jul 28 01:26:05.528497 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:05.527484 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 01:26:05.528497 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:05.527513 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 01:26:05.528497 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:05.527528 2578 projected.go:196] Error preparing data for projected volume kube-api-access-8cfb7 for pod openshift-network-diagnostics/network-check-target-2znt8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 01:26:05.528497 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:05.527622 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5060905d-b8ac-4bbd-94cb-b1a8a8952a9a-kube-api-access-8cfb7 podName:5060905d-b8ac-4bbd-94cb-b1a8a8952a9a nodeName:}" failed. No retries permitted until 2026-07-28 01:26:07.527599588 +0000 UTC m=+6.081724373 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-8cfb7" (UniqueName: "kubernetes.io/projected/5060905d-b8ac-4bbd-94cb-b1a8a8952a9a-kube-api-access-8cfb7") pod "network-check-target-2znt8" (UID: "5060905d-b8ac-4bbd-94cb-b1a8a8952a9a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 01:26:05.851347 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:05.851254 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 01:21:04 +0000 UTC" deadline="2028-02-29 07:48:17.73437737 +0000 UTC" Jul 28 01:26:05.851347 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:05.851301 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13950h22m11.883082358s" Jul 28 01:26:06.007739 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:06.007706 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brftp" Jul 28 01:26:06.007913 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:06.007863 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-brftp" podUID="005aafcd-65c4-45cb-aa21-484a0c94167e" Jul 28 01:26:06.008359 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:06.008336 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gfx9" Jul 28 01:26:06.008478 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:06.008451 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gfx9" podUID="1bf2b1e4-0123-4a0b-9a30-b2ea3794732c" Jul 28 01:26:06.008540 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:06.008521 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2znt8" Jul 28 01:26:06.008636 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:06.008601 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2znt8" podUID="5060905d-b8ac-4bbd-94cb-b1a8a8952a9a" Jul 28 01:26:07.053367 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:07.053103 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-186.ec2.internal" event={"ID":"9fe4fb4d5152991d383fb6f3c8471009","Type":"ContainerStarted","Data":"adb9f2963c2b201a53d5b4a71ae5598f4f0a2e4697c7dec591da232798772a16"} Jul 28 01:26:07.060694 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:07.060501 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-186.ec2.internal" event={"ID":"45f7ea7075c86f8ee5fac1e99899f141","Type":"ContainerStarted","Data":"85498a0e3804b5a24cc101601146e4a3bbfcdf747f1e5473f19e6d5fde0c5d3d"} Jul 28 01:26:07.069751 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:07.069699 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-186.ec2.internal" podStartSLOduration=4.069679077 podStartE2EDuration="4.069679077s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 01:26:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:26:07.068639392 +0000 UTC m=+5.622764192" watchObservedRunningTime="2026-07-28 01:26:07.069679077 +0000 UTC m=+5.623803876" Jul 28 01:26:07.443923 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:07.443880 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1bf2b1e4-0123-4a0b-9a30-b2ea3794732c-original-pull-secret\") pod \"global-pull-secret-syncer-7gfx9\" (UID: \"1bf2b1e4-0123-4a0b-9a30-b2ea3794732c\") " pod="kube-system/global-pull-secret-syncer-7gfx9" Jul 28 01:26:07.444120 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:07.443944 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/005aafcd-65c4-45cb-aa21-484a0c94167e-metrics-certs\") pod \"network-metrics-daemon-brftp\" (UID: \"005aafcd-65c4-45cb-aa21-484a0c94167e\") " pod="openshift-multus/network-metrics-daemon-brftp" Jul 28 01:26:07.444120 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:07.444083 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 01:26:07.444232 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:07.444156 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/005aafcd-65c4-45cb-aa21-484a0c94167e-metrics-certs podName:005aafcd-65c4-45cb-aa21-484a0c94167e nodeName:}" failed. No retries permitted until 2026-07-28 01:26:11.444126755 +0000 UTC m=+9.998251545 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/005aafcd-65c4-45cb-aa21-484a0c94167e-metrics-certs") pod "network-metrics-daemon-brftp" (UID: "005aafcd-65c4-45cb-aa21-484a0c94167e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 01:26:07.444569 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:07.444536 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 01:26:07.444653 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:07.444604 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bf2b1e4-0123-4a0b-9a30-b2ea3794732c-original-pull-secret podName:1bf2b1e4-0123-4a0b-9a30-b2ea3794732c nodeName:}" failed. No retries permitted until 2026-07-28 01:26:11.444588247 +0000 UTC m=+9.998713043 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1bf2b1e4-0123-4a0b-9a30-b2ea3794732c-original-pull-secret") pod "global-pull-secret-syncer-7gfx9" (UID: "1bf2b1e4-0123-4a0b-9a30-b2ea3794732c") : object "kube-system"/"original-pull-secret" not registered Jul 28 01:26:07.545088 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:07.545049 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cfb7\" (UniqueName: \"kubernetes.io/projected/5060905d-b8ac-4bbd-94cb-b1a8a8952a9a-kube-api-access-8cfb7\") pod \"network-check-target-2znt8\" (UID: \"5060905d-b8ac-4bbd-94cb-b1a8a8952a9a\") " pod="openshift-network-diagnostics/network-check-target-2znt8" Jul 28 01:26:07.545275 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:07.545207 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 01:26:07.545275 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:07.545230 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 01:26:07.545275 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:07.545245 2578 projected.go:196] Error preparing data for projected volume kube-api-access-8cfb7 for pod openshift-network-diagnostics/network-check-target-2znt8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 01:26:07.545422 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:07.545307 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5060905d-b8ac-4bbd-94cb-b1a8a8952a9a-kube-api-access-8cfb7 podName:5060905d-b8ac-4bbd-94cb-b1a8a8952a9a nodeName:}" failed. No retries permitted until 2026-07-28 01:26:11.545289219 +0000 UTC m=+10.099414012 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-8cfb7" (UniqueName: "kubernetes.io/projected/5060905d-b8ac-4bbd-94cb-b1a8a8952a9a-kube-api-access-8cfb7") pod "network-check-target-2znt8" (UID: "5060905d-b8ac-4bbd-94cb-b1a8a8952a9a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 01:26:08.005698 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:08.005661 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brftp" Jul 28 01:26:08.005858 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:08.005800 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-brftp" podUID="005aafcd-65c4-45cb-aa21-484a0c94167e" Jul 28 01:26:08.005858 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:08.005669 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gfx9" Jul 28 01:26:08.005988 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:08.005903 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gfx9" podUID="1bf2b1e4-0123-4a0b-9a30-b2ea3794732c" Jul 28 01:26:08.005988 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:08.005928 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2znt8" Jul 28 01:26:08.006088 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:08.005993 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2znt8" podUID="5060905d-b8ac-4bbd-94cb-b1a8a8952a9a" Jul 28 01:26:10.004730 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:10.004638 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2znt8" Jul 28 01:26:10.004730 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:10.004682 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gfx9" Jul 28 01:26:10.005209 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:10.004644 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brftp" Jul 28 01:26:10.005209 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:10.004778 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2znt8" podUID="5060905d-b8ac-4bbd-94cb-b1a8a8952a9a" Jul 28 01:26:10.005209 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:10.004882 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-brftp" podUID="005aafcd-65c4-45cb-aa21-484a0c94167e" Jul 28 01:26:10.005209 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:10.004947 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gfx9" podUID="1bf2b1e4-0123-4a0b-9a30-b2ea3794732c" Jul 28 01:26:10.071354 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:10.071317 2578 generic.go:356] "Generic (PLEG): container finished" podID="45f7ea7075c86f8ee5fac1e99899f141" containerID="85498a0e3804b5a24cc101601146e4a3bbfcdf747f1e5473f19e6d5fde0c5d3d" exitCode=0 Jul 28 01:26:10.071535 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:10.071377 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-186.ec2.internal" event={"ID":"45f7ea7075c86f8ee5fac1e99899f141","Type":"ContainerDied","Data":"85498a0e3804b5a24cc101601146e4a3bbfcdf747f1e5473f19e6d5fde0c5d3d"} Jul 28 01:26:11.479246 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:11.479208 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1bf2b1e4-0123-4a0b-9a30-b2ea3794732c-original-pull-secret\") pod \"global-pull-secret-syncer-7gfx9\" (UID: \"1bf2b1e4-0123-4a0b-9a30-b2ea3794732c\") " pod="kube-system/global-pull-secret-syncer-7gfx9" Jul 28 01:26:11.479708 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:11.479269 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/005aafcd-65c4-45cb-aa21-484a0c94167e-metrics-certs\") pod \"network-metrics-daemon-brftp\" (UID: \"005aafcd-65c4-45cb-aa21-484a0c94167e\") " pod="openshift-multus/network-metrics-daemon-brftp" Jul 28 01:26:11.479708 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:11.479422 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 01:26:11.479708 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:11.479471 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 01:26:11.479708 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:11.479490 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/005aafcd-65c4-45cb-aa21-484a0c94167e-metrics-certs podName:005aafcd-65c4-45cb-aa21-484a0c94167e nodeName:}" failed. No retries permitted until 2026-07-28 01:26:19.479471184 +0000 UTC m=+18.033595974 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/005aafcd-65c4-45cb-aa21-484a0c94167e-metrics-certs") pod "network-metrics-daemon-brftp" (UID: "005aafcd-65c4-45cb-aa21-484a0c94167e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 01:26:11.479708 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:11.479535 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bf2b1e4-0123-4a0b-9a30-b2ea3794732c-original-pull-secret podName:1bf2b1e4-0123-4a0b-9a30-b2ea3794732c nodeName:}" failed. No retries permitted until 2026-07-28 01:26:19.479516945 +0000 UTC m=+18.033641740 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1bf2b1e4-0123-4a0b-9a30-b2ea3794732c-original-pull-secret") pod "global-pull-secret-syncer-7gfx9" (UID: "1bf2b1e4-0123-4a0b-9a30-b2ea3794732c") : object "kube-system"/"original-pull-secret" not registered Jul 28 01:26:11.580224 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:11.580187 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cfb7\" (UniqueName: \"kubernetes.io/projected/5060905d-b8ac-4bbd-94cb-b1a8a8952a9a-kube-api-access-8cfb7\") pod \"network-check-target-2znt8\" (UID: \"5060905d-b8ac-4bbd-94cb-b1a8a8952a9a\") " pod="openshift-network-diagnostics/network-check-target-2znt8" Jul 28 01:26:11.580401 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:11.580386 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 01:26:11.580463 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:11.580411 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 01:26:11.580463 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:11.580424 2578 projected.go:196] Error preparing data for projected volume kube-api-access-8cfb7 for pod openshift-network-diagnostics/network-check-target-2znt8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 01:26:11.580584 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:11.580490 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5060905d-b8ac-4bbd-94cb-b1a8a8952a9a-kube-api-access-8cfb7 podName:5060905d-b8ac-4bbd-94cb-b1a8a8952a9a nodeName:}" failed. No retries permitted until 2026-07-28 01:26:19.580469952 +0000 UTC m=+18.134594749 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-8cfb7" (UniqueName: "kubernetes.io/projected/5060905d-b8ac-4bbd-94cb-b1a8a8952a9a-kube-api-access-8cfb7") pod "network-check-target-2znt8" (UID: "5060905d-b8ac-4bbd-94cb-b1a8a8952a9a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 01:26:12.006304 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:12.006268 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gfx9" Jul 28 01:26:12.006465 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:12.006394 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gfx9" podUID="1bf2b1e4-0123-4a0b-9a30-b2ea3794732c" Jul 28 01:26:12.006604 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:12.006584 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2znt8" Jul 28 01:26:12.006702 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:12.006680 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2znt8" podUID="5060905d-b8ac-4bbd-94cb-b1a8a8952a9a" Jul 28 01:26:12.006767 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:12.006743 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brftp" Jul 28 01:26:12.006843 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:12.006825 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-brftp" podUID="005aafcd-65c4-45cb-aa21-484a0c94167e" Jul 28 01:26:14.005767 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:14.005702 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brftp" Jul 28 01:26:14.005767 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:14.005737 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2znt8" Jul 28 01:26:14.006382 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:14.005702 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gfx9" Jul 28 01:26:14.006382 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:14.005873 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-brftp" podUID="005aafcd-65c4-45cb-aa21-484a0c94167e" Jul 28 01:26:14.006382 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:14.006305 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2znt8" podUID="5060905d-b8ac-4bbd-94cb-b1a8a8952a9a" Jul 28 01:26:14.006538 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:14.006395 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gfx9" podUID="1bf2b1e4-0123-4a0b-9a30-b2ea3794732c" Jul 28 01:26:16.004368 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:16.004335 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brftp" Jul 28 01:26:16.004368 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:16.004374 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gfx9" Jul 28 01:26:16.004904 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:16.004335 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2znt8" Jul 28 01:26:16.004904 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:16.004478 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-brftp" podUID="005aafcd-65c4-45cb-aa21-484a0c94167e" Jul 28 01:26:16.004904 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:16.004586 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2znt8" podUID="5060905d-b8ac-4bbd-94cb-b1a8a8952a9a" Jul 28 01:26:16.004904 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:16.004655 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gfx9" podUID="1bf2b1e4-0123-4a0b-9a30-b2ea3794732c" Jul 28 01:26:18.004686 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:18.004648 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gfx9" Jul 28 01:26:18.005162 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:18.004647 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2znt8" Jul 28 01:26:18.005162 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:18.004786 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gfx9" podUID="1bf2b1e4-0123-4a0b-9a30-b2ea3794732c" Jul 28 01:26:18.005162 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:18.004856 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2znt8" podUID="5060905d-b8ac-4bbd-94cb-b1a8a8952a9a" Jul 28 01:26:18.005162 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:18.004655 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brftp" Jul 28 01:26:18.005162 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:18.004962 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-brftp" podUID="005aafcd-65c4-45cb-aa21-484a0c94167e" Jul 28 01:26:19.541006 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:19.540972 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1bf2b1e4-0123-4a0b-9a30-b2ea3794732c-original-pull-secret\") pod \"global-pull-secret-syncer-7gfx9\" (UID: \"1bf2b1e4-0123-4a0b-9a30-b2ea3794732c\") " pod="kube-system/global-pull-secret-syncer-7gfx9" Jul 28 01:26:19.541460 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:19.541022 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/005aafcd-65c4-45cb-aa21-484a0c94167e-metrics-certs\") pod \"network-metrics-daemon-brftp\" (UID: \"005aafcd-65c4-45cb-aa21-484a0c94167e\") " pod="openshift-multus/network-metrics-daemon-brftp" Jul 28 01:26:19.541460 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:19.541128 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 01:26:19.541460 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:19.541139 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 01:26:19.541460 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:19.541188 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/005aafcd-65c4-45cb-aa21-484a0c94167e-metrics-certs podName:005aafcd-65c4-45cb-aa21-484a0c94167e nodeName:}" failed. No retries permitted until 2026-07-28 01:26:35.54117257 +0000 UTC m=+34.095297369 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/005aafcd-65c4-45cb-aa21-484a0c94167e-metrics-certs") pod "network-metrics-daemon-brftp" (UID: "005aafcd-65c4-45cb-aa21-484a0c94167e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 01:26:19.541460 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:19.541202 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bf2b1e4-0123-4a0b-9a30-b2ea3794732c-original-pull-secret podName:1bf2b1e4-0123-4a0b-9a30-b2ea3794732c nodeName:}" failed. No retries permitted until 2026-07-28 01:26:35.541195726 +0000 UTC m=+34.095320502 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1bf2b1e4-0123-4a0b-9a30-b2ea3794732c-original-pull-secret") pod "global-pull-secret-syncer-7gfx9" (UID: "1bf2b1e4-0123-4a0b-9a30-b2ea3794732c") : object "kube-system"/"original-pull-secret" not registered Jul 28 01:26:19.642426 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:19.642393 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cfb7\" (UniqueName: \"kubernetes.io/projected/5060905d-b8ac-4bbd-94cb-b1a8a8952a9a-kube-api-access-8cfb7\") pod \"network-check-target-2znt8\" (UID: \"5060905d-b8ac-4bbd-94cb-b1a8a8952a9a\") " pod="openshift-network-diagnostics/network-check-target-2znt8" Jul 28 01:26:19.642615 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:19.642558 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 01:26:19.642615 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:19.642578 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 01:26:19.642615 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:19.642590 2578 projected.go:196] Error preparing data for projected volume kube-api-access-8cfb7 for pod openshift-network-diagnostics/network-check-target-2znt8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 01:26:19.642818 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:19.642656 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5060905d-b8ac-4bbd-94cb-b1a8a8952a9a-kube-api-access-8cfb7 podName:5060905d-b8ac-4bbd-94cb-b1a8a8952a9a nodeName:}" failed. No retries permitted until 2026-07-28 01:26:35.64263421 +0000 UTC m=+34.196759006 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-8cfb7" (UniqueName: "kubernetes.io/projected/5060905d-b8ac-4bbd-94cb-b1a8a8952a9a-kube-api-access-8cfb7") pod "network-check-target-2znt8" (UID: "5060905d-b8ac-4bbd-94cb-b1a8a8952a9a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 01:26:20.005126 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:20.005070 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brftp" Jul 28 01:26:20.005126 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:20.005088 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gfx9" Jul 28 01:26:20.005126 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:20.005097 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2znt8" Jul 28 01:26:20.005393 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:20.005207 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-brftp" podUID="005aafcd-65c4-45cb-aa21-484a0c94167e" Jul 28 01:26:20.005393 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:20.005308 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gfx9" podUID="1bf2b1e4-0123-4a0b-9a30-b2ea3794732c" Jul 28 01:26:20.005493 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:20.005401 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2znt8" podUID="5060905d-b8ac-4bbd-94cb-b1a8a8952a9a" Jul 28 01:26:22.005120 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:22.005082 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brftp" Jul 28 01:26:22.005610 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:22.005205 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gfx9" Jul 28 01:26:22.005610 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:22.005214 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-brftp" podUID="005aafcd-65c4-45cb-aa21-484a0c94167e" Jul 28 01:26:22.005610 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:22.005321 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gfx9" podUID="1bf2b1e4-0123-4a0b-9a30-b2ea3794732c" Jul 28 01:26:22.005610 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:22.005386 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2znt8" Jul 28 01:26:22.005610 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:22.005452 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2znt8" podUID="5060905d-b8ac-4bbd-94cb-b1a8a8952a9a" Jul 28 01:26:24.005268 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:24.004972 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gfx9" Jul 28 01:26:24.005864 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:24.004976 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brftp" Jul 28 01:26:24.005864 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:24.005348 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gfx9" podUID="1bf2b1e4-0123-4a0b-9a30-b2ea3794732c" Jul 28 01:26:24.005864 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:24.005025 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2znt8" Jul 28 01:26:24.005864 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:24.005422 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-brftp" podUID="005aafcd-65c4-45cb-aa21-484a0c94167e" Jul 28 01:26:24.005864 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:24.005629 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2znt8" podUID="5060905d-b8ac-4bbd-94cb-b1a8a8952a9a" Jul 28 01:26:24.097269 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:24.097232 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-186.ec2.internal" event={"ID":"45f7ea7075c86f8ee5fac1e99899f141","Type":"ContainerStarted","Data":"d7a2724b2e0b2e87cf7ebf91e075514316aa56285893f5b9b45c74b93b3db1d9"} Jul 28 01:26:24.099503 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:24.099475 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" event={"ID":"3a77a2d8-54f1-4eae-b356-213cd00c7462","Type":"ContainerStarted","Data":"89e27be36f06b98c72b56a2804c2aa83bf80ebd0024b29e3fc9374fb04b3e8b0"} Jul 28 01:26:24.099644 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:24.099574 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" event={"ID":"3a77a2d8-54f1-4eae-b356-213cd00c7462","Type":"ContainerStarted","Data":"4caedc1c8a7eb0f661114bab6a5c97dcc4a76797eeaecee9a53fab5dd26cb35b"} Jul 28 01:26:24.099644 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:24.099620 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" event={"ID":"3a77a2d8-54f1-4eae-b356-213cd00c7462","Type":"ContainerStarted","Data":"91ecd85b161c253faf5a72cbf33cbb4b6b76f80d192ee3c4367a9b61c91693be"} Jul 28 01:26:24.099750 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:24.099649 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" event={"ID":"3a77a2d8-54f1-4eae-b356-213cd00c7462","Type":"ContainerStarted","Data":"4d3252de719f026b0a767c7edf4ca9f718f57abfb3cc722d41ce7e2a1e70b081"} Jul 28 01:26:24.100911 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:24.100860 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" event={"ID":"03b14c3f-225a-4d46-9f03-31f3c8da2916","Type":"ContainerStarted","Data":"381c67558138181866034909711a5a4bbc60900a2b7f4217b787f75769bf9afa"} Jul 28 01:26:24.102419 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:24.102394 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hnpn2" event={"ID":"751f8878-a0ed-4756-93c1-97a66b2afd10","Type":"ContainerStarted","Data":"a7ff80f1eee47c2436c4fb10379bb516f035da5c110c897f8e0dd080f90f49c6"} Jul 28 01:26:24.103721 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:24.103690 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ww5wn" event={"ID":"f80c0089-ca61-4ddb-850c-6c805e916aaf","Type":"ContainerStarted","Data":"57d7d5a53572b45c28091434bbfee5f4da432fc75ea7d3b40327e13e0198bd0d"} Jul 28 01:26:24.104965 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:24.104946 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-x598j" event={"ID":"7af7909b-e421-437d-b0a8-895ad38876c7","Type":"ContainerStarted","Data":"42fbfd9edca806ed6df126d6c3f010c8aa9c54af1ac798fd7c2828387a37200d"} Jul 28 01:26:24.106506 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:24.106486 2578 generic.go:356] "Generic (PLEG): container finished" podID="3394f81c-6c57-451b-8d8e-2b6504f34ed3" containerID="99a1c0428fd94945bbf546f62483e0ff7acb132e95e139df57f59ff920cb2b44" exitCode=0 Jul 28 01:26:24.106608 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:24.106581 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wgdbs" event={"ID":"3394f81c-6c57-451b-8d8e-2b6504f34ed3","Type":"ContainerDied","Data":"99a1c0428fd94945bbf546f62483e0ff7acb132e95e139df57f59ff920cb2b44"} Jul 28 01:26:24.107968 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:24.107935 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bgvmj" event={"ID":"e1808965-dc33-46a8-a355-b3ebbe849c65","Type":"ContainerStarted","Data":"59e8e0c8d4d1c7d8b7a66a03628a34bc53850eebef0997265a3c7ba7312e357b"} Jul 28 01:26:24.109380 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:24.109360 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2vjkb" event={"ID":"516e3ead-ffab-4201-9715-2684dcfc7aa6","Type":"ContainerStarted","Data":"6f6db428961b548d27ff3fb4ff364a7dffda6c66227357c892daa1b94cfce588"} Jul 28 01:26:24.146702 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:24.146647 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-186.ec2.internal" podStartSLOduration=21.146629425 podStartE2EDuration="21.146629425s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 01:26:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:26:24.116425342 +0000 UTC m=+22.670550141" watchObservedRunningTime="2026-07-28 01:26:24.146629425 +0000 UTC m=+22.700754226" Jul 28 01:26:24.171318 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:24.171265 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-2vjkb" podStartSLOduration=2.806661385 podStartE2EDuration="22.171246762s" totalImagesPullingTime="19.364585377s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 01:26:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:26:24.171117644 +0000 UTC m=+22.725242442" watchObservedRunningTime="2026-07-28 01:26:24.171246762 +0000 UTC m=+22.725371561" Jul 28 01:26:24.241172 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:24.241122 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-x598j" podStartSLOduration=2.839647226 podStartE2EDuration="22.241105809s" totalImagesPullingTime="19.401458583s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 01:26:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:26:24.241085305 +0000 UTC m=+22.795210105" watchObservedRunningTime="2026-07-28 01:26:24.241105809 +0000 UTC m=+22.795230627" Jul 28 01:26:24.241528 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:24.241501 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-kmk8x" podStartSLOduration=2.896328686 podStartE2EDuration="22.241491075s" totalImagesPullingTime="19.345162389s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 01:26:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:26:24.203355882 +0000 UTC m=+22.757480681" watchObservedRunningTime="2026-07-28 01:26:24.241491075 +0000 UTC m=+22.795615872" Jul 28 01:26:24.314669 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:24.314608 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-hnpn2" podStartSLOduration=2.967009858 podStartE2EDuration="22.314588469s" totalImagesPullingTime="19.347578611s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:26:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:26:24.273493944 +0000 UTC m=+22.827618744" watchObservedRunningTime="2026-07-28 01:26:24.314588469 +0000 UTC m=+22.868713249" Jul 28 01:26:24.315269 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:24.315239 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-bgvmj" podStartSLOduration=2.995086306 podStartE2EDuration="22.3152296s" totalImagesPullingTime="19.320143294s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 01:26:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:26:24.314579004 +0000 UTC m=+22.868703803" watchObservedRunningTime="2026-07-28 01:26:24.3152296 +0000 UTC m=+22.869354438" Jul 28 01:26:25.099858 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:25.099836 2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 28 01:26:25.113958 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:25.113930 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" event={"ID":"3a77a2d8-54f1-4eae-b356-213cd00c7462","Type":"ContainerStarted","Data":"a6842d3fee1426de585b6307cde95aaad3a283549c3a1c43ee29da52e681af5b"} Jul 28 01:26:25.114070 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:25.114040 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" event={"ID":"3a77a2d8-54f1-4eae-b356-213cd00c7462","Type":"ContainerStarted","Data":"1d088912d98af6ab2cbbd4bfbe1bbff70b379e30bdd163e0b53e7b42e647222e"} Jul 28 01:26:25.115505 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:25.115481 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ww5wn" event={"ID":"f80c0089-ca61-4ddb-850c-6c805e916aaf","Type":"ContainerStarted","Data":"72d0345972e644ee55f453caa684d9cfa76e4efae0f49f5809fd2e13554e663a"} Jul 28 01:26:25.116836 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:25.116809 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qs68r" event={"ID":"3f0ab34c-4530-44b3-9bea-e54cdbb50377","Type":"ContainerStarted","Data":"56c9195af150ac6ba91368169f172ac03cbe50db7afb020a7a53bef70b8808b4"} Jul 28 01:26:25.131817 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:25.131777 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qs68r" podStartSLOduration=3.745115354 podStartE2EDuration="23.131762946s" totalImagesPullingTime="19.386647592s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 01:26:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:26:25.131466912 +0000 UTC m=+23.685591708" watchObservedRunningTime="2026-07-28 01:26:25.131762946 +0000 UTC m=+23.685887754" Jul 28 01:26:25.867809 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:25.867693 2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T01:26:25.099851717Z","UUID":"dea0742c-d4fd-481e-8ace-98b7365319d5","Handler":null,"Name":"","Endpoint":""} Jul 28 01:26:25.869777 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:25.869453 2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 28 01:26:25.869777 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:25.869497 2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 28 01:26:26.004979 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:26.004949 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2znt8" Jul 28 01:26:26.005143 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:26.004949 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brftp" Jul 28 01:26:26.005143 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:26.005076 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2znt8" podUID="5060905d-b8ac-4bbd-94cb-b1a8a8952a9a" Jul 28 01:26:26.005143 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:26.004949 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gfx9" Jul 28 01:26:26.005309 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:26.005180 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-brftp" podUID="005aafcd-65c4-45cb-aa21-484a0c94167e" Jul 28 01:26:26.005309 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:26.005255 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gfx9" podUID="1bf2b1e4-0123-4a0b-9a30-b2ea3794732c" Jul 28 01:26:27.124686 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:27.124648 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" event={"ID":"3a77a2d8-54f1-4eae-b356-213cd00c7462","Type":"ContainerStarted","Data":"c6b293116f1d439d9233628f07e787c7e346980d806e46f676cb05c09e8908cf"} Jul 28 01:26:27.127087 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:27.127059 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ww5wn" event={"ID":"f80c0089-ca61-4ddb-850c-6c805e916aaf","Type":"ContainerStarted","Data":"b8ef7179d6571d3fd6445058bc558bdd3f284643b1aeab4d19ec8a53435aeabe"} Jul 28 01:26:27.144534 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:27.144482 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ww5wn" podStartSLOduration=3.345926727 podStartE2EDuration="25.144463983s" totalImagesPullingTime="21.798537256s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 01:26:02 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:26:27.143490675 +0000 UTC m=+25.697615473" watchObservedRunningTime="2026-07-28 01:26:27.144463983 +0000 UTC m=+25.698588782" Jul 28 01:26:28.004209 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:28.004178 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brftp" Jul 28 01:26:28.004209 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:28.004197 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gfx9" Jul 28 01:26:28.004426 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:28.004179 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2znt8" Jul 28 01:26:28.004426 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:28.004294 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-brftp" podUID="005aafcd-65c4-45cb-aa21-484a0c94167e" Jul 28 01:26:28.004426 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:28.004365 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2znt8" podUID="5060905d-b8ac-4bbd-94cb-b1a8a8952a9a" Jul 28 01:26:28.004574 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:28.004444 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gfx9" podUID="1bf2b1e4-0123-4a0b-9a30-b2ea3794732c" Jul 28 01:26:29.049233 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:29.049024 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-hnpn2" Jul 28 01:26:29.050906 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:29.050878 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-hnpn2" Jul 28 01:26:29.137329 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:29.137299 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" event={"ID":"3a77a2d8-54f1-4eae-b356-213cd00c7462","Type":"ContainerStarted","Data":"dc8d18de9e068a9ab828d0bda39b4fe17afdaf571f964dd6598bb28b2bdffd3b"} Jul 28 01:26:29.137682 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:29.137658 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-hnpn2" Jul 28 01:26:29.138088 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:29.138071 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-hnpn2" Jul 28 01:26:29.166868 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:29.166815 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" podStartSLOduration=7.7338526 podStartE2EDuration="27.16677507s" totalImagesPullingTime="19.43292247s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 01:26:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:26:29.165418609 +0000 UTC m=+27.719543407" watchObservedRunningTime="2026-07-28 01:26:29.16677507 +0000 UTC m=+27.720899889" Jul 28 01:26:30.007663 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:30.007635 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2znt8" Jul 28 01:26:30.007663 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:30.007655 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gfx9" Jul 28 01:26:30.007875 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:30.007655 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brftp" Jul 28 01:26:30.007875 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:30.007720 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2znt8" podUID="5060905d-b8ac-4bbd-94cb-b1a8a8952a9a" Jul 28 01:26:30.007875 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:30.007810 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gfx9" podUID="1bf2b1e4-0123-4a0b-9a30-b2ea3794732c" Jul 28 01:26:30.008022 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:30.007888 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-brftp" podUID="005aafcd-65c4-45cb-aa21-484a0c94167e" Jul 28 01:26:30.150808 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:30.150145 2578 generic.go:356] "Generic (PLEG): container finished" podID="3394f81c-6c57-451b-8d8e-2b6504f34ed3" containerID="81c3c891eac98ca675dd63150c460b850fd354621a034b6ee0b7d103a17c331d" exitCode=0 Jul 28 01:26:30.150808 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:30.150340 2578 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 28 01:26:30.152083 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:30.151650 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wgdbs" event={"ID":"3394f81c-6c57-451b-8d8e-2b6504f34ed3","Type":"ContainerDied","Data":"81c3c891eac98ca675dd63150c460b850fd354621a034b6ee0b7d103a17c331d"} Jul 28 01:26:30.152083 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:30.151714 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:30.152255 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:30.152123 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:30.171208 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:30.171005 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:30.171478 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:30.171459 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:31.121289 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:31.121260 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7gfx9"] Jul 28 01:26:31.121509 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:31.121398 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gfx9" Jul 28 01:26:31.121509 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:31.121491 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gfx9" podUID="1bf2b1e4-0123-4a0b-9a30-b2ea3794732c" Jul 28 01:26:31.122897 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:31.122871 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-brftp"] Jul 28 01:26:31.123017 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:31.122970 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brftp" Jul 28 01:26:31.123087 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:31.123046 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-brftp" podUID="005aafcd-65c4-45cb-aa21-484a0c94167e" Jul 28 01:26:31.128251 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:31.128230 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2znt8"] Jul 28 01:26:31.128369 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:31.128310 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2znt8" Jul 28 01:26:31.128419 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:31.128373 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2znt8" podUID="5060905d-b8ac-4bbd-94cb-b1a8a8952a9a" Jul 28 01:26:31.152838 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:31.152815 2578 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 28 01:26:32.157047 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:32.157016 2578 generic.go:356] "Generic (PLEG): container finished" podID="3394f81c-6c57-451b-8d8e-2b6504f34ed3" containerID="44f87c33ed11c80f12cc06f1339da1aed13f5eb1d7d1331f7dc460d69d26aad7" exitCode=0 Jul 28 01:26:32.157529 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:32.157088 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wgdbs" event={"ID":"3394f81c-6c57-451b-8d8e-2b6504f34ed3","Type":"ContainerDied","Data":"44f87c33ed11c80f12cc06f1339da1aed13f5eb1d7d1331f7dc460d69d26aad7"} Jul 28 01:26:32.157529 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:32.157151 2578 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 28 01:26:33.005072 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:33.005041 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2znt8" Jul 28 01:26:33.005072 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:33.005072 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brftp" Jul 28 01:26:33.005239 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:33.005164 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2znt8" podUID="5060905d-b8ac-4bbd-94cb-b1a8a8952a9a" Jul 28 01:26:33.005239 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:33.005221 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gfx9" Jul 28 01:26:33.005352 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:33.005327 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-brftp" podUID="005aafcd-65c4-45cb-aa21-484a0c94167e" Jul 28 01:26:33.005416 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:33.005403 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gfx9" podUID="1bf2b1e4-0123-4a0b-9a30-b2ea3794732c" Jul 28 01:26:34.125490 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:34.125294 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:26:34.125969 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:34.125726 2578 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 28 01:26:34.141725 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:34.141683 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" podUID="3a77a2d8-54f1-4eae-b356-213cd00c7462" containerName="ovnkube-controller" probeResult="failure" output="" Jul 28 01:26:34.150335 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:34.150301 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" podUID="3a77a2d8-54f1-4eae-b356-213cd00c7462" containerName="ovnkube-controller" probeResult="failure" output="" Jul 28 01:26:34.164244 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:34.164216 2578 generic.go:356] "Generic (PLEG): container finished" podID="3394f81c-6c57-451b-8d8e-2b6504f34ed3" containerID="cdc21658a8e5704a74797d8a6a6f21cb30a955fd951e28d35e4a084ccc8b5fe1" exitCode=0 Jul 28 01:26:34.164346 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:34.164261 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wgdbs" event={"ID":"3394f81c-6c57-451b-8d8e-2b6504f34ed3","Type":"ContainerDied","Data":"cdc21658a8e5704a74797d8a6a6f21cb30a955fd951e28d35e4a084ccc8b5fe1"} Jul 28 01:26:35.004891 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:35.004856 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gfx9" Jul 28 01:26:35.004891 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:35.004891 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brftp" Jul 28 01:26:35.005145 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:35.004891 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2znt8" Jul 28 01:26:35.005145 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:35.004988 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7gfx9" podUID="1bf2b1e4-0123-4a0b-9a30-b2ea3794732c" Jul 28 01:26:35.005145 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:35.005113 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-brftp" podUID="005aafcd-65c4-45cb-aa21-484a0c94167e" Jul 28 01:26:35.005284 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:35.005175 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2znt8" podUID="5060905d-b8ac-4bbd-94cb-b1a8a8952a9a" Jul 28 01:26:35.559741 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:35.559708 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1bf2b1e4-0123-4a0b-9a30-b2ea3794732c-original-pull-secret\") pod \"global-pull-secret-syncer-7gfx9\" (UID: \"1bf2b1e4-0123-4a0b-9a30-b2ea3794732c\") " pod="kube-system/global-pull-secret-syncer-7gfx9" Jul 28 01:26:35.559741 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:35.559748 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/005aafcd-65c4-45cb-aa21-484a0c94167e-metrics-certs\") pod \"network-metrics-daemon-brftp\" (UID: \"005aafcd-65c4-45cb-aa21-484a0c94167e\") " pod="openshift-multus/network-metrics-daemon-brftp" Jul 28 01:26:35.560206 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:35.559870 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 01:26:35.560206 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:35.559871 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 01:26:35.560206 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:35.559927 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/005aafcd-65c4-45cb-aa21-484a0c94167e-metrics-certs podName:005aafcd-65c4-45cb-aa21-484a0c94167e nodeName:}" failed. No retries permitted until 2026-07-28 01:27:07.559909647 +0000 UTC m=+66.114034437 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/005aafcd-65c4-45cb-aa21-484a0c94167e-metrics-certs") pod "network-metrics-daemon-brftp" (UID: "005aafcd-65c4-45cb-aa21-484a0c94167e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 01:26:35.560206 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:35.559941 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bf2b1e4-0123-4a0b-9a30-b2ea3794732c-original-pull-secret podName:1bf2b1e4-0123-4a0b-9a30-b2ea3794732c nodeName:}" failed. No retries permitted until 2026-07-28 01:27:07.5599351 +0000 UTC m=+66.114059876 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1bf2b1e4-0123-4a0b-9a30-b2ea3794732c-original-pull-secret") pod "global-pull-secret-syncer-7gfx9" (UID: "1bf2b1e4-0123-4a0b-9a30-b2ea3794732c") : object "kube-system"/"original-pull-secret" not registered Jul 28 01:26:35.660990 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:35.660958 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cfb7\" (UniqueName: \"kubernetes.io/projected/5060905d-b8ac-4bbd-94cb-b1a8a8952a9a-kube-api-access-8cfb7\") pod \"network-check-target-2znt8\" (UID: \"5060905d-b8ac-4bbd-94cb-b1a8a8952a9a\") " pod="openshift-network-diagnostics/network-check-target-2znt8" Jul 28 01:26:35.661161 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:35.661112 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 01:26:35.661161 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:35.661130 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 01:26:35.661161 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:35.661141 2578 projected.go:196] Error preparing data for projected volume kube-api-access-8cfb7 for pod openshift-network-diagnostics/network-check-target-2znt8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 01:26:35.661298 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:35.661199 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5060905d-b8ac-4bbd-94cb-b1a8a8952a9a-kube-api-access-8cfb7 podName:5060905d-b8ac-4bbd-94cb-b1a8a8952a9a nodeName:}" failed. No retries permitted until 2026-07-28 01:27:07.661181214 +0000 UTC m=+66.215306004 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-8cfb7" (UniqueName: "kubernetes.io/projected/5060905d-b8ac-4bbd-94cb-b1a8a8952a9a-kube-api-access-8cfb7") pod "network-check-target-2znt8" (UID: "5060905d-b8ac-4bbd-94cb-b1a8a8952a9a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 01:26:36.334435 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.334351 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-186.ec2.internal" event="NodeReady" Jul 28 01:26:36.334610 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.334501 2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 28 01:26:36.376816 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.376784 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-757879ff5c-x2xkc"] Jul 28 01:26:36.391991 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.391955 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-757879ff5c-x2xkc" Jul 28 01:26:36.395762 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.395737 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 28 01:26:36.396118 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.396098 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-m2p2f\"" Jul 28 01:26:36.396315 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.396300 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 28 01:26:36.396631 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.396608 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 28 01:26:36.398404 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.398340 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-8fbhx"] Jul 28 01:26:36.407932 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.407907 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 28 01:26:36.415960 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.415931 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-f297n"] Jul 28 01:26:36.416079 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.416064 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-8fbhx" Jul 28 01:26:36.420113 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.420091 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-4dshn\"" Jul 28 01:26:36.420598 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.420574 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 28 01:26:36.420709 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.420574 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 28 01:26:36.438352 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.438213 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-q8ldl"] Jul 28 01:26:36.438845 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.438819 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-f297n" Jul 28 01:26:36.444305 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.444282 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 28 01:26:36.444636 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.444527 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 28 01:26:36.444636 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.444526 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 28 01:26:36.444636 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.444570 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-b78xn\"" Jul 28 01:26:36.448935 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.446411 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 28 01:26:36.473566 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.463944 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-m7p9h"] Jul 28 01:26:36.473566 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.469758 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/31992241-f826-45d3-a26a-679d5d82465c-registry-certificates\") pod \"image-registry-757879ff5c-x2xkc\" (UID: \"31992241-f826-45d3-a26a-679d5d82465c\") " pod="openshift-image-registry/image-registry-757879ff5c-x2xkc" Jul 28 01:26:36.473566 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.469805 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/31992241-f826-45d3-a26a-679d5d82465c-trusted-ca\") pod \"image-registry-757879ff5c-x2xkc\" (UID: \"31992241-f826-45d3-a26a-679d5d82465c\") " pod="openshift-image-registry/image-registry-757879ff5c-x2xkc" Jul 28 01:26:36.473566 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.469834 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/31992241-f826-45d3-a26a-679d5d82465c-registry-tls\") pod \"image-registry-757879ff5c-x2xkc\" (UID: \"31992241-f826-45d3-a26a-679d5d82465c\") " pod="openshift-image-registry/image-registry-757879ff5c-x2xkc" Jul 28 01:26:36.473566 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.469868 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7vdf\" (UniqueName: \"kubernetes.io/projected/31992241-f826-45d3-a26a-679d5d82465c-kube-api-access-k7vdf\") pod \"image-registry-757879ff5c-x2xkc\" (UID: \"31992241-f826-45d3-a26a-679d5d82465c\") " pod="openshift-image-registry/image-registry-757879ff5c-x2xkc" Jul 28 01:26:36.473566 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.469901 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/31992241-f826-45d3-a26a-679d5d82465c-ca-trust-extracted\") pod \"image-registry-757879ff5c-x2xkc\" (UID: \"31992241-f826-45d3-a26a-679d5d82465c\") " pod="openshift-image-registry/image-registry-757879ff5c-x2xkc" Jul 28 01:26:36.473566 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.469935 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/31992241-f826-45d3-a26a-679d5d82465c-installation-pull-secrets\") pod \"image-registry-757879ff5c-x2xkc\" (UID: \"31992241-f826-45d3-a26a-679d5d82465c\") " pod="openshift-image-registry/image-registry-757879ff5c-x2xkc" Jul 28 01:26:36.473566 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.469959 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/31992241-f826-45d3-a26a-679d5d82465c-bound-sa-token\") pod \"image-registry-757879ff5c-x2xkc\" (UID: \"31992241-f826-45d3-a26a-679d5d82465c\") " pod="openshift-image-registry/image-registry-757879ff5c-x2xkc" Jul 28 01:26:36.473566 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.470002 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/31992241-f826-45d3-a26a-679d5d82465c-image-registry-private-configuration\") pod \"image-registry-757879ff5c-x2xkc\" (UID: \"31992241-f826-45d3-a26a-679d5d82465c\") " pod="openshift-image-registry/image-registry-757879ff5c-x2xkc" Jul 28 01:26:36.473566 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.470113 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-q8ldl" Jul 28 01:26:36.478764 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.474304 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-7fjd2\"" Jul 28 01:26:36.478764 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.474527 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 28 01:26:36.478764 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.474762 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 28 01:26:36.478764 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.474927 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 28 01:26:36.478764 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.475041 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 28 01:26:36.486709 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.485519 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 28 01:26:36.486709 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.486124 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-t5dxh"] Jul 28 01:26:36.487501 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.487478 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-m7p9h" Jul 28 01:26:36.490671 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.490649 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 28 01:26:36.490948 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.490925 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-bzh9v\"" Jul 28 01:26:36.491764 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.491747 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 28 01:26:36.492301 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.491826 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 28 01:26:36.492301 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.492152 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 28 01:26:36.500051 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.500028 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-757879ff5c-x2xkc"] Jul 28 01:26:36.500149 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.500065 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-m7p9h"] Jul 28 01:26:36.500149 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.500085 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-q8ldl"] Jul 28 01:26:36.500149 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.500104 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-8fbhx"] Jul 28 01:26:36.500149 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.500121 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-t5dxh"] Jul 28 01:26:36.500149 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.500144 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-f297n"] Jul 28 01:26:36.500358 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.500171 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-n5dq2"] Jul 28 01:26:36.500358 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.500231 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-t5dxh" Jul 28 01:26:36.503733 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.503481 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 28 01:26:36.503733 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.503672 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-jbb4n\"" Jul 28 01:26:36.504194 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.504172 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 28 01:26:36.504284 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.504198 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 28 01:26:36.516227 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.516198 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-phhjx"] Jul 28 01:26:36.516388 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.516344 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-n5dq2" Jul 28 01:26:36.523345 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.523296 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7zgnb\"" Jul 28 01:26:36.523720 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.523697 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 28 01:26:36.524033 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.524013 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 28 01:26:36.532146 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.532121 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-k2wlm"] Jul 28 01:26:36.532285 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.532261 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-phhjx" Jul 28 01:26:36.535213 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.535195 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 28 01:26:36.535491 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.535473 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 28 01:26:36.535773 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.535758 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-s8w2l\"" Jul 28 01:26:36.536001 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.535976 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 28 01:26:36.536634 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.536613 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 28 01:26:36.558559 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.558515 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-4pl5d"] Jul 28 01:26:36.558709 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.558638 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-k2wlm" Jul 28 01:26:36.561725 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.561704 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 28 01:26:36.562403 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.561733 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-ppvnt\"" Jul 28 01:26:36.562403 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.562231 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 28 01:26:36.562672 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.562624 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 28 01:26:36.566360 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.566342 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 28 01:26:36.571097 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.571078 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/31992241-f826-45d3-a26a-679d5d82465c-registry-tls\") pod \"image-registry-757879ff5c-x2xkc\" (UID: \"31992241-f826-45d3-a26a-679d5d82465c\") " pod="openshift-image-registry/image-registry-757879ff5c-x2xkc" Jul 28 01:26:36.571169 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.571112 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18e29850-170d-4a22-b890-95237b45246d-service-ca-bundle\") pod \"insights-operator-5885fdb759-q8ldl\" (UID: \"18e29850-170d-4a22-b890-95237b45246d\") " pod="openshift-insights/insights-operator-5885fdb759-q8ldl" Jul 28 01:26:36.571169 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.571145 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b4e84366-546a-4004-b614-6b3c35c61dfe-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-m7p9h\" (UID: \"b4e84366-546a-4004-b614-6b3c35c61dfe\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-m7p9h" Jul 28 01:26:36.571267 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.571168 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7vdf\" (UniqueName: \"kubernetes.io/projected/31992241-f826-45d3-a26a-679d5d82465c-kube-api-access-k7vdf\") pod \"image-registry-757879ff5c-x2xkc\" (UID: \"31992241-f826-45d3-a26a-679d5d82465c\") " pod="openshift-image-registry/image-registry-757879ff5c-x2xkc" Jul 28 01:26:36.571267 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:36.571233 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 01:26:36.571267 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:36.571246 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-757879ff5c-x2xkc: secret "image-registry-tls" not found Jul 28 01:26:36.571405 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:36.571298 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31992241-f826-45d3-a26a-679d5d82465c-registry-tls podName:31992241-f826-45d3-a26a-679d5d82465c nodeName:}" failed. No retries permitted until 2026-07-28 01:26:37.071277359 +0000 UTC m=+35.625402159 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/31992241-f826-45d3-a26a-679d5d82465c-registry-tls") pod "image-registry-757879ff5c-x2xkc" (UID: "31992241-f826-45d3-a26a-679d5d82465c") : secret "image-registry-tls" not found Jul 28 01:26:36.571405 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.571321 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l82k2\" (UniqueName: \"kubernetes.io/projected/56e7bae5-9b23-4f6b-8084-20287730dd30-kube-api-access-l82k2\") pod \"cluster-monitoring-operator-548b489cb8-f297n\" (UID: \"56e7bae5-9b23-4f6b-8084-20287730dd30\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-f297n" Jul 28 01:26:36.571405 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.571361 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/56e7bae5-9b23-4f6b-8084-20287730dd30-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-f297n\" (UID: \"56e7bae5-9b23-4f6b-8084-20287730dd30\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-f297n" Jul 28 01:26:36.571594 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.571417 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/31992241-f826-45d3-a26a-679d5d82465c-ca-trust-extracted\") pod \"image-registry-757879ff5c-x2xkc\" (UID: \"31992241-f826-45d3-a26a-679d5d82465c\") " pod="openshift-image-registry/image-registry-757879ff5c-x2xkc" Jul 28 01:26:36.571594 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.571444 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/31992241-f826-45d3-a26a-679d5d82465c-installation-pull-secrets\") pod \"image-registry-757879ff5c-x2xkc\" (UID: \"31992241-f826-45d3-a26a-679d5d82465c\") " pod="openshift-image-registry/image-registry-757879ff5c-x2xkc" Jul 28 01:26:36.571594 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.571481 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/31992241-f826-45d3-a26a-679d5d82465c-bound-sa-token\") pod \"image-registry-757879ff5c-x2xkc\" (UID: \"31992241-f826-45d3-a26a-679d5d82465c\") " pod="openshift-image-registry/image-registry-757879ff5c-x2xkc" Jul 28 01:26:36.571594 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.571511 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/56e7bae5-9b23-4f6b-8084-20287730dd30-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-f297n\" (UID: \"56e7bae5-9b23-4f6b-8084-20287730dd30\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-f297n" Jul 28 01:26:36.571594 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.571538 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2js5\" (UniqueName: \"kubernetes.io/projected/18e29850-170d-4a22-b890-95237b45246d-kube-api-access-m2js5\") pod \"insights-operator-5885fdb759-q8ldl\" (UID: \"18e29850-170d-4a22-b890-95237b45246d\") " pod="openshift-insights/insights-operator-5885fdb759-q8ldl" Jul 28 01:26:36.571816 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.571578 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhqgn\" (UniqueName: \"kubernetes.io/projected/dbab9a89-f5fa-4a89-ba0a-d0ab2b4b8957-kube-api-access-qhqgn\") pod \"volume-data-source-validator-549c6d54cf-8fbhx\" (UID: \"dbab9a89-f5fa-4a89-ba0a-d0ab2b4b8957\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-8fbhx" Jul 28 01:26:36.571816 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.571804 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/31992241-f826-45d3-a26a-679d5d82465c-ca-trust-extracted\") pod \"image-registry-757879ff5c-x2xkc\" (UID: \"31992241-f826-45d3-a26a-679d5d82465c\") " pod="openshift-image-registry/image-registry-757879ff5c-x2xkc" Jul 28 01:26:36.571923 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.571859 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/31992241-f826-45d3-a26a-679d5d82465c-image-registry-private-configuration\") pod \"image-registry-757879ff5c-x2xkc\" (UID: \"31992241-f826-45d3-a26a-679d5d82465c\") " pod="openshift-image-registry/image-registry-757879ff5c-x2xkc" Jul 28 01:26:36.571923 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.571886 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/31992241-f826-45d3-a26a-679d5d82465c-registry-certificates\") pod \"image-registry-757879ff5c-x2xkc\" (UID: \"31992241-f826-45d3-a26a-679d5d82465c\") " pod="openshift-image-registry/image-registry-757879ff5c-x2xkc" Jul 28 01:26:36.571923 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.571903 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18e29850-170d-4a22-b890-95237b45246d-tmp\") pod \"insights-operator-5885fdb759-q8ldl\" (UID: \"18e29850-170d-4a22-b890-95237b45246d\") " pod="openshift-insights/insights-operator-5885fdb759-q8ldl" Jul 28 01:26:36.572028 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.571926 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b4e84366-546a-4004-b614-6b3c35c61dfe-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-m7p9h\" (UID: \"b4e84366-546a-4004-b614-6b3c35c61dfe\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-m7p9h" Jul 28 01:26:36.572028 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.571955 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/18e29850-170d-4a22-b890-95237b45246d-snapshots\") pod \"insights-operator-5885fdb759-q8ldl\" (UID: \"18e29850-170d-4a22-b890-95237b45246d\") " pod="openshift-insights/insights-operator-5885fdb759-q8ldl" Jul 28 01:26:36.572028 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.571985 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhwgf\" (UniqueName: \"kubernetes.io/projected/b4e84366-546a-4004-b614-6b3c35c61dfe-kube-api-access-nhwgf\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-m7p9h\" (UID: \"b4e84366-546a-4004-b614-6b3c35c61dfe\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-m7p9h" Jul 28 01:26:36.572028 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.572023 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/31992241-f826-45d3-a26a-679d5d82465c-trusted-ca\") pod \"image-registry-757879ff5c-x2xkc\" (UID: \"31992241-f826-45d3-a26a-679d5d82465c\") " pod="openshift-image-registry/image-registry-757879ff5c-x2xkc" Jul 28 01:26:36.572274 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.572048 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18e29850-170d-4a22-b890-95237b45246d-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-q8ldl\" (UID: \"18e29850-170d-4a22-b890-95237b45246d\") " pod="openshift-insights/insights-operator-5885fdb759-q8ldl" Jul 28 01:26:36.572274 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.572070 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/18e29850-170d-4a22-b890-95237b45246d-serving-cert\") pod \"insights-operator-5885fdb759-q8ldl\" (UID: \"18e29850-170d-4a22-b890-95237b45246d\") " pod="openshift-insights/insights-operator-5885fdb759-q8ldl" Jul 28 01:26:36.573087 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.572991 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/31992241-f826-45d3-a26a-679d5d82465c-registry-certificates\") pod \"image-registry-757879ff5c-x2xkc\" (UID: \"31992241-f826-45d3-a26a-679d5d82465c\") " pod="openshift-image-registry/image-registry-757879ff5c-x2xkc" Jul 28 01:26:36.573199 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.573143 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/31992241-f826-45d3-a26a-679d5d82465c-trusted-ca\") pod \"image-registry-757879ff5c-x2xkc\" (UID: \"31992241-f826-45d3-a26a-679d5d82465c\") " pod="openshift-image-registry/image-registry-757879ff5c-x2xkc" Jul 28 01:26:36.574953 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.574512 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-74577ddb85-mm25q"] Jul 28 01:26:36.575613 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.575565 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-4pl5d" Jul 28 01:26:36.577690 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.577292 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/31992241-f826-45d3-a26a-679d5d82465c-image-registry-private-configuration\") pod \"image-registry-757879ff5c-x2xkc\" (UID: \"31992241-f826-45d3-a26a-679d5d82465c\") " pod="openshift-image-registry/image-registry-757879ff5c-x2xkc" Jul 28 01:26:36.578440 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.578415 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-jsw2s\"" Jul 28 01:26:36.578565 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.578454 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 28 01:26:36.578640 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.578621 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/31992241-f826-45d3-a26a-679d5d82465c-installation-pull-secrets\") pod \"image-registry-757879ff5c-x2xkc\" (UID: \"31992241-f826-45d3-a26a-679d5d82465c\") " pod="openshift-image-registry/image-registry-757879ff5c-x2xkc" Jul 28 01:26:36.579659 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.579639 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 28 01:26:36.580535 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.580519 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 28 01:26:36.586085 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.586032 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7vdf\" (UniqueName: \"kubernetes.io/projected/31992241-f826-45d3-a26a-679d5d82465c-kube-api-access-k7vdf\") pod \"image-registry-757879ff5c-x2xkc\" (UID: \"31992241-f826-45d3-a26a-679d5d82465c\") " pod="openshift-image-registry/image-registry-757879ff5c-x2xkc" Jul 28 01:26:36.591492 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.591464 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/31992241-f826-45d3-a26a-679d5d82465c-bound-sa-token\") pod \"image-registry-757879ff5c-x2xkc\" (UID: \"31992241-f826-45d3-a26a-679d5d82465c\") " pod="openshift-image-registry/image-registry-757879ff5c-x2xkc" Jul 28 01:26:36.593019 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.592942 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-sc92r"] Jul 28 01:26:36.593268 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.593250 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-74577ddb85-mm25q" Jul 28 01:26:36.596351 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.596330 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 28 01:26:36.596436 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.596384 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 28 01:26:36.596436 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.596423 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 28 01:26:36.596561 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.596432 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 28 01:26:36.596561 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.596388 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 28 01:26:36.596665 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.596598 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 28 01:26:36.596665 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.596631 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-56bw5\"" Jul 28 01:26:36.612924 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.612900 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-hrdd9"] Jul 28 01:26:36.613048 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.613031 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-sc92r" Jul 28 01:26:36.615803 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.615774 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-x7c4m\"" Jul 28 01:26:36.616032 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.616016 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 28 01:26:36.616032 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.616032 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 28 01:26:36.616172 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.616019 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 28 01:26:36.633187 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.633168 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-4pl5d"] Jul 28 01:26:36.633292 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.633203 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-phhjx"] Jul 28 01:26:36.633292 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.633215 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-k2wlm"] Jul 28 01:26:36.633292 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.633227 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-sc92r"] Jul 28 01:26:36.633292 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.633239 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-n5dq2"] Jul 28 01:26:36.633292 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.633253 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-74577ddb85-mm25q"] Jul 28 01:26:36.633292 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.633265 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-hrdd9"] Jul 28 01:26:36.633292 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.633271 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-hrdd9" Jul 28 01:26:36.635818 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.635794 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 28 01:26:36.636239 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.636226 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-tl76z\"" Jul 28 01:26:36.636427 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.636415 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 28 01:26:36.672829 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.672805 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-metrics-certs\") pod \"router-default-74577ddb85-mm25q\" (UID: \"923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71\") " pod="openshift-ingress/router-default-74577ddb85-mm25q" Jul 28 01:26:36.672829 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.672833 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3503116a-1398-42ea-ac86-743f5f51e989-config\") pod \"console-operator-584dcb6b54-k2wlm\" (UID: \"3503116a-1398-42ea-ac86-743f5f51e989\") " pod="openshift-console-operator/console-operator-584dcb6b54-k2wlm" Jul 28 01:26:36.673024 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.672860 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/18e29850-170d-4a22-b890-95237b45246d-serving-cert\") pod \"insights-operator-5885fdb759-q8ldl\" (UID: \"18e29850-170d-4a22-b890-95237b45246d\") " pod="openshift-insights/insights-operator-5885fdb759-q8ldl" Jul 28 01:26:36.673024 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.672902 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b4e84366-546a-4004-b614-6b3c35c61dfe-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-m7p9h\" (UID: \"b4e84366-546a-4004-b614-6b3c35c61dfe\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-m7p9h" Jul 28 01:26:36.673024 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.672924 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1f2321ec-7f9d-4e33-9bed-d48fd717ca07-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4pl5d\" (UID: \"1f2321ec-7f9d-4e33-9bed-d48fd717ca07\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4pl5d" Jul 28 01:26:36.673024 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.672948 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2js5\" (UniqueName: \"kubernetes.io/projected/18e29850-170d-4a22-b890-95237b45246d-kube-api-access-m2js5\") pod \"insights-operator-5885fdb759-q8ldl\" (UID: \"18e29850-170d-4a22-b890-95237b45246d\") " pod="openshift-insights/insights-operator-5885fdb759-q8ldl" Jul 28 01:26:36.673024 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.672978 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5873b663-f99f-453e-8aaf-d57bf3fd0105-metrics-tls\") pod \"dns-default-n5dq2\" (UID: \"5873b663-f99f-453e-8aaf-d57bf3fd0105\") " pod="openshift-dns/dns-default-n5dq2" Jul 28 01:26:36.673024 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.673001 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3503116a-1398-42ea-ac86-743f5f51e989-serving-cert\") pod \"console-operator-584dcb6b54-k2wlm\" (UID: \"3503116a-1398-42ea-ac86-743f5f51e989\") " pod="openshift-console-operator/console-operator-584dcb6b54-k2wlm" Jul 28 01:26:36.673229 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.673031 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b4e84366-546a-4004-b614-6b3c35c61dfe-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-m7p9h\" (UID: \"b4e84366-546a-4004-b614-6b3c35c61dfe\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-m7p9h" Jul 28 01:26:36.673229 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.673076 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18e29850-170d-4a22-b890-95237b45246d-tmp\") pod \"insights-operator-5885fdb759-q8ldl\" (UID: \"18e29850-170d-4a22-b890-95237b45246d\") " pod="openshift-insights/insights-operator-5885fdb759-q8ldl" Jul 28 01:26:36.673229 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.673107 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-service-ca-bundle\") pod \"router-default-74577ddb85-mm25q\" (UID: \"923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71\") " pod="openshift-ingress/router-default-74577ddb85-mm25q" Jul 28 01:26:36.673229 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.673154 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhwgf\" (UniqueName: \"kubernetes.io/projected/b4e84366-546a-4004-b614-6b3c35c61dfe-kube-api-access-nhwgf\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-m7p9h\" (UID: \"b4e84366-546a-4004-b614-6b3c35c61dfe\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-m7p9h" Jul 28 01:26:36.673229 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.673192 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18e29850-170d-4a22-b890-95237b45246d-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-q8ldl\" (UID: \"18e29850-170d-4a22-b890-95237b45246d\") " pod="openshift-insights/insights-operator-5885fdb759-q8ldl" Jul 28 01:26:36.673419 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.673244 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56wdv\" (UniqueName: \"kubernetes.io/projected/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-kube-api-access-56wdv\") pod \"router-default-74577ddb85-mm25q\" (UID: \"923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71\") " pod="openshift-ingress/router-default-74577ddb85-mm25q" Jul 28 01:26:36.673419 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.673388 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3503116a-1398-42ea-ac86-743f5f51e989-trusted-ca\") pod \"console-operator-584dcb6b54-k2wlm\" (UID: \"3503116a-1398-42ea-ac86-743f5f51e989\") " pod="openshift-console-operator/console-operator-584dcb6b54-k2wlm" Jul 28 01:26:36.673512 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.673437 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2af2ea06-6e9c-4367-90ae-755b5f034806-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-t5dxh\" (UID: \"2af2ea06-6e9c-4367-90ae-755b5f034806\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-t5dxh" Jul 28 01:26:36.673512 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.673460 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18e29850-170d-4a22-b890-95237b45246d-tmp\") pod \"insights-operator-5885fdb759-q8ldl\" (UID: \"18e29850-170d-4a22-b890-95237b45246d\") " pod="openshift-insights/insights-operator-5885fdb759-q8ldl" Jul 28 01:26:36.673642 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.673529 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18e29850-170d-4a22-b890-95237b45246d-service-ca-bundle\") pod \"insights-operator-5885fdb759-q8ldl\" (UID: \"18e29850-170d-4a22-b890-95237b45246d\") " pod="openshift-insights/insights-operator-5885fdb759-q8ldl" Jul 28 01:26:36.673642 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.673580 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/afab8f75-0a1a-43a3-9a98-be26e28ab36f-serving-cert\") pod \"service-ca-operator-5d6c75f84f-phhjx\" (UID: \"afab8f75-0a1a-43a3-9a98-be26e28ab36f\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-phhjx" Jul 28 01:26:36.673642 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.673613 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l82k2\" (UniqueName: \"kubernetes.io/projected/56e7bae5-9b23-4f6b-8084-20287730dd30-kube-api-access-l82k2\") pod \"cluster-monitoring-operator-548b489cb8-f297n\" (UID: \"56e7bae5-9b23-4f6b-8084-20287730dd30\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-f297n" Jul 28 01:26:36.673642 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.673638 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v8gn\" (UniqueName: \"kubernetes.io/projected/5873b663-f99f-453e-8aaf-d57bf3fd0105-kube-api-access-4v8gn\") pod \"dns-default-n5dq2\" (UID: \"5873b663-f99f-453e-8aaf-d57bf3fd0105\") " pod="openshift-dns/dns-default-n5dq2" Jul 28 01:26:36.673846 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.673658 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b4e84366-546a-4004-b614-6b3c35c61dfe-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-m7p9h\" (UID: \"b4e84366-546a-4004-b614-6b3c35c61dfe\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-m7p9h" Jul 28 01:26:36.673846 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.673666 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/56e7bae5-9b23-4f6b-8084-20287730dd30-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-f297n\" (UID: \"56e7bae5-9b23-4f6b-8084-20287730dd30\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-f297n" Jul 28 01:26:36.673846 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.673693 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1f2321ec-7f9d-4e33-9bed-d48fd717ca07-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-4pl5d\" (UID: \"1f2321ec-7f9d-4e33-9bed-d48fd717ca07\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4pl5d" Jul 28 01:26:36.673846 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.673718 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhqgn\" (UniqueName: \"kubernetes.io/projected/dbab9a89-f5fa-4a89-ba0a-d0ab2b4b8957-kube-api-access-qhqgn\") pod \"volume-data-source-validator-549c6d54cf-8fbhx\" (UID: \"dbab9a89-f5fa-4a89-ba0a-d0ab2b4b8957\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-8fbhx" Jul 28 01:26:36.673846 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.673744 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-default-certificate\") pod \"router-default-74577ddb85-mm25q\" (UID: \"923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71\") " pod="openshift-ingress/router-default-74577ddb85-mm25q" Jul 28 01:26:36.673846 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.673776 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5873b663-f99f-453e-8aaf-d57bf3fd0105-tmp-dir\") pod \"dns-default-n5dq2\" (UID: \"5873b663-f99f-453e-8aaf-d57bf3fd0105\") " pod="openshift-dns/dns-default-n5dq2" Jul 28 01:26:36.673846 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.673818 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j56rn\" (UniqueName: \"kubernetes.io/projected/2af2ea06-6e9c-4367-90ae-755b5f034806-kube-api-access-j56rn\") pod \"cluster-samples-operator-6748489656-t5dxh\" (UID: \"2af2ea06-6e9c-4367-90ae-755b5f034806\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-t5dxh" Jul 28 01:26:36.674183 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.673856 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/56e7bae5-9b23-4f6b-8084-20287730dd30-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-f297n\" (UID: \"56e7bae5-9b23-4f6b-8084-20287730dd30\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-f297n" Jul 28 01:26:36.674183 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:36.673886 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 28 01:26:36.674183 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.673990 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-stats-auth\") pod \"router-default-74577ddb85-mm25q\" (UID: \"923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71\") " pod="openshift-ingress/router-default-74577ddb85-mm25q" Jul 28 01:26:36.674183 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.673996 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18e29850-170d-4a22-b890-95237b45246d-service-ca-bundle\") pod \"insights-operator-5885fdb759-q8ldl\" (UID: \"18e29850-170d-4a22-b890-95237b45246d\") " pod="openshift-insights/insights-operator-5885fdb759-q8ldl" Jul 28 01:26:36.674183 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:36.674061 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56e7bae5-9b23-4f6b-8084-20287730dd30-cluster-monitoring-operator-tls podName:56e7bae5-9b23-4f6b-8084-20287730dd30 nodeName:}" failed. No retries permitted until 2026-07-28 01:26:37.174042725 +0000 UTC m=+35.728167512 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/56e7bae5-9b23-4f6b-8084-20287730dd30-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-f297n" (UID: "56e7bae5-9b23-4f6b-8084-20287730dd30") : secret "cluster-monitoring-operator-tls" not found Jul 28 01:26:36.674183 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.674082 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18e29850-170d-4a22-b890-95237b45246d-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-q8ldl\" (UID: \"18e29850-170d-4a22-b890-95237b45246d\") " pod="openshift-insights/insights-operator-5885fdb759-q8ldl" Jul 28 01:26:36.674183 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.674105 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/afab8f75-0a1a-43a3-9a98-be26e28ab36f-config\") pod \"service-ca-operator-5d6c75f84f-phhjx\" (UID: \"afab8f75-0a1a-43a3-9a98-be26e28ab36f\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-phhjx" Jul 28 01:26:36.674183 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.674143 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmvws\" (UniqueName: \"kubernetes.io/projected/afab8f75-0a1a-43a3-9a98-be26e28ab36f-kube-api-access-wmvws\") pod \"service-ca-operator-5d6c75f84f-phhjx\" (UID: \"afab8f75-0a1a-43a3-9a98-be26e28ab36f\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-phhjx" Jul 28 01:26:36.674183 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.674188 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5873b663-f99f-453e-8aaf-d57bf3fd0105-config-volume\") pod \"dns-default-n5dq2\" (UID: \"5873b663-f99f-453e-8aaf-d57bf3fd0105\") " pod="openshift-dns/dns-default-n5dq2" Jul 28 01:26:36.674647 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.674228 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/18e29850-170d-4a22-b890-95237b45246d-snapshots\") pod \"insights-operator-5885fdb759-q8ldl\" (UID: \"18e29850-170d-4a22-b890-95237b45246d\") " pod="openshift-insights/insights-operator-5885fdb759-q8ldl" Jul 28 01:26:36.674647 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.674256 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wftsm\" (UniqueName: \"kubernetes.io/projected/3503116a-1398-42ea-ac86-743f5f51e989-kube-api-access-wftsm\") pod \"console-operator-584dcb6b54-k2wlm\" (UID: \"3503116a-1398-42ea-ac86-743f5f51e989\") " pod="openshift-console-operator/console-operator-584dcb6b54-k2wlm" Jul 28 01:26:36.674750 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.674735 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/56e7bae5-9b23-4f6b-8084-20287730dd30-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-f297n\" (UID: \"56e7bae5-9b23-4f6b-8084-20287730dd30\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-f297n" Jul 28 01:26:36.674816 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.674794 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/18e29850-170d-4a22-b890-95237b45246d-snapshots\") pod \"insights-operator-5885fdb759-q8ldl\" (UID: \"18e29850-170d-4a22-b890-95237b45246d\") " pod="openshift-insights/insights-operator-5885fdb759-q8ldl" Jul 28 01:26:36.675455 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.675435 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b4e84366-546a-4004-b614-6b3c35c61dfe-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-m7p9h\" (UID: \"b4e84366-546a-4004-b614-6b3c35c61dfe\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-m7p9h" Jul 28 01:26:36.675598 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.675583 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/18e29850-170d-4a22-b890-95237b45246d-serving-cert\") pod \"insights-operator-5885fdb759-q8ldl\" (UID: \"18e29850-170d-4a22-b890-95237b45246d\") " pod="openshift-insights/insights-operator-5885fdb759-q8ldl" Jul 28 01:26:36.682371 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.682348 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhqgn\" (UniqueName: \"kubernetes.io/projected/dbab9a89-f5fa-4a89-ba0a-d0ab2b4b8957-kube-api-access-qhqgn\") pod \"volume-data-source-validator-549c6d54cf-8fbhx\" (UID: \"dbab9a89-f5fa-4a89-ba0a-d0ab2b4b8957\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-8fbhx" Jul 28 01:26:36.682640 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.682619 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhwgf\" (UniqueName: \"kubernetes.io/projected/b4e84366-546a-4004-b614-6b3c35c61dfe-kube-api-access-nhwgf\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-m7p9h\" (UID: \"b4e84366-546a-4004-b614-6b3c35c61dfe\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-m7p9h" Jul 28 01:26:36.683091 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.683076 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2js5\" (UniqueName: \"kubernetes.io/projected/18e29850-170d-4a22-b890-95237b45246d-kube-api-access-m2js5\") pod \"insights-operator-5885fdb759-q8ldl\" (UID: \"18e29850-170d-4a22-b890-95237b45246d\") " pod="openshift-insights/insights-operator-5885fdb759-q8ldl" Jul 28 01:26:36.683231 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.683215 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l82k2\" (UniqueName: \"kubernetes.io/projected/56e7bae5-9b23-4f6b-8084-20287730dd30-kube-api-access-l82k2\") pod \"cluster-monitoring-operator-548b489cb8-f297n\" (UID: \"56e7bae5-9b23-4f6b-8084-20287730dd30\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-f297n" Jul 28 01:26:36.727884 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.727850 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-8fbhx" Jul 28 01:26:36.775513 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.775470 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56wdv\" (UniqueName: \"kubernetes.io/projected/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-kube-api-access-56wdv\") pod \"router-default-74577ddb85-mm25q\" (UID: \"923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71\") " pod="openshift-ingress/router-default-74577ddb85-mm25q" Jul 28 01:26:36.775759 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.775523 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3503116a-1398-42ea-ac86-743f5f51e989-trusted-ca\") pod \"console-operator-584dcb6b54-k2wlm\" (UID: \"3503116a-1398-42ea-ac86-743f5f51e989\") " pod="openshift-console-operator/console-operator-584dcb6b54-k2wlm" Jul 28 01:26:36.775759 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.775562 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2af2ea06-6e9c-4367-90ae-755b5f034806-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-t5dxh\" (UID: \"2af2ea06-6e9c-4367-90ae-755b5f034806\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-t5dxh" Jul 28 01:26:36.775759 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.775608 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/afab8f75-0a1a-43a3-9a98-be26e28ab36f-serving-cert\") pod \"service-ca-operator-5d6c75f84f-phhjx\" (UID: \"afab8f75-0a1a-43a3-9a98-be26e28ab36f\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-phhjx" Jul 28 01:26:36.775759 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:36.775739 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 28 01:26:36.775971 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:36.775819 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2af2ea06-6e9c-4367-90ae-755b5f034806-samples-operator-tls podName:2af2ea06-6e9c-4367-90ae-755b5f034806 nodeName:}" failed. No retries permitted until 2026-07-28 01:26:37.275795353 +0000 UTC m=+35.829920150 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2af2ea06-6e9c-4367-90ae-755b5f034806-samples-operator-tls") pod "cluster-samples-operator-6748489656-t5dxh" (UID: "2af2ea06-6e9c-4367-90ae-755b5f034806") : secret "samples-operator-tls" not found Jul 28 01:26:36.775971 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.775845 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v8gn\" (UniqueName: \"kubernetes.io/projected/5873b663-f99f-453e-8aaf-d57bf3fd0105-kube-api-access-4v8gn\") pod \"dns-default-n5dq2\" (UID: \"5873b663-f99f-453e-8aaf-d57bf3fd0105\") " pod="openshift-dns/dns-default-n5dq2" Jul 28 01:26:36.775971 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.775880 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtzrz\" (UniqueName: \"kubernetes.io/projected/3b34c5ff-3aed-4bba-990e-878c2de528ed-kube-api-access-gtzrz\") pod \"ingress-canary-sc92r\" (UID: \"3b34c5ff-3aed-4bba-990e-878c2de528ed\") " pod="openshift-ingress-canary/ingress-canary-sc92r" Jul 28 01:26:36.775971 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.775925 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1f2321ec-7f9d-4e33-9bed-d48fd717ca07-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-4pl5d\" (UID: \"1f2321ec-7f9d-4e33-9bed-d48fd717ca07\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4pl5d" Jul 28 01:26:36.775971 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.775956 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-default-certificate\") pod \"router-default-74577ddb85-mm25q\" (UID: \"923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71\") " pod="openshift-ingress/router-default-74577ddb85-mm25q" Jul 28 01:26:36.776224 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.775990 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5873b663-f99f-453e-8aaf-d57bf3fd0105-tmp-dir\") pod \"dns-default-n5dq2\" (UID: \"5873b663-f99f-453e-8aaf-d57bf3fd0105\") " pod="openshift-dns/dns-default-n5dq2" Jul 28 01:26:36.776224 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.776063 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j56rn\" (UniqueName: \"kubernetes.io/projected/2af2ea06-6e9c-4367-90ae-755b5f034806-kube-api-access-j56rn\") pod \"cluster-samples-operator-6748489656-t5dxh\" (UID: \"2af2ea06-6e9c-4367-90ae-755b5f034806\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-t5dxh" Jul 28 01:26:36.776224 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.776098 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-stats-auth\") pod \"router-default-74577ddb85-mm25q\" (UID: \"923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71\") " pod="openshift-ingress/router-default-74577ddb85-mm25q" Jul 28 01:26:36.776224 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.776125 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/afab8f75-0a1a-43a3-9a98-be26e28ab36f-config\") pod \"service-ca-operator-5d6c75f84f-phhjx\" (UID: \"afab8f75-0a1a-43a3-9a98-be26e28ab36f\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-phhjx" Jul 28 01:26:36.776224 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.776150 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmvws\" (UniqueName: \"kubernetes.io/projected/afab8f75-0a1a-43a3-9a98-be26e28ab36f-kube-api-access-wmvws\") pod \"service-ca-operator-5d6c75f84f-phhjx\" (UID: \"afab8f75-0a1a-43a3-9a98-be26e28ab36f\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-phhjx" Jul 28 01:26:36.776224 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.776189 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5873b663-f99f-453e-8aaf-d57bf3fd0105-config-volume\") pod \"dns-default-n5dq2\" (UID: \"5873b663-f99f-453e-8aaf-d57bf3fd0105\") " pod="openshift-dns/dns-default-n5dq2" Jul 28 01:26:36.776512 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.776227 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3b34c5ff-3aed-4bba-990e-878c2de528ed-cert\") pod \"ingress-canary-sc92r\" (UID: \"3b34c5ff-3aed-4bba-990e-878c2de528ed\") " pod="openshift-ingress-canary/ingress-canary-sc92r" Jul 28 01:26:36.776512 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.776259 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wftsm\" (UniqueName: \"kubernetes.io/projected/3503116a-1398-42ea-ac86-743f5f51e989-kube-api-access-wftsm\") pod \"console-operator-584dcb6b54-k2wlm\" (UID: \"3503116a-1398-42ea-ac86-743f5f51e989\") " pod="openshift-console-operator/console-operator-584dcb6b54-k2wlm" Jul 28 01:26:36.776512 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.776291 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-metrics-certs\") pod \"router-default-74577ddb85-mm25q\" (UID: \"923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71\") " pod="openshift-ingress/router-default-74577ddb85-mm25q" Jul 28 01:26:36.776512 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.776316 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3503116a-1398-42ea-ac86-743f5f51e989-config\") pod \"console-operator-584dcb6b54-k2wlm\" (UID: \"3503116a-1398-42ea-ac86-743f5f51e989\") " pod="openshift-console-operator/console-operator-584dcb6b54-k2wlm" Jul 28 01:26:36.776733 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.776692 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1f2321ec-7f9d-4e33-9bed-d48fd717ca07-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-4pl5d\" (UID: \"1f2321ec-7f9d-4e33-9bed-d48fd717ca07\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4pl5d" Jul 28 01:26:36.776787 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.776733 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3503116a-1398-42ea-ac86-743f5f51e989-trusted-ca\") pod \"console-operator-584dcb6b54-k2wlm\" (UID: \"3503116a-1398-42ea-ac86-743f5f51e989\") " pod="openshift-console-operator/console-operator-584dcb6b54-k2wlm" Jul 28 01:26:36.776849 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:36.776828 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 28 01:26:36.776905 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:36.776884 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-metrics-certs podName:923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71 nodeName:}" failed. No retries permitted until 2026-07-28 01:26:37.276867981 +0000 UTC m=+35.830992758 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-metrics-certs") pod "router-default-74577ddb85-mm25q" (UID: "923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71") : secret "router-metrics-certs-default" not found Jul 28 01:26:36.777590 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.776966 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3503116a-1398-42ea-ac86-743f5f51e989-config\") pod \"console-operator-584dcb6b54-k2wlm\" (UID: \"3503116a-1398-42ea-ac86-743f5f51e989\") " pod="openshift-console-operator/console-operator-584dcb6b54-k2wlm" Jul 28 01:26:36.777590 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.777131 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1f2321ec-7f9d-4e33-9bed-d48fd717ca07-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4pl5d\" (UID: \"1f2321ec-7f9d-4e33-9bed-d48fd717ca07\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4pl5d" Jul 28 01:26:36.777590 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.777214 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwdkf\" (UniqueName: \"kubernetes.io/projected/b91abb25-2132-46b6-b0b5-7f00d007fac3-kube-api-access-dwdkf\") pod \"network-check-source-7d88cb55d8-hrdd9\" (UID: \"b91abb25-2132-46b6-b0b5-7f00d007fac3\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-hrdd9" Jul 28 01:26:36.777590 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.777249 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5873b663-f99f-453e-8aaf-d57bf3fd0105-config-volume\") pod \"dns-default-n5dq2\" (UID: \"5873b663-f99f-453e-8aaf-d57bf3fd0105\") " pod="openshift-dns/dns-default-n5dq2" Jul 28 01:26:36.777590 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:36.777304 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 28 01:26:36.777590 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.777332 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5873b663-f99f-453e-8aaf-d57bf3fd0105-metrics-tls\") pod \"dns-default-n5dq2\" (UID: \"5873b663-f99f-453e-8aaf-d57bf3fd0105\") " pod="openshift-dns/dns-default-n5dq2" Jul 28 01:26:36.777590 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:36.777350 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f2321ec-7f9d-4e33-9bed-d48fd717ca07-networking-console-plugin-cert podName:1f2321ec-7f9d-4e33-9bed-d48fd717ca07 nodeName:}" failed. No retries permitted until 2026-07-28 01:26:37.277335544 +0000 UTC m=+35.831460334 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1f2321ec-7f9d-4e33-9bed-d48fd717ca07-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-4pl5d" (UID: "1f2321ec-7f9d-4e33-9bed-d48fd717ca07") : secret "networking-console-plugin-cert" not found Jul 28 01:26:36.777590 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.777301 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/afab8f75-0a1a-43a3-9a98-be26e28ab36f-config\") pod \"service-ca-operator-5d6c75f84f-phhjx\" (UID: \"afab8f75-0a1a-43a3-9a98-be26e28ab36f\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-phhjx" Jul 28 01:26:36.777590 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.777378 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3503116a-1398-42ea-ac86-743f5f51e989-serving-cert\") pod \"console-operator-584dcb6b54-k2wlm\" (UID: \"3503116a-1398-42ea-ac86-743f5f51e989\") " pod="openshift-console-operator/console-operator-584dcb6b54-k2wlm" Jul 28 01:26:36.777590 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:36.777389 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 01:26:36.777590 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.777457 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-service-ca-bundle\") pod \"router-default-74577ddb85-mm25q\" (UID: \"923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71\") " pod="openshift-ingress/router-default-74577ddb85-mm25q" Jul 28 01:26:36.777590 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:36.777529 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5873b663-f99f-453e-8aaf-d57bf3fd0105-metrics-tls podName:5873b663-f99f-453e-8aaf-d57bf3fd0105 nodeName:}" failed. No retries permitted until 2026-07-28 01:26:37.277510501 +0000 UTC m=+35.831635291 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5873b663-f99f-453e-8aaf-d57bf3fd0105-metrics-tls") pod "dns-default-n5dq2" (UID: "5873b663-f99f-453e-8aaf-d57bf3fd0105") : secret "dns-default-metrics-tls" not found Jul 28 01:26:36.778223 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:36.777617 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-service-ca-bundle podName:923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71 nodeName:}" failed. No retries permitted until 2026-07-28 01:26:37.277602229 +0000 UTC m=+35.831727232 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-service-ca-bundle") pod "router-default-74577ddb85-mm25q" (UID: "923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71") : configmap references non-existent config key: service-ca.crt Jul 28 01:26:36.778223 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.777635 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5873b663-f99f-453e-8aaf-d57bf3fd0105-tmp-dir\") pod \"dns-default-n5dq2\" (UID: \"5873b663-f99f-453e-8aaf-d57bf3fd0105\") " pod="openshift-dns/dns-default-n5dq2" Jul 28 01:26:36.778584 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.778564 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/afab8f75-0a1a-43a3-9a98-be26e28ab36f-serving-cert\") pod \"service-ca-operator-5d6c75f84f-phhjx\" (UID: \"afab8f75-0a1a-43a3-9a98-be26e28ab36f\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-phhjx" Jul 28 01:26:36.778929 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.778906 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-default-certificate\") pod \"router-default-74577ddb85-mm25q\" (UID: \"923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71\") " pod="openshift-ingress/router-default-74577ddb85-mm25q" Jul 28 01:26:36.780010 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.779987 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3503116a-1398-42ea-ac86-743f5f51e989-serving-cert\") pod \"console-operator-584dcb6b54-k2wlm\" (UID: \"3503116a-1398-42ea-ac86-743f5f51e989\") " pod="openshift-console-operator/console-operator-584dcb6b54-k2wlm" Jul 28 01:26:36.780103 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.780086 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-stats-auth\") pod \"router-default-74577ddb85-mm25q\" (UID: \"923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71\") " pod="openshift-ingress/router-default-74577ddb85-mm25q" Jul 28 01:26:36.785761 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.785741 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56wdv\" (UniqueName: \"kubernetes.io/projected/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-kube-api-access-56wdv\") pod \"router-default-74577ddb85-mm25q\" (UID: \"923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71\") " pod="openshift-ingress/router-default-74577ddb85-mm25q" Jul 28 01:26:36.786852 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.786827 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wftsm\" (UniqueName: \"kubernetes.io/projected/3503116a-1398-42ea-ac86-743f5f51e989-kube-api-access-wftsm\") pod \"console-operator-584dcb6b54-k2wlm\" (UID: \"3503116a-1398-42ea-ac86-743f5f51e989\") " pod="openshift-console-operator/console-operator-584dcb6b54-k2wlm" Jul 28 01:26:36.787195 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.787168 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j56rn\" (UniqueName: \"kubernetes.io/projected/2af2ea06-6e9c-4367-90ae-755b5f034806-kube-api-access-j56rn\") pod \"cluster-samples-operator-6748489656-t5dxh\" (UID: \"2af2ea06-6e9c-4367-90ae-755b5f034806\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-t5dxh" Jul 28 01:26:36.787600 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.787583 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v8gn\" (UniqueName: \"kubernetes.io/projected/5873b663-f99f-453e-8aaf-d57bf3fd0105-kube-api-access-4v8gn\") pod \"dns-default-n5dq2\" (UID: \"5873b663-f99f-453e-8aaf-d57bf3fd0105\") " pod="openshift-dns/dns-default-n5dq2" Jul 28 01:26:36.788184 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.788169 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmvws\" (UniqueName: \"kubernetes.io/projected/afab8f75-0a1a-43a3-9a98-be26e28ab36f-kube-api-access-wmvws\") pod \"service-ca-operator-5d6c75f84f-phhjx\" (UID: \"afab8f75-0a1a-43a3-9a98-be26e28ab36f\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-phhjx" Jul 28 01:26:36.795113 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.795094 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-q8ldl" Jul 28 01:26:36.800885 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.800865 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-m7p9h" Jul 28 01:26:36.843086 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.843007 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-phhjx" Jul 28 01:26:36.868954 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.868922 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-k2wlm" Jul 28 01:26:36.877999 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.877980 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwdkf\" (UniqueName: \"kubernetes.io/projected/b91abb25-2132-46b6-b0b5-7f00d007fac3-kube-api-access-dwdkf\") pod \"network-check-source-7d88cb55d8-hrdd9\" (UID: \"b91abb25-2132-46b6-b0b5-7f00d007fac3\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-hrdd9" Jul 28 01:26:36.878111 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.878085 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtzrz\" (UniqueName: \"kubernetes.io/projected/3b34c5ff-3aed-4bba-990e-878c2de528ed-kube-api-access-gtzrz\") pod \"ingress-canary-sc92r\" (UID: \"3b34c5ff-3aed-4bba-990e-878c2de528ed\") " pod="openshift-ingress-canary/ingress-canary-sc92r" Jul 28 01:26:36.878191 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.878169 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3b34c5ff-3aed-4bba-990e-878c2de528ed-cert\") pod \"ingress-canary-sc92r\" (UID: \"3b34c5ff-3aed-4bba-990e-878c2de528ed\") " pod="openshift-ingress-canary/ingress-canary-sc92r" Jul 28 01:26:36.880996 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.880975 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3b34c5ff-3aed-4bba-990e-878c2de528ed-cert\") pod \"ingress-canary-sc92r\" (UID: \"3b34c5ff-3aed-4bba-990e-878c2de528ed\") " pod="openshift-ingress-canary/ingress-canary-sc92r" Jul 28 01:26:36.898872 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.898845 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtzrz\" (UniqueName: \"kubernetes.io/projected/3b34c5ff-3aed-4bba-990e-878c2de528ed-kube-api-access-gtzrz\") pod \"ingress-canary-sc92r\" (UID: \"3b34c5ff-3aed-4bba-990e-878c2de528ed\") " pod="openshift-ingress-canary/ingress-canary-sc92r" Jul 28 01:26:36.905574 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.905534 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwdkf\" (UniqueName: \"kubernetes.io/projected/b91abb25-2132-46b6-b0b5-7f00d007fac3-kube-api-access-dwdkf\") pod \"network-check-source-7d88cb55d8-hrdd9\" (UID: \"b91abb25-2132-46b6-b0b5-7f00d007fac3\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-hrdd9" Jul 28 01:26:36.922610 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.922580 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-sc92r" Jul 28 01:26:36.941461 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:36.941431 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-hrdd9" Jul 28 01:26:37.004657 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:37.004627 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gfx9" Jul 28 01:26:37.004823 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:37.004633 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2znt8" Jul 28 01:26:37.004876 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:37.004636 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brftp" Jul 28 01:26:37.012868 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:37.012848 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 28 01:26:37.013079 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:37.013060 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 28 01:26:37.013156 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:37.013080 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cq5js\"" Jul 28 01:26:37.013231 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:37.013211 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lkkxh\"" Jul 28 01:26:37.080984 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:37.080944 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/31992241-f826-45d3-a26a-679d5d82465c-registry-tls\") pod \"image-registry-757879ff5c-x2xkc\" (UID: \"31992241-f826-45d3-a26a-679d5d82465c\") " pod="openshift-image-registry/image-registry-757879ff5c-x2xkc" Jul 28 01:26:37.081177 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:37.081104 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 01:26:37.081177 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:37.081129 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-757879ff5c-x2xkc: secret "image-registry-tls" not found Jul 28 01:26:37.081285 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:37.081208 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31992241-f826-45d3-a26a-679d5d82465c-registry-tls podName:31992241-f826-45d3-a26a-679d5d82465c nodeName:}" failed. No retries permitted until 2026-07-28 01:26:38.081187617 +0000 UTC m=+36.635312407 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/31992241-f826-45d3-a26a-679d5d82465c-registry-tls") pod "image-registry-757879ff5c-x2xkc" (UID: "31992241-f826-45d3-a26a-679d5d82465c") : secret "image-registry-tls" not found Jul 28 01:26:37.182487 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:37.182410 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/56e7bae5-9b23-4f6b-8084-20287730dd30-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-f297n\" (UID: \"56e7bae5-9b23-4f6b-8084-20287730dd30\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-f297n" Jul 28 01:26:37.182694 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:37.182675 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 28 01:26:37.182774 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:37.182760 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56e7bae5-9b23-4f6b-8084-20287730dd30-cluster-monitoring-operator-tls podName:56e7bae5-9b23-4f6b-8084-20287730dd30 nodeName:}" failed. No retries permitted until 2026-07-28 01:26:38.182741572 +0000 UTC m=+36.736866350 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/56e7bae5-9b23-4f6b-8084-20287730dd30-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-f297n" (UID: "56e7bae5-9b23-4f6b-8084-20287730dd30") : secret "cluster-monitoring-operator-tls" not found Jul 28 01:26:37.283774 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:37.283743 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-service-ca-bundle\") pod \"router-default-74577ddb85-mm25q\" (UID: \"923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71\") " pod="openshift-ingress/router-default-74577ddb85-mm25q" Jul 28 01:26:37.283954 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:37.283793 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2af2ea06-6e9c-4367-90ae-755b5f034806-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-t5dxh\" (UID: \"2af2ea06-6e9c-4367-90ae-755b5f034806\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-t5dxh" Jul 28 01:26:37.283954 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:37.283887 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 28 01:26:37.283954 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:37.283921 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-service-ca-bundle podName:923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71 nodeName:}" failed. No retries permitted until 2026-07-28 01:26:38.283899498 +0000 UTC m=+36.838024275 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-service-ca-bundle") pod "router-default-74577ddb85-mm25q" (UID: "923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71") : configmap references non-existent config key: service-ca.crt Jul 28 01:26:37.283954 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:37.283953 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2af2ea06-6e9c-4367-90ae-755b5f034806-samples-operator-tls podName:2af2ea06-6e9c-4367-90ae-755b5f034806 nodeName:}" failed. No retries permitted until 2026-07-28 01:26:38.283944569 +0000 UTC m=+36.838069353 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2af2ea06-6e9c-4367-90ae-755b5f034806-samples-operator-tls") pod "cluster-samples-operator-6748489656-t5dxh" (UID: "2af2ea06-6e9c-4367-90ae-755b5f034806") : secret "samples-operator-tls" not found Jul 28 01:26:37.284196 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:37.284077 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-metrics-certs\") pod \"router-default-74577ddb85-mm25q\" (UID: \"923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71\") " pod="openshift-ingress/router-default-74577ddb85-mm25q" Jul 28 01:26:37.284196 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:37.284124 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1f2321ec-7f9d-4e33-9bed-d48fd717ca07-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4pl5d\" (UID: \"1f2321ec-7f9d-4e33-9bed-d48fd717ca07\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4pl5d" Jul 28 01:26:37.284289 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:37.284206 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 28 01:26:37.284289 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:37.284219 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 28 01:26:37.284289 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:37.284250 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-metrics-certs podName:923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71 nodeName:}" failed. No retries permitted until 2026-07-28 01:26:38.284235819 +0000 UTC m=+36.838360617 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-metrics-certs") pod "router-default-74577ddb85-mm25q" (UID: "923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71") : secret "router-metrics-certs-default" not found Jul 28 01:26:37.284289 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:37.284280 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f2321ec-7f9d-4e33-9bed-d48fd717ca07-networking-console-plugin-cert podName:1f2321ec-7f9d-4e33-9bed-d48fd717ca07 nodeName:}" failed. No retries permitted until 2026-07-28 01:26:38.28426649 +0000 UTC m=+36.838391267 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1f2321ec-7f9d-4e33-9bed-d48fd717ca07-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-4pl5d" (UID: "1f2321ec-7f9d-4e33-9bed-d48fd717ca07") : secret "networking-console-plugin-cert" not found Jul 28 01:26:37.284436 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:37.284274 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5873b663-f99f-453e-8aaf-d57bf3fd0105-metrics-tls\") pod \"dns-default-n5dq2\" (UID: \"5873b663-f99f-453e-8aaf-d57bf3fd0105\") " pod="openshift-dns/dns-default-n5dq2" Jul 28 01:26:37.284436 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:37.284337 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 01:26:37.284436 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:37.284382 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5873b663-f99f-453e-8aaf-d57bf3fd0105-metrics-tls podName:5873b663-f99f-453e-8aaf-d57bf3fd0105 nodeName:}" failed. No retries permitted until 2026-07-28 01:26:38.284368268 +0000 UTC m=+36.838493047 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5873b663-f99f-453e-8aaf-d57bf3fd0105-metrics-tls") pod "dns-default-n5dq2" (UID: "5873b663-f99f-453e-8aaf-d57bf3fd0105") : secret "dns-default-metrics-tls" not found Jul 28 01:26:38.090459 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:38.090420 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/31992241-f826-45d3-a26a-679d5d82465c-registry-tls\") pod \"image-registry-757879ff5c-x2xkc\" (UID: \"31992241-f826-45d3-a26a-679d5d82465c\") " pod="openshift-image-registry/image-registry-757879ff5c-x2xkc" Jul 28 01:26:38.091078 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:38.090591 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 01:26:38.091078 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:38.090616 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-757879ff5c-x2xkc: secret "image-registry-tls" not found Jul 28 01:26:38.091078 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:38.090679 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31992241-f826-45d3-a26a-679d5d82465c-registry-tls podName:31992241-f826-45d3-a26a-679d5d82465c nodeName:}" failed. No retries permitted until 2026-07-28 01:26:40.09066289 +0000 UTC m=+38.644787665 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/31992241-f826-45d3-a26a-679d5d82465c-registry-tls") pod "image-registry-757879ff5c-x2xkc" (UID: "31992241-f826-45d3-a26a-679d5d82465c") : secret "image-registry-tls" not found Jul 28 01:26:38.191681 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:38.191647 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/56e7bae5-9b23-4f6b-8084-20287730dd30-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-f297n\" (UID: \"56e7bae5-9b23-4f6b-8084-20287730dd30\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-f297n" Jul 28 01:26:38.191866 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:38.191806 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 28 01:26:38.191926 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:38.191874 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56e7bae5-9b23-4f6b-8084-20287730dd30-cluster-monitoring-operator-tls podName:56e7bae5-9b23-4f6b-8084-20287730dd30 nodeName:}" failed. No retries permitted until 2026-07-28 01:26:40.191858727 +0000 UTC m=+38.745983517 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/56e7bae5-9b23-4f6b-8084-20287730dd30-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-f297n" (UID: "56e7bae5-9b23-4f6b-8084-20287730dd30") : secret "cluster-monitoring-operator-tls" not found Jul 28 01:26:38.294374 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:38.292642 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1f2321ec-7f9d-4e33-9bed-d48fd717ca07-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4pl5d\" (UID: \"1f2321ec-7f9d-4e33-9bed-d48fd717ca07\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4pl5d" Jul 28 01:26:38.294374 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:38.292914 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5873b663-f99f-453e-8aaf-d57bf3fd0105-metrics-tls\") pod \"dns-default-n5dq2\" (UID: \"5873b663-f99f-453e-8aaf-d57bf3fd0105\") " pod="openshift-dns/dns-default-n5dq2" Jul 28 01:26:38.294374 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:38.292958 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-service-ca-bundle\") pod \"router-default-74577ddb85-mm25q\" (UID: \"923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71\") " pod="openshift-ingress/router-default-74577ddb85-mm25q" Jul 28 01:26:38.294374 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:38.293002 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2af2ea06-6e9c-4367-90ae-755b5f034806-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-t5dxh\" (UID: \"2af2ea06-6e9c-4367-90ae-755b5f034806\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-t5dxh" Jul 28 01:26:38.294374 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:38.293121 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-metrics-certs\") pod \"router-default-74577ddb85-mm25q\" (UID: \"923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71\") " pod="openshift-ingress/router-default-74577ddb85-mm25q" Jul 28 01:26:38.294374 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:38.293258 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 28 01:26:38.294374 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:38.293318 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-metrics-certs podName:923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71 nodeName:}" failed. No retries permitted until 2026-07-28 01:26:40.293300411 +0000 UTC m=+38.847425203 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-metrics-certs") pod "router-default-74577ddb85-mm25q" (UID: "923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71") : secret "router-metrics-certs-default" not found Jul 28 01:26:38.294374 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:38.293397 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 28 01:26:38.294374 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:38.293426 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f2321ec-7f9d-4e33-9bed-d48fd717ca07-networking-console-plugin-cert podName:1f2321ec-7f9d-4e33-9bed-d48fd717ca07 nodeName:}" failed. No retries permitted until 2026-07-28 01:26:40.293416498 +0000 UTC m=+38.847541288 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1f2321ec-7f9d-4e33-9bed-d48fd717ca07-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-4pl5d" (UID: "1f2321ec-7f9d-4e33-9bed-d48fd717ca07") : secret "networking-console-plugin-cert" not found Jul 28 01:26:38.294374 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:38.293480 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 01:26:38.294374 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:38.293505 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5873b663-f99f-453e-8aaf-d57bf3fd0105-metrics-tls podName:5873b663-f99f-453e-8aaf-d57bf3fd0105 nodeName:}" failed. No retries permitted until 2026-07-28 01:26:40.293496128 +0000 UTC m=+38.847620906 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5873b663-f99f-453e-8aaf-d57bf3fd0105-metrics-tls") pod "dns-default-n5dq2" (UID: "5873b663-f99f-453e-8aaf-d57bf3fd0105") : secret "dns-default-metrics-tls" not found Jul 28 01:26:38.294374 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:38.293653 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 28 01:26:38.294374 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:38.293694 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2af2ea06-6e9c-4367-90ae-755b5f034806-samples-operator-tls podName:2af2ea06-6e9c-4367-90ae-755b5f034806 nodeName:}" failed. No retries permitted until 2026-07-28 01:26:40.293682492 +0000 UTC m=+38.847807283 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2af2ea06-6e9c-4367-90ae-755b5f034806-samples-operator-tls") pod "cluster-samples-operator-6748489656-t5dxh" (UID: "2af2ea06-6e9c-4367-90ae-755b5f034806") : secret "samples-operator-tls" not found Jul 28 01:26:38.294374 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:38.293749 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-service-ca-bundle podName:923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71 nodeName:}" failed. No retries permitted until 2026-07-28 01:26:40.293740407 +0000 UTC m=+38.847865185 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-service-ca-bundle") pod "router-default-74577ddb85-mm25q" (UID: "923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71") : configmap references non-existent config key: service-ca.crt Jul 28 01:26:38.454603 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:38.454488 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-sc92r"] Jul 28 01:26:38.457641 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:38.457614 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-8fbhx"] Jul 28 01:26:38.458839 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:38.458804 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-m7p9h"] Jul 28 01:26:38.460220 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:38.460165 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-k2wlm"] Jul 28 01:26:38.465214 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:38.465195 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-hrdd9"] Jul 28 01:26:38.477709 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:38.477680 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-q8ldl"] Jul 28 01:26:38.480707 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:38.480685 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-phhjx"] Jul 28 01:26:38.484414 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:38.484386 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18e29850_170d_4a22_b890_95237b45246d.slice/crio-4bb0730563e1ff627db13319a7d2776c160304847a423af0fe88723e1f6da34b WatchSource:0}: Error finding container 4bb0730563e1ff627db13319a7d2776c160304847a423af0fe88723e1f6da34b: Status 404 returned error can't find the container with id 4bb0730563e1ff627db13319a7d2776c160304847a423af0fe88723e1f6da34b Jul 28 01:26:38.485141 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:38.485115 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafab8f75_0a1a_43a3_9a98_be26e28ab36f.slice/crio-e18f5a4264a47484b2c8134a1faeb03b85fb9701a71fb256dad60942276537f6 WatchSource:0}: Error finding container e18f5a4264a47484b2c8134a1faeb03b85fb9701a71fb256dad60942276537f6: Status 404 returned error can't find the container with id e18f5a4264a47484b2c8134a1faeb03b85fb9701a71fb256dad60942276537f6 Jul 28 01:26:38.485954 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:38.485837 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbab9a89_f5fa_4a89_ba0a_d0ab2b4b8957.slice/crio-190df4999f3ad9d475534a654a2e40cbc4e8638b4da8362509ab52fbf1319ecc WatchSource:0}: Error finding container 190df4999f3ad9d475534a654a2e40cbc4e8638b4da8362509ab52fbf1319ecc: Status 404 returned error can't find the container with id 190df4999f3ad9d475534a654a2e40cbc4e8638b4da8362509ab52fbf1319ecc Jul 28 01:26:38.486892 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:38.486653 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b34c5ff_3aed_4bba_990e_878c2de528ed.slice/crio-89e7c12516286c34e21bf4f2cc80efba5f11bf3b59e5b68bdfb2ea60353e455f WatchSource:0}: Error finding container 89e7c12516286c34e21bf4f2cc80efba5f11bf3b59e5b68bdfb2ea60353e455f: Status 404 returned error can't find the container with id 89e7c12516286c34e21bf4f2cc80efba5f11bf3b59e5b68bdfb2ea60353e455f Jul 28 01:26:38.488443 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:38.488305 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb91abb25_2132_46b6_b0b5_7f00d007fac3.slice/crio-4c085ffc8d71bd8c25bfa3dbd8f34aa69ad3e4e6c2cd252a83e0b85a4301614e WatchSource:0}: Error finding container 4c085ffc8d71bd8c25bfa3dbd8f34aa69ad3e4e6c2cd252a83e0b85a4301614e: Status 404 returned error can't find the container with id 4c085ffc8d71bd8c25bfa3dbd8f34aa69ad3e4e6c2cd252a83e0b85a4301614e Jul 28 01:26:38.490322 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:26:38.489855 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4e84366_546a_4004_b614_6b3c35c61dfe.slice/crio-c455d7d97a6b21af3bf69b0f09ed9813ea860293607966186fa53448a1ba73c6 WatchSource:0}: Error finding container c455d7d97a6b21af3bf69b0f09ed9813ea860293607966186fa53448a1ba73c6: Status 404 returned error can't find the container with id c455d7d97a6b21af3bf69b0f09ed9813ea860293607966186fa53448a1ba73c6 Jul 28 01:26:39.180840 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:39.180710 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-k2wlm" event={"ID":"3503116a-1398-42ea-ac86-743f5f51e989","Type":"ContainerStarted","Data":"49a50742a1666ed01cf74412f07c4baf0d685de81ad52bdd11a7b698a86665e2"} Jul 28 01:26:39.183778 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:39.183653 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-sc92r" event={"ID":"3b34c5ff-3aed-4bba-990e-878c2de528ed","Type":"ContainerStarted","Data":"89e7c12516286c34e21bf4f2cc80efba5f11bf3b59e5b68bdfb2ea60353e455f"} Jul 28 01:26:39.186121 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:39.186044 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-q8ldl" event={"ID":"18e29850-170d-4a22-b890-95237b45246d","Type":"ContainerStarted","Data":"4bb0730563e1ff627db13319a7d2776c160304847a423af0fe88723e1f6da34b"} Jul 28 01:26:39.195134 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:39.195036 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-8fbhx" event={"ID":"dbab9a89-f5fa-4a89-ba0a-d0ab2b4b8957","Type":"ContainerStarted","Data":"190df4999f3ad9d475534a654a2e40cbc4e8638b4da8362509ab52fbf1319ecc"} Jul 28 01:26:39.199129 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:39.199051 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-m7p9h" event={"ID":"b4e84366-546a-4004-b614-6b3c35c61dfe","Type":"ContainerStarted","Data":"c455d7d97a6b21af3bf69b0f09ed9813ea860293607966186fa53448a1ba73c6"} Jul 28 01:26:39.201034 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:39.200975 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-hrdd9" event={"ID":"b91abb25-2132-46b6-b0b5-7f00d007fac3","Type":"ContainerStarted","Data":"4c085ffc8d71bd8c25bfa3dbd8f34aa69ad3e4e6c2cd252a83e0b85a4301614e"} Jul 28 01:26:39.210924 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:39.209981 2578 generic.go:356] "Generic (PLEG): container finished" podID="3394f81c-6c57-451b-8d8e-2b6504f34ed3" containerID="5de4af77b0d71c37159820048268528bc7a0ef71141a4064c2b304a736bc854e" exitCode=0 Jul 28 01:26:39.210924 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:39.210064 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wgdbs" event={"ID":"3394f81c-6c57-451b-8d8e-2b6504f34ed3","Type":"ContainerDied","Data":"5de4af77b0d71c37159820048268528bc7a0ef71141a4064c2b304a736bc854e"} Jul 28 01:26:39.218089 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:39.218008 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-phhjx" event={"ID":"afab8f75-0a1a-43a3-9a98-be26e28ab36f","Type":"ContainerStarted","Data":"e18f5a4264a47484b2c8134a1faeb03b85fb9701a71fb256dad60942276537f6"} Jul 28 01:26:40.114441 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:40.114269 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/31992241-f826-45d3-a26a-679d5d82465c-registry-tls\") pod \"image-registry-757879ff5c-x2xkc\" (UID: \"31992241-f826-45d3-a26a-679d5d82465c\") " pod="openshift-image-registry/image-registry-757879ff5c-x2xkc" Jul 28 01:26:40.114571 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:40.114507 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 01:26:40.114571 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:40.114523 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-757879ff5c-x2xkc: secret "image-registry-tls" not found Jul 28 01:26:40.114684 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:40.114594 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31992241-f826-45d3-a26a-679d5d82465c-registry-tls podName:31992241-f826-45d3-a26a-679d5d82465c nodeName:}" failed. No retries permitted until 2026-07-28 01:26:44.11457511 +0000 UTC m=+42.668699891 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/31992241-f826-45d3-a26a-679d5d82465c-registry-tls") pod "image-registry-757879ff5c-x2xkc" (UID: "31992241-f826-45d3-a26a-679d5d82465c") : secret "image-registry-tls" not found Jul 28 01:26:40.217859 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:40.217626 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/56e7bae5-9b23-4f6b-8084-20287730dd30-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-f297n\" (UID: \"56e7bae5-9b23-4f6b-8084-20287730dd30\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-f297n" Jul 28 01:26:40.218299 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:40.218055 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 28 01:26:40.218299 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:40.218122 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56e7bae5-9b23-4f6b-8084-20287730dd30-cluster-monitoring-operator-tls podName:56e7bae5-9b23-4f6b-8084-20287730dd30 nodeName:}" failed. No retries permitted until 2026-07-28 01:26:44.218101855 +0000 UTC m=+42.772226643 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/56e7bae5-9b23-4f6b-8084-20287730dd30-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-f297n" (UID: "56e7bae5-9b23-4f6b-8084-20287730dd30") : secret "cluster-monitoring-operator-tls" not found Jul 28 01:26:40.226289 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:40.225274 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-8fbhx" event={"ID":"dbab9a89-f5fa-4a89-ba0a-d0ab2b4b8957","Type":"ContainerStarted","Data":"7fb6f55b4332bfb084a3ba3534894a8c69bbdb463d754d93c417d214cdde9c2b"} Jul 28 01:26:40.246459 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:40.245404 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-8fbhx" podStartSLOduration=15.676141972 podStartE2EDuration="17.245383355s" totalImagesPullingTime="1.569241383s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:26:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:26:40.244637437 +0000 UTC m=+38.798762236" watchObservedRunningTime="2026-07-28 01:26:40.245383355 +0000 UTC m=+38.799508149" Jul 28 01:26:40.246459 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:40.246149 2578 generic.go:356] "Generic (PLEG): container finished" podID="3394f81c-6c57-451b-8d8e-2b6504f34ed3" containerID="4de82c6a792c5265998562f917387d16ed21db1eeaf7b88dee06f2f78d79aba3" exitCode=0 Jul 28 01:26:40.246459 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:40.246213 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wgdbs" event={"ID":"3394f81c-6c57-451b-8d8e-2b6504f34ed3","Type":"ContainerDied","Data":"4de82c6a792c5265998562f917387d16ed21db1eeaf7b88dee06f2f78d79aba3"} Jul 28 01:26:40.319031 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:40.318985 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-metrics-certs\") pod \"router-default-74577ddb85-mm25q\" (UID: \"923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71\") " pod="openshift-ingress/router-default-74577ddb85-mm25q" Jul 28 01:26:40.319217 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:40.319050 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1f2321ec-7f9d-4e33-9bed-d48fd717ca07-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4pl5d\" (UID: \"1f2321ec-7f9d-4e33-9bed-d48fd717ca07\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4pl5d" Jul 28 01:26:40.319217 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:40.319087 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5873b663-f99f-453e-8aaf-d57bf3fd0105-metrics-tls\") pod \"dns-default-n5dq2\" (UID: \"5873b663-f99f-453e-8aaf-d57bf3fd0105\") " pod="openshift-dns/dns-default-n5dq2" Jul 28 01:26:40.319217 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:40.319124 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-service-ca-bundle\") pod \"router-default-74577ddb85-mm25q\" (UID: \"923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71\") " pod="openshift-ingress/router-default-74577ddb85-mm25q" Jul 28 01:26:40.319217 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:40.319166 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2af2ea06-6e9c-4367-90ae-755b5f034806-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-t5dxh\" (UID: \"2af2ea06-6e9c-4367-90ae-755b5f034806\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-t5dxh" Jul 28 01:26:40.319437 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:40.319350 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 28 01:26:40.319437 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:40.319414 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2af2ea06-6e9c-4367-90ae-755b5f034806-samples-operator-tls podName:2af2ea06-6e9c-4367-90ae-755b5f034806 nodeName:}" failed. No retries permitted until 2026-07-28 01:26:44.31939435 +0000 UTC m=+42.873519128 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2af2ea06-6e9c-4367-90ae-755b5f034806-samples-operator-tls") pod "cluster-samples-operator-6748489656-t5dxh" (UID: "2af2ea06-6e9c-4367-90ae-755b5f034806") : secret "samples-operator-tls" not found Jul 28 01:26:40.320230 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:40.319841 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 28 01:26:40.320230 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:40.319890 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-metrics-certs podName:923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71 nodeName:}" failed. No retries permitted until 2026-07-28 01:26:44.319876506 +0000 UTC m=+42.874001282 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-metrics-certs") pod "router-default-74577ddb85-mm25q" (UID: "923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71") : secret "router-metrics-certs-default" not found Jul 28 01:26:40.320230 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:40.319942 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 28 01:26:40.320230 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:40.319977 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f2321ec-7f9d-4e33-9bed-d48fd717ca07-networking-console-plugin-cert podName:1f2321ec-7f9d-4e33-9bed-d48fd717ca07 nodeName:}" failed. No retries permitted until 2026-07-28 01:26:44.319965071 +0000 UTC m=+42.874089863 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1f2321ec-7f9d-4e33-9bed-d48fd717ca07-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-4pl5d" (UID: "1f2321ec-7f9d-4e33-9bed-d48fd717ca07") : secret "networking-console-plugin-cert" not found Jul 28 01:26:40.320230 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:40.320031 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 01:26:40.320230 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:40.320059 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5873b663-f99f-453e-8aaf-d57bf3fd0105-metrics-tls podName:5873b663-f99f-453e-8aaf-d57bf3fd0105 nodeName:}" failed. No retries permitted until 2026-07-28 01:26:44.320049886 +0000 UTC m=+42.874174677 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5873b663-f99f-453e-8aaf-d57bf3fd0105-metrics-tls") pod "dns-default-n5dq2" (UID: "5873b663-f99f-453e-8aaf-d57bf3fd0105") : secret "dns-default-metrics-tls" not found Jul 28 01:26:40.320230 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:40.320122 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-service-ca-bundle podName:923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71 nodeName:}" failed. No retries permitted until 2026-07-28 01:26:44.320114187 +0000 UTC m=+42.874238978 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-service-ca-bundle") pod "router-default-74577ddb85-mm25q" (UID: "923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71") : configmap references non-existent config key: service-ca.crt Jul 28 01:26:41.254253 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:41.254067 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wgdbs" event={"ID":"3394f81c-6c57-451b-8d8e-2b6504f34ed3","Type":"ContainerStarted","Data":"57357ddf130cf4711abe1e75b4bd0ef76fa1c463619fd1dfc7d7e0f59f4e1f8a"} Jul 28 01:26:41.279075 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:41.279020 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wgdbs" podStartSLOduration=8.704772232 podStartE2EDuration="39.27900053s" totalImagesPullingTime="30.574228298s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 01:26:02 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:26:41.277464016 +0000 UTC m=+39.831588814" watchObservedRunningTime="2026-07-28 01:26:41.27900053 +0000 UTC m=+39.833125328" Jul 28 01:26:44.155657 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:44.155615 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/31992241-f826-45d3-a26a-679d5d82465c-registry-tls\") pod \"image-registry-757879ff5c-x2xkc\" (UID: \"31992241-f826-45d3-a26a-679d5d82465c\") " pod="openshift-image-registry/image-registry-757879ff5c-x2xkc" Jul 28 01:26:44.156123 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:44.155737 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 01:26:44.156123 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:44.155748 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-757879ff5c-x2xkc: secret "image-registry-tls" not found Jul 28 01:26:44.156123 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:44.155797 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31992241-f826-45d3-a26a-679d5d82465c-registry-tls podName:31992241-f826-45d3-a26a-679d5d82465c nodeName:}" failed. No retries permitted until 2026-07-28 01:26:52.155783752 +0000 UTC m=+50.709908541 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/31992241-f826-45d3-a26a-679d5d82465c-registry-tls") pod "image-registry-757879ff5c-x2xkc" (UID: "31992241-f826-45d3-a26a-679d5d82465c") : secret "image-registry-tls" not found Jul 28 01:26:44.256753 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:44.256708 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/56e7bae5-9b23-4f6b-8084-20287730dd30-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-f297n\" (UID: \"56e7bae5-9b23-4f6b-8084-20287730dd30\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-f297n" Jul 28 01:26:44.256950 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:44.256881 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 28 01:26:44.257012 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:44.256955 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56e7bae5-9b23-4f6b-8084-20287730dd30-cluster-monitoring-operator-tls podName:56e7bae5-9b23-4f6b-8084-20287730dd30 nodeName:}" failed. No retries permitted until 2026-07-28 01:26:52.256937475 +0000 UTC m=+50.811062251 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/56e7bae5-9b23-4f6b-8084-20287730dd30-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-f297n" (UID: "56e7bae5-9b23-4f6b-8084-20287730dd30") : secret "cluster-monitoring-operator-tls" not found Jul 28 01:26:44.357580 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:44.357525 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2af2ea06-6e9c-4367-90ae-755b5f034806-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-t5dxh\" (UID: \"2af2ea06-6e9c-4367-90ae-755b5f034806\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-t5dxh" Jul 28 01:26:44.357736 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:44.357655 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 28 01:26:44.357736 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:44.357680 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-metrics-certs\") pod \"router-default-74577ddb85-mm25q\" (UID: \"923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71\") " pod="openshift-ingress/router-default-74577ddb85-mm25q" Jul 28 01:26:44.357736 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:44.357723 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2af2ea06-6e9c-4367-90ae-755b5f034806-samples-operator-tls podName:2af2ea06-6e9c-4367-90ae-755b5f034806 nodeName:}" failed. No retries permitted until 2026-07-28 01:26:52.357705125 +0000 UTC m=+50.911829923 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2af2ea06-6e9c-4367-90ae-755b5f034806-samples-operator-tls") pod "cluster-samples-operator-6748489656-t5dxh" (UID: "2af2ea06-6e9c-4367-90ae-755b5f034806") : secret "samples-operator-tls" not found Jul 28 01:26:44.357857 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:44.357744 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1f2321ec-7f9d-4e33-9bed-d48fd717ca07-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4pl5d\" (UID: \"1f2321ec-7f9d-4e33-9bed-d48fd717ca07\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4pl5d" Jul 28 01:26:44.357857 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:44.357772 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 28 01:26:44.357857 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:44.357807 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-metrics-certs podName:923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71 nodeName:}" failed. No retries permitted until 2026-07-28 01:26:52.357795634 +0000 UTC m=+50.911920417 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-metrics-certs") pod "router-default-74577ddb85-mm25q" (UID: "923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71") : secret "router-metrics-certs-default" not found Jul 28 01:26:44.357857 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:44.357825 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5873b663-f99f-453e-8aaf-d57bf3fd0105-metrics-tls\") pod \"dns-default-n5dq2\" (UID: \"5873b663-f99f-453e-8aaf-d57bf3fd0105\") " pod="openshift-dns/dns-default-n5dq2" Jul 28 01:26:44.357857 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:44.357853 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 28 01:26:44.358021 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:44.357864 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-service-ca-bundle\") pod \"router-default-74577ddb85-mm25q\" (UID: \"923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71\") " pod="openshift-ingress/router-default-74577ddb85-mm25q" Jul 28 01:26:44.358021 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:44.357887 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f2321ec-7f9d-4e33-9bed-d48fd717ca07-networking-console-plugin-cert podName:1f2321ec-7f9d-4e33-9bed-d48fd717ca07 nodeName:}" failed. No retries permitted until 2026-07-28 01:26:52.357876196 +0000 UTC m=+50.912000972 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1f2321ec-7f9d-4e33-9bed-d48fd717ca07-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-4pl5d" (UID: "1f2321ec-7f9d-4e33-9bed-d48fd717ca07") : secret "networking-console-plugin-cert" not found Jul 28 01:26:44.358021 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:44.357998 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 01:26:44.358140 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:44.358041 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-service-ca-bundle podName:923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71 nodeName:}" failed. No retries permitted until 2026-07-28 01:26:52.358033426 +0000 UTC m=+50.912158205 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-service-ca-bundle") pod "router-default-74577ddb85-mm25q" (UID: "923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71") : configmap references non-existent config key: service-ca.crt Jul 28 01:26:44.358140 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:44.358054 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5873b663-f99f-453e-8aaf-d57bf3fd0105-metrics-tls podName:5873b663-f99f-453e-8aaf-d57bf3fd0105 nodeName:}" failed. No retries permitted until 2026-07-28 01:26:52.358048245 +0000 UTC m=+50.912173021 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5873b663-f99f-453e-8aaf-d57bf3fd0105-metrics-tls") pod "dns-default-n5dq2" (UID: "5873b663-f99f-453e-8aaf-d57bf3fd0105") : secret "dns-default-metrics-tls" not found Jul 28 01:26:46.282991 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:46.282157 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-k2wlm" event={"ID":"3503116a-1398-42ea-ac86-743f5f51e989","Type":"ContainerStarted","Data":"64862c220089dbfc63640aa7ab30d69c0f48c9028d57e4c84aaec802ddbe0051"} Jul 28 01:26:46.282991 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:46.282944 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-k2wlm" Jul 28 01:26:46.284254 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:46.284139 2578 patch_prober.go:28] interesting pod/console-operator-584dcb6b54-k2wlm container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.14:8443/readyz\": dial tcp 10.132.0.14:8443: connect: connection refused" start-of-body= Jul 28 01:26:46.284254 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:46.284199 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-584dcb6b54-k2wlm" podUID="3503116a-1398-42ea-ac86-743f5f51e989" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.14:8443/readyz\": dial tcp 10.132.0.14:8443: connect: connection refused" Jul 28 01:26:46.286515 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:46.286062 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-sc92r" event={"ID":"3b34c5ff-3aed-4bba-990e-878c2de528ed","Type":"ContainerStarted","Data":"663458520d1a70633bf857f12e60446e059ec06624ddf7cc4e591d81c36eae8b"} Jul 28 01:26:46.289337 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:46.288947 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-q8ldl" event={"ID":"18e29850-170d-4a22-b890-95237b45246d","Type":"ContainerStarted","Data":"85115ddad03ce821d6c8694458d557811d29c5b8b52af002330528ccc3ad7c5a"} Jul 28 01:26:46.291297 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:46.291273 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-m7p9h" event={"ID":"b4e84366-546a-4004-b614-6b3c35c61dfe","Type":"ContainerStarted","Data":"d696346e26724c48f017cca5624800af92888ee5cfb70e030136efd63d2c606c"} Jul 28 01:26:46.296276 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:46.295828 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-phhjx" event={"ID":"afab8f75-0a1a-43a3-9a98-be26e28ab36f","Type":"ContainerStarted","Data":"7fc70f009a95d7ad0140b60eecbb66f6b6082374780112020192a807031ca676"} Jul 28 01:26:46.302594 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:46.300232 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-k2wlm" podStartSLOduration=15.696530021000001 podStartE2EDuration="23.300212978s" totalImagesPullingTime="7.603682957s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:26:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:26:46.300157788 +0000 UTC m=+44.854282587" watchObservedRunningTime="2026-07-28 01:26:46.300212978 +0000 UTC m=+44.854337778" Jul 28 01:26:46.318338 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:46.318280 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-phhjx" podStartSLOduration=15.710028284 podStartE2EDuration="23.318259151s" totalImagesPullingTime="7.608230867s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:26:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:26:46.317991578 +0000 UTC m=+44.872116377" watchObservedRunningTime="2026-07-28 01:26:46.318259151 +0000 UTC m=+44.872383951" Jul 28 01:26:46.340074 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:46.339343 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-q8ldl" podStartSLOduration=15.730633138 podStartE2EDuration="23.339322267s" totalImagesPullingTime="7.608689129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:26:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:26:46.337941785 +0000 UTC m=+44.892066585" watchObservedRunningTime="2026-07-28 01:26:46.339322267 +0000 UTC m=+44.893447066" Jul 28 01:26:46.382430 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:46.380924 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-m7p9h" podStartSLOduration=15.776739248 podStartE2EDuration="23.380903235s" totalImagesPullingTime="7.604163987s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:26:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:26:46.359803407 +0000 UTC m=+44.913928203" watchObservedRunningTime="2026-07-28 01:26:46.380903235 +0000 UTC m=+44.935028040" Jul 28 01:26:46.405961 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:46.404439 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-sc92r" podStartSLOduration=2.798309287 podStartE2EDuration="10.404417155s" totalImagesPullingTime="7.606107868s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:26:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:26:46.383041069 +0000 UTC m=+44.937165868" watchObservedRunningTime="2026-07-28 01:26:46.404417155 +0000 UTC m=+44.958541958" Jul 28 01:26:46.407163 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:46.406765 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-hrdd9" podStartSLOduration=15.792474532 podStartE2EDuration="23.40675025s" totalImagesPullingTime="7.614275718s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:26:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:26:46.404177542 +0000 UTC m=+44.958302340" watchObservedRunningTime="2026-07-28 01:26:46.40675025 +0000 UTC m=+44.960875050" Jul 28 01:26:46.869735 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:46.869698 2578 patch_prober.go:28] interesting pod/console-operator-584dcb6b54-k2wlm container/console-operator namespace/openshift-console-operator: Liveness probe status=failure output="Get \"https://10.132.0.14:8443/healthz\": dial tcp 10.132.0.14:8443: connect: connection refused" start-of-body= Jul 28 01:26:46.870037 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:46.869763 2578 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-console-operator/console-operator-584dcb6b54-k2wlm" podUID="3503116a-1398-42ea-ac86-743f5f51e989" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.14:8443/healthz\": dial tcp 10.132.0.14:8443: connect: connection refused" Jul 28 01:26:47.255241 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:47.255208 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-68grr"] Jul 28 01:26:47.286285 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:47.286247 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-68grr"] Jul 28 01:26:47.286751 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:47.286380 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-68grr" Jul 28 01:26:47.289795 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:47.289764 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 28 01:26:47.290063 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:47.290045 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-spb28\"" Jul 28 01:26:47.290872 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:47.290859 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 28 01:26:47.301633 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:47.301594 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-k2wlm_3503116a-1398-42ea-ac86-743f5f51e989/console-operator/0.log" Jul 28 01:26:47.301633 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:47.301640 2578 generic.go:356] "Generic (PLEG): container finished" podID="3503116a-1398-42ea-ac86-743f5f51e989" containerID="64862c220089dbfc63640aa7ab30d69c0f48c9028d57e4c84aaec802ddbe0051" exitCode=255 Jul 28 01:26:47.301804 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:47.301735 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-k2wlm" event={"ID":"3503116a-1398-42ea-ac86-743f5f51e989","Type":"ContainerDied","Data":"64862c220089dbfc63640aa7ab30d69c0f48c9028d57e4c84aaec802ddbe0051"} Jul 28 01:26:47.301941 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:47.301926 2578 scope.go:117] "RemoveContainer" containerID="64862c220089dbfc63640aa7ab30d69c0f48c9028d57e4c84aaec802ddbe0051" Jul 28 01:26:47.304215 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:47.304193 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-hrdd9" event={"ID":"b91abb25-2132-46b6-b0b5-7f00d007fac3","Type":"ContainerStarted","Data":"9aadf115352b8a2cdd743515d0c76cd7d8b0b3a55ab3845c25d976a8ae810e82"} Jul 28 01:26:47.393059 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:47.392976 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0888b376-095e-4e9c-9671-92986acaf8d8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-68grr\" (UID: \"0888b376-095e-4e9c-9671-92986acaf8d8\") " pod="openshift-insights/insights-runtime-extractor-68grr" Jul 28 01:26:47.393305 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:47.393282 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0888b376-095e-4e9c-9671-92986acaf8d8-data-volume\") pod \"insights-runtime-extractor-68grr\" (UID: \"0888b376-095e-4e9c-9671-92986acaf8d8\") " pod="openshift-insights/insights-runtime-extractor-68grr" Jul 28 01:26:47.393455 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:47.393430 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0888b376-095e-4e9c-9671-92986acaf8d8-crio-socket\") pod \"insights-runtime-extractor-68grr\" (UID: \"0888b376-095e-4e9c-9671-92986acaf8d8\") " pod="openshift-insights/insights-runtime-extractor-68grr" Jul 28 01:26:47.393805 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:47.393778 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7969\" (UniqueName: \"kubernetes.io/projected/0888b376-095e-4e9c-9671-92986acaf8d8-kube-api-access-t7969\") pod \"insights-runtime-extractor-68grr\" (UID: \"0888b376-095e-4e9c-9671-92986acaf8d8\") " pod="openshift-insights/insights-runtime-extractor-68grr" Jul 28 01:26:47.393900 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:47.393842 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0888b376-095e-4e9c-9671-92986acaf8d8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-68grr\" (UID: \"0888b376-095e-4e9c-9671-92986acaf8d8\") " pod="openshift-insights/insights-runtime-extractor-68grr" Jul 28 01:26:47.494571 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:47.494525 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0888b376-095e-4e9c-9671-92986acaf8d8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-68grr\" (UID: \"0888b376-095e-4e9c-9671-92986acaf8d8\") " pod="openshift-insights/insights-runtime-extractor-68grr" Jul 28 01:26:47.494704 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:47.494625 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 01:26:47.494704 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:47.494694 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0888b376-095e-4e9c-9671-92986acaf8d8-kube-rbac-proxy-cm podName:0888b376-095e-4e9c-9671-92986acaf8d8 nodeName:}" failed. No retries permitted until 2026-07-28 01:26:47.994678633 +0000 UTC m=+46.548803422 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0888b376-095e-4e9c-9671-92986acaf8d8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-68grr" (UID: "0888b376-095e-4e9c-9671-92986acaf8d8") : configmap "kube-rbac-proxy" not found Jul 28 01:26:47.494801 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:47.494744 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0888b376-095e-4e9c-9671-92986acaf8d8-data-volume\") pod \"insights-runtime-extractor-68grr\" (UID: \"0888b376-095e-4e9c-9671-92986acaf8d8\") " pod="openshift-insights/insights-runtime-extractor-68grr" Jul 28 01:26:47.494801 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:47.494768 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0888b376-095e-4e9c-9671-92986acaf8d8-crio-socket\") pod \"insights-runtime-extractor-68grr\" (UID: \"0888b376-095e-4e9c-9671-92986acaf8d8\") " pod="openshift-insights/insights-runtime-extractor-68grr" Jul 28 01:26:47.494896 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:47.494830 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7969\" (UniqueName: \"kubernetes.io/projected/0888b376-095e-4e9c-9671-92986acaf8d8-kube-api-access-t7969\") pod \"insights-runtime-extractor-68grr\" (UID: \"0888b376-095e-4e9c-9671-92986acaf8d8\") " pod="openshift-insights/insights-runtime-extractor-68grr" Jul 28 01:26:47.494896 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:47.494854 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0888b376-095e-4e9c-9671-92986acaf8d8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-68grr\" (UID: \"0888b376-095e-4e9c-9671-92986acaf8d8\") " pod="openshift-insights/insights-runtime-extractor-68grr" Jul 28 01:26:47.494996 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:47.494893 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0888b376-095e-4e9c-9671-92986acaf8d8-crio-socket\") pod \"insights-runtime-extractor-68grr\" (UID: \"0888b376-095e-4e9c-9671-92986acaf8d8\") " pod="openshift-insights/insights-runtime-extractor-68grr" Jul 28 01:26:47.494996 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:47.494944 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 01:26:47.494996 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:47.494979 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0888b376-095e-4e9c-9671-92986acaf8d8-insights-runtime-extractor-tls podName:0888b376-095e-4e9c-9671-92986acaf8d8 nodeName:}" failed. No retries permitted until 2026-07-28 01:26:47.99497006 +0000 UTC m=+46.549094836 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0888b376-095e-4e9c-9671-92986acaf8d8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-68grr" (UID: "0888b376-095e-4e9c-9671-92986acaf8d8") : secret "insights-runtime-extractor-tls" not found Jul 28 01:26:47.495137 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:47.495087 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0888b376-095e-4e9c-9671-92986acaf8d8-data-volume\") pod \"insights-runtime-extractor-68grr\" (UID: \"0888b376-095e-4e9c-9671-92986acaf8d8\") " pod="openshift-insights/insights-runtime-extractor-68grr" Jul 28 01:26:47.506055 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:47.506034 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7969\" (UniqueName: \"kubernetes.io/projected/0888b376-095e-4e9c-9671-92986acaf8d8-kube-api-access-t7969\") pod \"insights-runtime-extractor-68grr\" (UID: \"0888b376-095e-4e9c-9671-92986acaf8d8\") " pod="openshift-insights/insights-runtime-extractor-68grr" Jul 28 01:26:47.999671 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:47.999631 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0888b376-095e-4e9c-9671-92986acaf8d8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-68grr\" (UID: \"0888b376-095e-4e9c-9671-92986acaf8d8\") " pod="openshift-insights/insights-runtime-extractor-68grr" Jul 28 01:26:47.999888 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:47.999710 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0888b376-095e-4e9c-9671-92986acaf8d8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-68grr\" (UID: \"0888b376-095e-4e9c-9671-92986acaf8d8\") " pod="openshift-insights/insights-runtime-extractor-68grr" Jul 28 01:26:47.999888 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:47.999804 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 01:26:47.999888 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:47.999842 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 01:26:47.999888 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:47.999875 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0888b376-095e-4e9c-9671-92986acaf8d8-insights-runtime-extractor-tls podName:0888b376-095e-4e9c-9671-92986acaf8d8 nodeName:}" failed. No retries permitted until 2026-07-28 01:26:48.999859702 +0000 UTC m=+47.553984478 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0888b376-095e-4e9c-9671-92986acaf8d8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-68grr" (UID: "0888b376-095e-4e9c-9671-92986acaf8d8") : secret "insights-runtime-extractor-tls" not found Jul 28 01:26:48.000103 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:47.999897 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0888b376-095e-4e9c-9671-92986acaf8d8-kube-rbac-proxy-cm podName:0888b376-095e-4e9c-9671-92986acaf8d8 nodeName:}" failed. No retries permitted until 2026-07-28 01:26:48.999884591 +0000 UTC m=+47.554009368 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0888b376-095e-4e9c-9671-92986acaf8d8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-68grr" (UID: "0888b376-095e-4e9c-9671-92986acaf8d8") : configmap "kube-rbac-proxy" not found Jul 28 01:26:48.308819 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:48.308746 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-k2wlm_3503116a-1398-42ea-ac86-743f5f51e989/console-operator/1.log" Jul 28 01:26:48.309197 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:48.309144 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-k2wlm_3503116a-1398-42ea-ac86-743f5f51e989/console-operator/0.log" Jul 28 01:26:48.309197 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:48.309175 2578 generic.go:356] "Generic (PLEG): container finished" podID="3503116a-1398-42ea-ac86-743f5f51e989" containerID="ed32b5ca304cdcf4d7b38fd853b0d4258a086bcf37e33996ca044b02a000ea90" exitCode=255 Jul 28 01:26:48.309278 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:48.309208 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-k2wlm" event={"ID":"3503116a-1398-42ea-ac86-743f5f51e989","Type":"ContainerDied","Data":"ed32b5ca304cdcf4d7b38fd853b0d4258a086bcf37e33996ca044b02a000ea90"} Jul 28 01:26:48.309278 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:48.309248 2578 scope.go:117] "RemoveContainer" containerID="64862c220089dbfc63640aa7ab30d69c0f48c9028d57e4c84aaec802ddbe0051" Jul 28 01:26:48.309537 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:48.309519 2578 scope.go:117] "RemoveContainer" containerID="ed32b5ca304cdcf4d7b38fd853b0d4258a086bcf37e33996ca044b02a000ea90" Jul 28 01:26:48.309758 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:48.309735 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-k2wlm_openshift-console-operator(3503116a-1398-42ea-ac86-743f5f51e989)\"" pod="openshift-console-operator/console-operator-584dcb6b54-k2wlm" podUID="3503116a-1398-42ea-ac86-743f5f51e989" Jul 28 01:26:49.009007 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:49.008969 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0888b376-095e-4e9c-9671-92986acaf8d8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-68grr\" (UID: \"0888b376-095e-4e9c-9671-92986acaf8d8\") " pod="openshift-insights/insights-runtime-extractor-68grr" Jul 28 01:26:49.009185 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:49.009051 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0888b376-095e-4e9c-9671-92986acaf8d8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-68grr\" (UID: \"0888b376-095e-4e9c-9671-92986acaf8d8\") " pod="openshift-insights/insights-runtime-extractor-68grr" Jul 28 01:26:49.009185 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:49.009120 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 01:26:49.009185 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:49.009171 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 01:26:49.009185 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:49.009186 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0888b376-095e-4e9c-9671-92986acaf8d8-insights-runtime-extractor-tls podName:0888b376-095e-4e9c-9671-92986acaf8d8 nodeName:}" failed. No retries permitted until 2026-07-28 01:26:51.009170363 +0000 UTC m=+49.563295175 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0888b376-095e-4e9c-9671-92986acaf8d8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-68grr" (UID: "0888b376-095e-4e9c-9671-92986acaf8d8") : secret "insights-runtime-extractor-tls" not found Jul 28 01:26:49.009320 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:49.009219 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0888b376-095e-4e9c-9671-92986acaf8d8-kube-rbac-proxy-cm podName:0888b376-095e-4e9c-9671-92986acaf8d8 nodeName:}" failed. No retries permitted until 2026-07-28 01:26:51.009206652 +0000 UTC m=+49.563331453 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0888b376-095e-4e9c-9671-92986acaf8d8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-68grr" (UID: "0888b376-095e-4e9c-9671-92986acaf8d8") : configmap "kube-rbac-proxy" not found Jul 28 01:26:49.046626 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:49.046598 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2vjkb_516e3ead-ffab-4201-9715-2684dcfc7aa6/dns-node-resolver/0.log" Jul 28 01:26:49.313537 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:49.313454 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-k2wlm_3503116a-1398-42ea-ac86-743f5f51e989/console-operator/1.log" Jul 28 01:26:49.313982 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:49.313793 2578 scope.go:117] "RemoveContainer" containerID="ed32b5ca304cdcf4d7b38fd853b0d4258a086bcf37e33996ca044b02a000ea90" Jul 28 01:26:49.313982 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:49.313946 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-k2wlm_openshift-console-operator(3503116a-1398-42ea-ac86-743f5f51e989)\"" pod="openshift-console-operator/console-operator-584dcb6b54-k2wlm" podUID="3503116a-1398-42ea-ac86-743f5f51e989" Jul 28 01:26:50.249019 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:50.248993 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bgvmj_e1808965-dc33-46a8-a355-b3ebbe849c65/node-ca/0.log" Jul 28 01:26:51.027236 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:51.027199 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0888b376-095e-4e9c-9671-92986acaf8d8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-68grr\" (UID: \"0888b376-095e-4e9c-9671-92986acaf8d8\") " pod="openshift-insights/insights-runtime-extractor-68grr" Jul 28 01:26:51.027671 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:51.027274 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0888b376-095e-4e9c-9671-92986acaf8d8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-68grr\" (UID: \"0888b376-095e-4e9c-9671-92986acaf8d8\") " pod="openshift-insights/insights-runtime-extractor-68grr" Jul 28 01:26:51.027671 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:51.027345 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 01:26:51.027671 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:51.027352 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 01:26:51.027671 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:51.027395 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0888b376-095e-4e9c-9671-92986acaf8d8-kube-rbac-proxy-cm podName:0888b376-095e-4e9c-9671-92986acaf8d8 nodeName:}" failed. No retries permitted until 2026-07-28 01:26:55.02738117 +0000 UTC m=+53.581505959 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0888b376-095e-4e9c-9671-92986acaf8d8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-68grr" (UID: "0888b376-095e-4e9c-9671-92986acaf8d8") : configmap "kube-rbac-proxy" not found Jul 28 01:26:51.027671 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:51.027409 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0888b376-095e-4e9c-9671-92986acaf8d8-insights-runtime-extractor-tls podName:0888b376-095e-4e9c-9671-92986acaf8d8 nodeName:}" failed. No retries permitted until 2026-07-28 01:26:55.027403483 +0000 UTC m=+53.581528259 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0888b376-095e-4e9c-9671-92986acaf8d8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-68grr" (UID: "0888b376-095e-4e9c-9671-92986acaf8d8") : secret "insights-runtime-extractor-tls" not found Jul 28 01:26:51.249649 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:51.249620 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-sc92r_3b34c5ff-3aed-4bba-990e-878c2de528ed/serve-healthcheck-canary/0.log" Jul 28 01:26:52.237254 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:52.237214 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/31992241-f826-45d3-a26a-679d5d82465c-registry-tls\") pod \"image-registry-757879ff5c-x2xkc\" (UID: \"31992241-f826-45d3-a26a-679d5d82465c\") " pod="openshift-image-registry/image-registry-757879ff5c-x2xkc" Jul 28 01:26:52.237663 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:52.237379 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 01:26:52.237663 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:52.237400 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-757879ff5c-x2xkc: secret "image-registry-tls" not found Jul 28 01:26:52.237663 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:52.237479 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31992241-f826-45d3-a26a-679d5d82465c-registry-tls podName:31992241-f826-45d3-a26a-679d5d82465c nodeName:}" failed. No retries permitted until 2026-07-28 01:27:08.237461181 +0000 UTC m=+66.791585957 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/31992241-f826-45d3-a26a-679d5d82465c-registry-tls") pod "image-registry-757879ff5c-x2xkc" (UID: "31992241-f826-45d3-a26a-679d5d82465c") : secret "image-registry-tls" not found Jul 28 01:26:52.338012 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:52.337975 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/56e7bae5-9b23-4f6b-8084-20287730dd30-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-f297n\" (UID: \"56e7bae5-9b23-4f6b-8084-20287730dd30\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-f297n" Jul 28 01:26:52.338185 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:52.338124 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 28 01:26:52.338228 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:52.338205 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56e7bae5-9b23-4f6b-8084-20287730dd30-cluster-monitoring-operator-tls podName:56e7bae5-9b23-4f6b-8084-20287730dd30 nodeName:}" failed. No retries permitted until 2026-07-28 01:27:08.338187228 +0000 UTC m=+66.892312024 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/56e7bae5-9b23-4f6b-8084-20287730dd30-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-f297n" (UID: "56e7bae5-9b23-4f6b-8084-20287730dd30") : secret "cluster-monitoring-operator-tls" not found Jul 28 01:26:52.439407 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:52.439370 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-metrics-certs\") pod \"router-default-74577ddb85-mm25q\" (UID: \"923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71\") " pod="openshift-ingress/router-default-74577ddb85-mm25q" Jul 28 01:26:52.439593 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:52.439415 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1f2321ec-7f9d-4e33-9bed-d48fd717ca07-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4pl5d\" (UID: \"1f2321ec-7f9d-4e33-9bed-d48fd717ca07\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4pl5d" Jul 28 01:26:52.439593 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:52.439437 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5873b663-f99f-453e-8aaf-d57bf3fd0105-metrics-tls\") pod \"dns-default-n5dq2\" (UID: \"5873b663-f99f-453e-8aaf-d57bf3fd0105\") " pod="openshift-dns/dns-default-n5dq2" Jul 28 01:26:52.439593 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:52.439515 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 01:26:52.439593 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:52.439517 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 28 01:26:52.439593 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:52.439584 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-service-ca-bundle\") pod \"router-default-74577ddb85-mm25q\" (UID: \"923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71\") " pod="openshift-ingress/router-default-74577ddb85-mm25q" Jul 28 01:26:52.439810 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:52.439607 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f2321ec-7f9d-4e33-9bed-d48fd717ca07-networking-console-plugin-cert podName:1f2321ec-7f9d-4e33-9bed-d48fd717ca07 nodeName:}" failed. No retries permitted until 2026-07-28 01:27:08.43959188 +0000 UTC m=+66.993716661 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1f2321ec-7f9d-4e33-9bed-d48fd717ca07-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-4pl5d" (UID: "1f2321ec-7f9d-4e33-9bed-d48fd717ca07") : secret "networking-console-plugin-cert" not found Jul 28 01:26:52.439810 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:52.439517 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 28 01:26:52.439810 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:52.439654 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5873b663-f99f-453e-8aaf-d57bf3fd0105-metrics-tls podName:5873b663-f99f-453e-8aaf-d57bf3fd0105 nodeName:}" failed. No retries permitted until 2026-07-28 01:27:08.439638184 +0000 UTC m=+66.993762964 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5873b663-f99f-453e-8aaf-d57bf3fd0105-metrics-tls") pod "dns-default-n5dq2" (UID: "5873b663-f99f-453e-8aaf-d57bf3fd0105") : secret "dns-default-metrics-tls" not found Jul 28 01:26:52.439810 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:52.439677 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-metrics-certs podName:923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71 nodeName:}" failed. No retries permitted until 2026-07-28 01:27:08.439668096 +0000 UTC m=+66.993792875 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-metrics-certs") pod "router-default-74577ddb85-mm25q" (UID: "923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71") : secret "router-metrics-certs-default" not found Jul 28 01:26:52.439810 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:52.439700 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-service-ca-bundle podName:923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71 nodeName:}" failed. No retries permitted until 2026-07-28 01:27:08.439690666 +0000 UTC m=+66.993815443 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-service-ca-bundle") pod "router-default-74577ddb85-mm25q" (UID: "923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71") : configmap references non-existent config key: service-ca.crt Jul 28 01:26:52.439810 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:52.439760 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2af2ea06-6e9c-4367-90ae-755b5f034806-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-t5dxh\" (UID: \"2af2ea06-6e9c-4367-90ae-755b5f034806\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-t5dxh" Jul 28 01:26:52.439996 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:52.439883 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 28 01:26:52.439996 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:52.439928 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2af2ea06-6e9c-4367-90ae-755b5f034806-samples-operator-tls podName:2af2ea06-6e9c-4367-90ae-755b5f034806 nodeName:}" failed. No retries permitted until 2026-07-28 01:27:08.439914954 +0000 UTC m=+66.994039745 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2af2ea06-6e9c-4367-90ae-755b5f034806-samples-operator-tls") pod "cluster-samples-operator-6748489656-t5dxh" (UID: "2af2ea06-6e9c-4367-90ae-755b5f034806") : secret "samples-operator-tls" not found Jul 28 01:26:55.066187 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:55.066148 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0888b376-095e-4e9c-9671-92986acaf8d8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-68grr\" (UID: \"0888b376-095e-4e9c-9671-92986acaf8d8\") " pod="openshift-insights/insights-runtime-extractor-68grr" Jul 28 01:26:55.066664 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:55.066296 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0888b376-095e-4e9c-9671-92986acaf8d8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-68grr\" (UID: \"0888b376-095e-4e9c-9671-92986acaf8d8\") " pod="openshift-insights/insights-runtime-extractor-68grr" Jul 28 01:26:55.066664 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:55.066305 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 01:26:55.066664 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:55.066366 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0888b376-095e-4e9c-9671-92986acaf8d8-insights-runtime-extractor-tls podName:0888b376-095e-4e9c-9671-92986acaf8d8 nodeName:}" failed. No retries permitted until 2026-07-28 01:27:03.066350346 +0000 UTC m=+61.620475123 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0888b376-095e-4e9c-9671-92986acaf8d8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-68grr" (UID: "0888b376-095e-4e9c-9671-92986acaf8d8") : secret "insights-runtime-extractor-tls" not found Jul 28 01:26:55.066812 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:55.066796 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0888b376-095e-4e9c-9671-92986acaf8d8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-68grr\" (UID: \"0888b376-095e-4e9c-9671-92986acaf8d8\") " pod="openshift-insights/insights-runtime-extractor-68grr" Jul 28 01:26:56.282694 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:56.282665 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-k2wlm" Jul 28 01:26:56.283059 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:56.282995 2578 scope.go:117] "RemoveContainer" containerID="ed32b5ca304cdcf4d7b38fd853b0d4258a086bcf37e33996ca044b02a000ea90" Jul 28 01:26:56.283161 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:56.283141 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-k2wlm_openshift-console-operator(3503116a-1398-42ea-ac86-743f5f51e989)\"" pod="openshift-console-operator/console-operator-584dcb6b54-k2wlm" podUID="3503116a-1398-42ea-ac86-743f5f51e989" Jul 28 01:26:56.869044 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:56.869009 2578 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-k2wlm" Jul 28 01:26:56.869333 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:26:56.869321 2578 scope.go:117] "RemoveContainer" containerID="ed32b5ca304cdcf4d7b38fd853b0d4258a086bcf37e33996ca044b02a000ea90" Jul 28 01:26:56.869470 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:26:56.869454 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-k2wlm_openshift-console-operator(3503116a-1398-42ea-ac86-743f5f51e989)\"" pod="openshift-console-operator/console-operator-584dcb6b54-k2wlm" podUID="3503116a-1398-42ea-ac86-743f5f51e989" Jul 28 01:27:03.138942 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:03.138903 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0888b376-095e-4e9c-9671-92986acaf8d8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-68grr\" (UID: \"0888b376-095e-4e9c-9671-92986acaf8d8\") " pod="openshift-insights/insights-runtime-extractor-68grr" Jul 28 01:27:03.141260 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:03.141235 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0888b376-095e-4e9c-9671-92986acaf8d8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-68grr\" (UID: \"0888b376-095e-4e9c-9671-92986acaf8d8\") " pod="openshift-insights/insights-runtime-extractor-68grr" Jul 28 01:27:03.200045 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:03.200014 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-spb28\"" Jul 28 01:27:03.207313 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:03.207296 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-68grr" Jul 28 01:27:03.342571 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:03.342521 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-68grr"] Jul 28 01:27:03.345804 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:27:03.345777 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0888b376_095e_4e9c_9671_92986acaf8d8.slice/crio-1f85f886d750fa6359f9992679ccc6bf72cc022d5d42fdd204878074f849b412 WatchSource:0}: Error finding container 1f85f886d750fa6359f9992679ccc6bf72cc022d5d42fdd204878074f849b412: Status 404 returned error can't find the container with id 1f85f886d750fa6359f9992679ccc6bf72cc022d5d42fdd204878074f849b412 Jul 28 01:27:03.355254 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:03.355225 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-68grr" event={"ID":"0888b376-095e-4e9c-9671-92986acaf8d8","Type":"ContainerStarted","Data":"1f85f886d750fa6359f9992679ccc6bf72cc022d5d42fdd204878074f849b412"} Jul 28 01:27:04.154515 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:04.154487 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-c6m66" Jul 28 01:27:07.583620 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:07.583571 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1bf2b1e4-0123-4a0b-9a30-b2ea3794732c-original-pull-secret\") pod \"global-pull-secret-syncer-7gfx9\" (UID: \"1bf2b1e4-0123-4a0b-9a30-b2ea3794732c\") " pod="kube-system/global-pull-secret-syncer-7gfx9" Jul 28 01:27:07.584081 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:07.583637 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/005aafcd-65c4-45cb-aa21-484a0c94167e-metrics-certs\") pod \"network-metrics-daemon-brftp\" (UID: \"005aafcd-65c4-45cb-aa21-484a0c94167e\") " pod="openshift-multus/network-metrics-daemon-brftp" Jul 28 01:27:07.586197 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:07.586167 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 28 01:27:07.587121 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:07.587097 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 28 01:27:07.596291 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:07.596270 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/005aafcd-65c4-45cb-aa21-484a0c94167e-metrics-certs\") pod \"network-metrics-daemon-brftp\" (UID: \"005aafcd-65c4-45cb-aa21-484a0c94167e\") " pod="openshift-multus/network-metrics-daemon-brftp" Jul 28 01:27:07.596403 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:07.596302 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1bf2b1e4-0123-4a0b-9a30-b2ea3794732c-original-pull-secret\") pod \"global-pull-secret-syncer-7gfx9\" (UID: \"1bf2b1e4-0123-4a0b-9a30-b2ea3794732c\") " pod="kube-system/global-pull-secret-syncer-7gfx9" Jul 28 01:27:07.616490 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:07.616458 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7gfx9" Jul 28 01:27:07.632553 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:07.632515 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cq5js\"" Jul 28 01:27:07.640536 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:07.640510 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-brftp" Jul 28 01:27:07.684926 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:07.684894 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cfb7\" (UniqueName: \"kubernetes.io/projected/5060905d-b8ac-4bbd-94cb-b1a8a8952a9a-kube-api-access-8cfb7\") pod \"network-check-target-2znt8\" (UID: \"5060905d-b8ac-4bbd-94cb-b1a8a8952a9a\") " pod="openshift-network-diagnostics/network-check-target-2znt8" Jul 28 01:27:07.687533 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:07.687503 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cfb7\" (UniqueName: \"kubernetes.io/projected/5060905d-b8ac-4bbd-94cb-b1a8a8952a9a-kube-api-access-8cfb7\") pod \"network-check-target-2znt8\" (UID: \"5060905d-b8ac-4bbd-94cb-b1a8a8952a9a\") " pod="openshift-network-diagnostics/network-check-target-2znt8" Jul 28 01:27:07.926385 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:07.926306 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lkkxh\"" Jul 28 01:27:07.934762 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:07.934734 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2znt8" Jul 28 01:27:08.238652 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:08.238569 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-brftp"] Jul 28 01:27:08.250247 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:08.250127 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2znt8"] Jul 28 01:27:08.270325 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:08.270305 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7gfx9"] Jul 28 01:27:08.291592 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:08.291570 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/31992241-f826-45d3-a26a-679d5d82465c-registry-tls\") pod \"image-registry-757879ff5c-x2xkc\" (UID: \"31992241-f826-45d3-a26a-679d5d82465c\") " pod="openshift-image-registry/image-registry-757879ff5c-x2xkc" Jul 28 01:27:08.293826 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:08.293805 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/31992241-f826-45d3-a26a-679d5d82465c-registry-tls\") pod \"image-registry-757879ff5c-x2xkc\" (UID: \"31992241-f826-45d3-a26a-679d5d82465c\") " pod="openshift-image-registry/image-registry-757879ff5c-x2xkc" Jul 28 01:27:08.365470 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:27:08.365437 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod005aafcd_65c4_45cb_aa21_484a0c94167e.slice/crio-e1b15bd8a2d3aa3e343a36de06c43472d21e475051ae8c935c003791d6e2e8a4 WatchSource:0}: Error finding container e1b15bd8a2d3aa3e343a36de06c43472d21e475051ae8c935c003791d6e2e8a4: Status 404 returned error can't find the container with id e1b15bd8a2d3aa3e343a36de06c43472d21e475051ae8c935c003791d6e2e8a4 Jul 28 01:27:08.366204 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:27:08.366172 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5060905d_b8ac_4bbd_94cb_b1a8a8952a9a.slice/crio-7828365f1dd407b3b5ff914d832047cf30e46464b23c29ae70d9d2fc220301e0 WatchSource:0}: Error finding container 7828365f1dd407b3b5ff914d832047cf30e46464b23c29ae70d9d2fc220301e0: Status 404 returned error can't find the container with id 7828365f1dd407b3b5ff914d832047cf30e46464b23c29ae70d9d2fc220301e0 Jul 28 01:27:08.366875 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:27:08.366855 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bf2b1e4_0123_4a0b_9a30_b2ea3794732c.slice/crio-a8678c56e179803c29d44e4f8a080a39298086612e183e8b937bc9278526c54e WatchSource:0}: Error finding container a8678c56e179803c29d44e4f8a080a39298086612e183e8b937bc9278526c54e: Status 404 returned error can't find the container with id a8678c56e179803c29d44e4f8a080a39298086612e183e8b937bc9278526c54e Jul 28 01:27:08.372573 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:08.372527 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2znt8" event={"ID":"5060905d-b8ac-4bbd-94cb-b1a8a8952a9a","Type":"ContainerStarted","Data":"7828365f1dd407b3b5ff914d832047cf30e46464b23c29ae70d9d2fc220301e0"} Jul 28 01:27:08.373367 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:08.373348 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7gfx9" event={"ID":"1bf2b1e4-0123-4a0b-9a30-b2ea3794732c","Type":"ContainerStarted","Data":"a8678c56e179803c29d44e4f8a080a39298086612e183e8b937bc9278526c54e"} Jul 28 01:27:08.374289 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:08.374271 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-brftp" event={"ID":"005aafcd-65c4-45cb-aa21-484a0c94167e","Type":"ContainerStarted","Data":"e1b15bd8a2d3aa3e343a36de06c43472d21e475051ae8c935c003791d6e2e8a4"} Jul 28 01:27:08.392265 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:08.392241 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/56e7bae5-9b23-4f6b-8084-20287730dd30-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-f297n\" (UID: \"56e7bae5-9b23-4f6b-8084-20287730dd30\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-f297n" Jul 28 01:27:08.395178 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:08.395156 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/56e7bae5-9b23-4f6b-8084-20287730dd30-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-f297n\" (UID: \"56e7bae5-9b23-4f6b-8084-20287730dd30\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-f297n" Jul 28 01:27:08.492856 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:08.492834 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-metrics-certs\") pod \"router-default-74577ddb85-mm25q\" (UID: \"923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71\") " pod="openshift-ingress/router-default-74577ddb85-mm25q" Jul 28 01:27:08.492943 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:08.492882 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1f2321ec-7f9d-4e33-9bed-d48fd717ca07-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4pl5d\" (UID: \"1f2321ec-7f9d-4e33-9bed-d48fd717ca07\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4pl5d" Jul 28 01:27:08.492943 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:08.492935 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5873b663-f99f-453e-8aaf-d57bf3fd0105-metrics-tls\") pod \"dns-default-n5dq2\" (UID: \"5873b663-f99f-453e-8aaf-d57bf3fd0105\") " pod="openshift-dns/dns-default-n5dq2" Jul 28 01:27:08.493074 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:08.492973 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-service-ca-bundle\") pod \"router-default-74577ddb85-mm25q\" (UID: \"923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71\") " pod="openshift-ingress/router-default-74577ddb85-mm25q" Jul 28 01:27:08.493074 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:08.493016 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2af2ea06-6e9c-4367-90ae-755b5f034806-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-t5dxh\" (UID: \"2af2ea06-6e9c-4367-90ae-755b5f034806\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-t5dxh" Jul 28 01:27:08.494099 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:08.494037 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-service-ca-bundle\") pod \"router-default-74577ddb85-mm25q\" (UID: \"923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71\") " pod="openshift-ingress/router-default-74577ddb85-mm25q" Jul 28 01:27:08.495718 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:08.495692 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5873b663-f99f-453e-8aaf-d57bf3fd0105-metrics-tls\") pod \"dns-default-n5dq2\" (UID: \"5873b663-f99f-453e-8aaf-d57bf3fd0105\") " pod="openshift-dns/dns-default-n5dq2" Jul 28 01:27:08.495832 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:08.495733 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1f2321ec-7f9d-4e33-9bed-d48fd717ca07-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4pl5d\" (UID: \"1f2321ec-7f9d-4e33-9bed-d48fd717ca07\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4pl5d" Jul 28 01:27:08.495832 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:08.495740 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71-metrics-certs\") pod \"router-default-74577ddb85-mm25q\" (UID: \"923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71\") " pod="openshift-ingress/router-default-74577ddb85-mm25q" Jul 28 01:27:08.496181 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:08.496164 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2af2ea06-6e9c-4367-90ae-755b5f034806-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-t5dxh\" (UID: \"2af2ea06-6e9c-4367-90ae-755b5f034806\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-t5dxh" Jul 28 01:27:08.507027 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:08.507001 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-m2p2f\"" Jul 28 01:27:08.514903 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:08.514877 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-757879ff5c-x2xkc" Jul 28 01:27:08.552277 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:08.552250 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-b78xn\"" Jul 28 01:27:08.560452 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:08.560428 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-f297n" Jul 28 01:27:08.614148 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:08.613297 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-jbb4n\"" Jul 28 01:27:08.620681 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:08.620655 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-t5dxh" Jul 28 01:27:08.629961 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:08.629933 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7zgnb\"" Jul 28 01:27:08.637823 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:08.637796 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-n5dq2" Jul 28 01:27:08.676990 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:08.676961 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-757879ff5c-x2xkc"] Jul 28 01:27:08.710019 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:08.709995 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-jsw2s\"" Jul 28 01:27:08.715698 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:08.715476 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-56bw5\"" Jul 28 01:27:08.718060 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:08.718039 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-4pl5d" Jul 28 01:27:08.724105 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:08.723698 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-74577ddb85-mm25q" Jul 28 01:27:08.747808 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:08.747787 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-f297n"] Jul 28 01:27:08.817584 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:08.817237 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-t5dxh"] Jul 28 01:27:08.853574 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:08.853253 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-n5dq2"] Jul 28 01:27:08.863928 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:27:08.863887 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5873b663_f99f_453e_8aaf_d57bf3fd0105.slice/crio-aa0c1ef92ab34b490fdc794dcc9e0e92b33226069b3cc9ef1517ac49f4b81367 WatchSource:0}: Error finding container aa0c1ef92ab34b490fdc794dcc9e0e92b33226069b3cc9ef1517ac49f4b81367: Status 404 returned error can't find the container with id aa0c1ef92ab34b490fdc794dcc9e0e92b33226069b3cc9ef1517ac49f4b81367 Jul 28 01:27:08.919868 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:08.919781 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-4pl5d"] Jul 28 01:27:08.925305 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:27:08.925267 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f2321ec_7f9d_4e33_9bed_d48fd717ca07.slice/crio-09ff9d3fccc7cda62b36319b0696bdd7b55cc5f4788687cb69586f530fba0103 WatchSource:0}: Error finding container 09ff9d3fccc7cda62b36319b0696bdd7b55cc5f4788687cb69586f530fba0103: Status 404 returned error can't find the container with id 09ff9d3fccc7cda62b36319b0696bdd7b55cc5f4788687cb69586f530fba0103 Jul 28 01:27:08.960968 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:08.960836 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-74577ddb85-mm25q"] Jul 28 01:27:08.966010 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:27:08.965972 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod923f7f4a_e4e7_42a0_a7ef_85f35c3c2a71.slice/crio-484985425b9530175a2e6d52d03c6dca43acf4b997cd1ce5c6d5e3a6a6a15d25 WatchSource:0}: Error finding container 484985425b9530175a2e6d52d03c6dca43acf4b997cd1ce5c6d5e3a6a6a15d25: Status 404 returned error can't find the container with id 484985425b9530175a2e6d52d03c6dca43acf4b997cd1ce5c6d5e3a6a6a15d25 Jul 28 01:27:09.383844 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:09.383802 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-4pl5d" event={"ID":"1f2321ec-7f9d-4e33-9bed-d48fd717ca07","Type":"ContainerStarted","Data":"09ff9d3fccc7cda62b36319b0696bdd7b55cc5f4788687cb69586f530fba0103"} Jul 28 01:27:09.389448 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:09.389415 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-68grr" event={"ID":"0888b376-095e-4e9c-9671-92986acaf8d8","Type":"ContainerStarted","Data":"a14dc3c3f12d4b7629959101a60cf064e549fc84ddc3312eb2b4a43648194316"} Jul 28 01:27:09.392457 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:09.391644 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-74577ddb85-mm25q" event={"ID":"923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71","Type":"ContainerStarted","Data":"89ef6891bc65231aa3714299782555ae2c66b49056c28ad0dc62122e23e5c64b"} Jul 28 01:27:09.392457 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:09.391692 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-74577ddb85-mm25q" event={"ID":"923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71","Type":"ContainerStarted","Data":"484985425b9530175a2e6d52d03c6dca43acf4b997cd1ce5c6d5e3a6a6a15d25"} Jul 28 01:27:09.394711 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:09.394639 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-n5dq2" event={"ID":"5873b663-f99f-453e-8aaf-d57bf3fd0105","Type":"ContainerStarted","Data":"aa0c1ef92ab34b490fdc794dcc9e0e92b33226069b3cc9ef1517ac49f4b81367"} Jul 28 01:27:09.397308 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:09.397286 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-t5dxh" event={"ID":"2af2ea06-6e9c-4367-90ae-755b5f034806","Type":"ContainerStarted","Data":"f979d7db2e1aecf28d1b647e2f3a9f98bf08830eb923e51264e9934f5d9bd540"} Jul 28 01:27:09.399103 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:09.399077 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-f297n" event={"ID":"56e7bae5-9b23-4f6b-8084-20287730dd30","Type":"ContainerStarted","Data":"d80d4d57778bdbff9ce952deaef21bc5e3d65e0e25524427fc6fe5c8d8b02149"} Jul 28 01:27:09.404156 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:09.404130 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-757879ff5c-x2xkc" event={"ID":"31992241-f826-45d3-a26a-679d5d82465c","Type":"ContainerStarted","Data":"242629c75af41eed1e5ba5b158952524e9c68c1568129a60711b91b9ae59361c"} Jul 28 01:27:09.404256 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:09.404174 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-757879ff5c-x2xkc" event={"ID":"31992241-f826-45d3-a26a-679d5d82465c","Type":"ContainerStarted","Data":"8b4da04ae3d2569e2751500eefaf54c7f84bcb423f60ac4937cb2be7460d2149"} Jul 28 01:27:09.404803 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:09.404736 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-757879ff5c-x2xkc" Jul 28 01:27:09.407274 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:09.406800 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2znt8" event={"ID":"5060905d-b8ac-4bbd-94cb-b1a8a8952a9a","Type":"ContainerStarted","Data":"c308f86e6a12e42f2cd09d2b173845f01b74bfd8d19c4fea3bff84707cc57db8"} Jul 28 01:27:09.407274 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:09.407243 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2znt8" Jul 28 01:27:09.425288 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:09.424758 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-74577ddb85-mm25q" podStartSLOduration=46.424738751 podStartE2EDuration="46.424738751s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:26:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:27:09.423636418 +0000 UTC m=+67.977761219" watchObservedRunningTime="2026-07-28 01:27:09.424738751 +0000 UTC m=+67.978863530" Jul 28 01:27:09.464748 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:09.464691 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-757879ff5c-x2xkc" podStartSLOduration=67.464667599 podStartE2EDuration="1m7.464667599s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:26:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:27:09.447699485 +0000 UTC m=+68.001824284" watchObservedRunningTime="2026-07-28 01:27:09.464667599 +0000 UTC m=+68.018792396" Jul 28 01:27:09.465063 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:09.465027 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2znt8" podStartSLOduration=67.465016713 podStartE2EDuration="1m7.465016713s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:26:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:27:09.464647935 +0000 UTC m=+68.018772746" watchObservedRunningTime="2026-07-28 01:27:09.465016713 +0000 UTC m=+68.019141512" Jul 28 01:27:09.724652 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:09.724504 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-74577ddb85-mm25q" Jul 28 01:27:09.727437 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:09.727231 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-74577ddb85-mm25q" Jul 28 01:27:10.005902 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:10.005123 2578 scope.go:117] "RemoveContainer" containerID="ed32b5ca304cdcf4d7b38fd853b0d4258a086bcf37e33996ca044b02a000ea90" Jul 28 01:27:10.424766 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:10.424734 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-k2wlm_3503116a-1398-42ea-ac86-743f5f51e989/console-operator/1.log" Jul 28 01:27:10.424930 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:10.424835 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-k2wlm" event={"ID":"3503116a-1398-42ea-ac86-743f5f51e989","Type":"ContainerStarted","Data":"5ce307f09ec2911f9203a53ebb44ec66ff8fa67a823a54bdbf2294d881ace2c3"} Jul 28 01:27:10.427264 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:10.426686 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-k2wlm" Jul 28 01:27:10.434783 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:10.432907 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-brftp" event={"ID":"005aafcd-65c4-45cb-aa21-484a0c94167e","Type":"ContainerStarted","Data":"f0399525147d101efe4b53f1a0312fcb1fcb7987f434cc7932ba588cb82f5208"} Jul 28 01:27:10.434783 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:10.432975 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-74577ddb85-mm25q" Jul 28 01:27:10.435482 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:10.435439 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-74577ddb85-mm25q" Jul 28 01:27:11.426205 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:11.426032 2578 patch_prober.go:28] interesting pod/console-operator-584dcb6b54-k2wlm container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.14:8443/readyz\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 28 01:27:11.426205 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:11.426102 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-584dcb6b54-k2wlm" podUID="3503116a-1398-42ea-ac86-743f5f51e989" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.14:8443/readyz\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Jul 28 01:27:11.762803 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:11.762775 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-k2wlm" Jul 28 01:27:16.456674 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:16.456640 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-4pl5d" event={"ID":"1f2321ec-7f9d-4e33-9bed-d48fd717ca07","Type":"ContainerStarted","Data":"711140fc0428c2e29fd82189f2c20e828b2ecd15dec02989488782238ae04a92"} Jul 28 01:27:16.459021 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:16.458993 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-brftp" event={"ID":"005aafcd-65c4-45cb-aa21-484a0c94167e","Type":"ContainerStarted","Data":"c3a7921c730f05ad19b63585ce526917984182a71334729008188e462f4b3683"} Jul 28 01:27:16.462282 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:16.462254 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-t5dxh" event={"ID":"2af2ea06-6e9c-4367-90ae-755b5f034806","Type":"ContainerStarted","Data":"d53f8fbc1fdf49c6ee4cb7ca201714024880cbce35521385953377397b2e6572"} Jul 28 01:27:16.475750 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:16.475678 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-4pl5d" podStartSLOduration=47.015348982 podStartE2EDuration="53.475659882s" totalImagesPullingTime="6.4603109s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:26:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:27:16.473810104 +0000 UTC m=+75.027934905" watchObservedRunningTime="2026-07-28 01:27:16.475659882 +0000 UTC m=+75.029784683" Jul 28 01:27:16.495561 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:16.494944 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-brftp" podStartSLOduration=72.701988173 podStartE2EDuration="1m14.494920954s" totalImagesPullingTime="1.792932781s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:26:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:27:16.490804189 +0000 UTC m=+75.044928988" watchObservedRunningTime="2026-07-28 01:27:16.494920954 +0000 UTC m=+75.049045754" Jul 28 01:27:17.468629 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:17.468581 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-68grr" event={"ID":"0888b376-095e-4e9c-9671-92986acaf8d8","Type":"ContainerStarted","Data":"88840927ae972c4e2d2ab8ac95e21d6f4b823a45ef8f56b8bfe92d124c7692c2"} Jul 28 01:27:17.470993 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:17.470958 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-n5dq2" event={"ID":"5873b663-f99f-453e-8aaf-d57bf3fd0105","Type":"ContainerStarted","Data":"ca42415f48cba4684003b02b75fb73f235eef0c5e371331d8b8fe5f058e2bf56"} Jul 28 01:27:17.471176 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:17.471013 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-n5dq2" event={"ID":"5873b663-f99f-453e-8aaf-d57bf3fd0105","Type":"ContainerStarted","Data":"9a0bafcce19429b5beb81a189fc3aa7a3fd6bf5fae4e91fc522956eb315a8c33"} Jul 28 01:27:17.471176 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:17.471062 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-n5dq2" Jul 28 01:27:17.473166 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:17.473138 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-t5dxh" event={"ID":"2af2ea06-6e9c-4367-90ae-755b5f034806","Type":"ContainerStarted","Data":"2fb567964e53573a881a195357411a2cb6279831d2ea3962dcff868a9a43889f"} Jul 28 01:27:17.474719 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:17.474696 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-f297n" event={"ID":"56e7bae5-9b23-4f6b-8084-20287730dd30","Type":"ContainerStarted","Data":"c8d10d651b061bf5155d351d614714f6268b064d6a0132e34f7e74627fe9d205"} Jul 28 01:27:17.476253 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:17.476225 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7gfx9" event={"ID":"1bf2b1e4-0123-4a0b-9a30-b2ea3794732c","Type":"ContainerStarted","Data":"949253120f04f64b430742aa99babf19ef1cf4918a555a42d0fdf186d759f9dc"} Jul 28 01:27:17.494809 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:17.494755 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-n5dq2" podStartSLOduration=34.285586223 podStartE2EDuration="41.494737524s" totalImagesPullingTime="7.209151301s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:26:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:27:17.493762929 +0000 UTC m=+76.047887730" watchObservedRunningTime="2026-07-28 01:27:17.494737524 +0000 UTC m=+76.048862324" Jul 28 01:27:17.519784 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:17.519733 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-f297n" podStartSLOduration=47.039296238 podStartE2EDuration="54.51971739s" totalImagesPullingTime="7.480421152s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:26:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:27:17.518937648 +0000 UTC m=+76.073062448" watchObservedRunningTime="2026-07-28 01:27:17.51971739 +0000 UTC m=+76.073842187" Jul 28 01:27:17.548076 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:17.548020 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-t5dxh" podStartSLOduration=47.358218772 podStartE2EDuration="54.54800366s" totalImagesPullingTime="7.189784888s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:26:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:27:17.54611448 +0000 UTC m=+76.100239279" watchObservedRunningTime="2026-07-28 01:27:17.54800366 +0000 UTC m=+76.102128462" Jul 28 01:27:17.571813 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:17.571766 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-7gfx9" podStartSLOduration=67.725634733 podStartE2EDuration="1m15.571752435s" totalImagesPullingTime="7.846117702s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 01:26:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:27:17.570043341 +0000 UTC m=+76.124168140" watchObservedRunningTime="2026-07-28 01:27:17.571752435 +0000 UTC m=+76.125877233" Jul 28 01:27:21.494424 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:21.494387 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-68grr" event={"ID":"0888b376-095e-4e9c-9671-92986acaf8d8","Type":"ContainerStarted","Data":"2a47cb3182c79991a3ff924d91de406af186aff584a38950ea3e569b37b3fabe"} Jul 28 01:27:21.513382 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:21.513332 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-68grr" podStartSLOduration=17.350658695 podStartE2EDuration="34.513316689s" totalImagesPullingTime="17.162657994s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 01:26:47 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:27:21.511754487 +0000 UTC m=+80.065879286" watchObservedRunningTime="2026-07-28 01:27:21.513316689 +0000 UTC m=+80.067441486" Jul 28 01:27:25.080432 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:25.080308 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-57bb8cdcd4-8l59f"] Jul 28 01:27:25.082907 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:25.082884 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57bb8cdcd4-8l59f" Jul 28 01:27:25.086497 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:25.086474 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-x9z89\"" Jul 28 01:27:25.086632 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:25.086513 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 28 01:27:25.086632 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:25.086596 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 28 01:27:25.086632 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:25.086622 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 28 01:27:25.086823 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:25.086662 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 28 01:27:25.086823 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:25.086700 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 28 01:27:25.086823 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:25.086811 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 28 01:27:25.087006 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:25.086811 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 28 01:27:25.090488 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:25.090471 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 28 01:27:25.095253 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:25.095231 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57bb8cdcd4-8l59f"] Jul 28 01:27:25.229256 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:25.229226 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1b43f6b8-dabc-40de-944b-25145570ed23-service-ca\") pod \"console-57bb8cdcd4-8l59f\" (UID: \"1b43f6b8-dabc-40de-944b-25145570ed23\") " pod="openshift-console/console-57bb8cdcd4-8l59f" Jul 28 01:27:25.229435 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:25.229269 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1b43f6b8-dabc-40de-944b-25145570ed23-console-oauth-config\") pod \"console-57bb8cdcd4-8l59f\" (UID: \"1b43f6b8-dabc-40de-944b-25145570ed23\") " pod="openshift-console/console-57bb8cdcd4-8l59f" Jul 28 01:27:25.229435 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:25.229335 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1b43f6b8-dabc-40de-944b-25145570ed23-oauth-serving-cert\") pod \"console-57bb8cdcd4-8l59f\" (UID: \"1b43f6b8-dabc-40de-944b-25145570ed23\") " pod="openshift-console/console-57bb8cdcd4-8l59f" Jul 28 01:27:25.229435 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:25.229419 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b43f6b8-dabc-40de-944b-25145570ed23-console-serving-cert\") pod \"console-57bb8cdcd4-8l59f\" (UID: \"1b43f6b8-dabc-40de-944b-25145570ed23\") " pod="openshift-console/console-57bb8cdcd4-8l59f" Jul 28 01:27:25.229630 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:25.229450 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1b43f6b8-dabc-40de-944b-25145570ed23-console-config\") pod \"console-57bb8cdcd4-8l59f\" (UID: \"1b43f6b8-dabc-40de-944b-25145570ed23\") " pod="openshift-console/console-57bb8cdcd4-8l59f" Jul 28 01:27:25.229630 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:25.229483 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vx2x8\" (UniqueName: \"kubernetes.io/projected/1b43f6b8-dabc-40de-944b-25145570ed23-kube-api-access-vx2x8\") pod \"console-57bb8cdcd4-8l59f\" (UID: \"1b43f6b8-dabc-40de-944b-25145570ed23\") " pod="openshift-console/console-57bb8cdcd4-8l59f" Jul 28 01:27:25.229630 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:25.229515 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b43f6b8-dabc-40de-944b-25145570ed23-trusted-ca-bundle\") pod \"console-57bb8cdcd4-8l59f\" (UID: \"1b43f6b8-dabc-40de-944b-25145570ed23\") " pod="openshift-console/console-57bb8cdcd4-8l59f" Jul 28 01:27:25.330956 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:25.330834 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1b43f6b8-dabc-40de-944b-25145570ed23-service-ca\") pod \"console-57bb8cdcd4-8l59f\" (UID: \"1b43f6b8-dabc-40de-944b-25145570ed23\") " pod="openshift-console/console-57bb8cdcd4-8l59f" Jul 28 01:27:25.330956 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:25.330910 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1b43f6b8-dabc-40de-944b-25145570ed23-console-oauth-config\") pod \"console-57bb8cdcd4-8l59f\" (UID: \"1b43f6b8-dabc-40de-944b-25145570ed23\") " pod="openshift-console/console-57bb8cdcd4-8l59f" Jul 28 01:27:25.330956 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:25.330932 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1b43f6b8-dabc-40de-944b-25145570ed23-oauth-serving-cert\") pod \"console-57bb8cdcd4-8l59f\" (UID: \"1b43f6b8-dabc-40de-944b-25145570ed23\") " pod="openshift-console/console-57bb8cdcd4-8l59f" Jul 28 01:27:25.331223 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:25.330969 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b43f6b8-dabc-40de-944b-25145570ed23-console-serving-cert\") pod \"console-57bb8cdcd4-8l59f\" (UID: \"1b43f6b8-dabc-40de-944b-25145570ed23\") " pod="openshift-console/console-57bb8cdcd4-8l59f" Jul 28 01:27:25.331223 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:25.331118 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1b43f6b8-dabc-40de-944b-25145570ed23-console-config\") pod \"console-57bb8cdcd4-8l59f\" (UID: \"1b43f6b8-dabc-40de-944b-25145570ed23\") " pod="openshift-console/console-57bb8cdcd4-8l59f" Jul 28 01:27:25.332298 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:25.331505 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx2x8\" (UniqueName: \"kubernetes.io/projected/1b43f6b8-dabc-40de-944b-25145570ed23-kube-api-access-vx2x8\") pod \"console-57bb8cdcd4-8l59f\" (UID: \"1b43f6b8-dabc-40de-944b-25145570ed23\") " pod="openshift-console/console-57bb8cdcd4-8l59f" Jul 28 01:27:25.332298 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:25.331602 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b43f6b8-dabc-40de-944b-25145570ed23-trusted-ca-bundle\") pod \"console-57bb8cdcd4-8l59f\" (UID: \"1b43f6b8-dabc-40de-944b-25145570ed23\") " pod="openshift-console/console-57bb8cdcd4-8l59f" Jul 28 01:27:25.332298 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:25.332030 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1b43f6b8-dabc-40de-944b-25145570ed23-oauth-serving-cert\") pod \"console-57bb8cdcd4-8l59f\" (UID: \"1b43f6b8-dabc-40de-944b-25145570ed23\") " pod="openshift-console/console-57bb8cdcd4-8l59f" Jul 28 01:27:25.332298 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:25.332118 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1b43f6b8-dabc-40de-944b-25145570ed23-service-ca\") pod \"console-57bb8cdcd4-8l59f\" (UID: \"1b43f6b8-dabc-40de-944b-25145570ed23\") " pod="openshift-console/console-57bb8cdcd4-8l59f" Jul 28 01:27:25.332298 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:25.332218 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1b43f6b8-dabc-40de-944b-25145570ed23-console-config\") pod \"console-57bb8cdcd4-8l59f\" (UID: \"1b43f6b8-dabc-40de-944b-25145570ed23\") " pod="openshift-console/console-57bb8cdcd4-8l59f" Jul 28 01:27:25.334176 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:25.334148 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1b43f6b8-dabc-40de-944b-25145570ed23-console-oauth-config\") pod \"console-57bb8cdcd4-8l59f\" (UID: \"1b43f6b8-dabc-40de-944b-25145570ed23\") " pod="openshift-console/console-57bb8cdcd4-8l59f" Jul 28 01:27:25.334436 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:25.334402 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b43f6b8-dabc-40de-944b-25145570ed23-console-serving-cert\") pod \"console-57bb8cdcd4-8l59f\" (UID: \"1b43f6b8-dabc-40de-944b-25145570ed23\") " pod="openshift-console/console-57bb8cdcd4-8l59f" Jul 28 01:27:25.335150 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:25.335125 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b43f6b8-dabc-40de-944b-25145570ed23-trusted-ca-bundle\") pod \"console-57bb8cdcd4-8l59f\" (UID: \"1b43f6b8-dabc-40de-944b-25145570ed23\") " pod="openshift-console/console-57bb8cdcd4-8l59f" Jul 28 01:27:25.340009 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:25.339986 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vx2x8\" (UniqueName: \"kubernetes.io/projected/1b43f6b8-dabc-40de-944b-25145570ed23-kube-api-access-vx2x8\") pod \"console-57bb8cdcd4-8l59f\" (UID: \"1b43f6b8-dabc-40de-944b-25145570ed23\") " pod="openshift-console/console-57bb8cdcd4-8l59f" Jul 28 01:27:25.392102 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:25.392066 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57bb8cdcd4-8l59f" Jul 28 01:27:25.520355 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:25.520323 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57bb8cdcd4-8l59f"] Jul 28 01:27:25.523691 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:27:25.523656 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b43f6b8_dabc_40de_944b_25145570ed23.slice/crio-be5cd286066cae1f139bb01ed38f08088891277bda55272f1587b810614e3275 WatchSource:0}: Error finding container be5cd286066cae1f139bb01ed38f08088891277bda55272f1587b810614e3275: Status 404 returned error can't find the container with id be5cd286066cae1f139bb01ed38f08088891277bda55272f1587b810614e3275 Jul 28 01:27:26.516282 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:26.516234 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57bb8cdcd4-8l59f" event={"ID":"1b43f6b8-dabc-40de-944b-25145570ed23","Type":"ContainerStarted","Data":"be5cd286066cae1f139bb01ed38f08088891277bda55272f1587b810614e3275"} Jul 28 01:27:27.482266 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:27.482232 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-n5dq2" Jul 28 01:27:28.526477 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:28.526440 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57bb8cdcd4-8l59f" event={"ID":"1b43f6b8-dabc-40de-944b-25145570ed23","Type":"ContainerStarted","Data":"527260c0f0a2b01c91231fe7e4d9b80496895a685b44effc91d3b2928ec35187"} Jul 28 01:27:28.543425 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:28.543379 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-57bb8cdcd4-8l59f" podStartSLOduration=0.853637415 podStartE2EDuration="3.543363567s" totalImagesPullingTime="2.689726152s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:27:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:27:28.542912022 +0000 UTC m=+87.097036833" watchObservedRunningTime="2026-07-28 01:27:28.543363567 +0000 UTC m=+87.097488439" Jul 28 01:27:31.441663 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:31.441636 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-757879ff5c-x2xkc" Jul 28 01:27:35.393105 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:35.393057 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-57bb8cdcd4-8l59f" Jul 28 01:27:35.393485 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:35.393142 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-57bb8cdcd4-8l59f" Jul 28 01:27:35.397854 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:35.397829 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-57bb8cdcd4-8l59f" Jul 28 01:27:35.555682 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:35.555651 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-57bb8cdcd4-8l59f" Jul 28 01:27:37.594687 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.594652 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrvns"] Jul 28 01:27:37.604510 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.604485 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrvns" Jul 28 01:27:37.608579 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.608539 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 28 01:27:37.608694 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.608577 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 28 01:27:37.608694 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.608566 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 28 01:27:37.608694 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.608637 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-bb2cn\"" Jul 28 01:27:37.623342 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.623316 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsx95\" (UniqueName: \"kubernetes.io/projected/ab39056b-6a5b-41cd-9ccb-cccd5662180a-kube-api-access-fsx95\") pod \"openshift-state-metrics-b8f49f5c6-vrvns\" (UID: \"ab39056b-6a5b-41cd-9ccb-cccd5662180a\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrvns" Jul 28 01:27:37.623441 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.623367 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ab39056b-6a5b-41cd-9ccb-cccd5662180a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-vrvns\" (UID: \"ab39056b-6a5b-41cd-9ccb-cccd5662180a\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrvns" Jul 28 01:27:37.623492 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.623443 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ab39056b-6a5b-41cd-9ccb-cccd5662180a-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-vrvns\" (UID: \"ab39056b-6a5b-41cd-9ccb-cccd5662180a\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrvns" Jul 28 01:27:37.623567 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.623500 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ab39056b-6a5b-41cd-9ccb-cccd5662180a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-vrvns\" (UID: \"ab39056b-6a5b-41cd-9ccb-cccd5662180a\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrvns" Jul 28 01:27:37.629936 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.629916 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrvns"] Jul 28 01:27:37.633342 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.633322 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-t7gkq"] Jul 28 01:27:37.646338 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.646321 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-t7gkq" Jul 28 01:27:37.649825 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.649808 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 28 01:27:37.649825 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.649815 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-s548w\"" Jul 28 01:27:37.649951 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.649853 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 28 01:27:37.649951 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.649867 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 28 01:27:37.724416 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.724385 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ab39056b-6a5b-41cd-9ccb-cccd5662180a-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-vrvns\" (UID: \"ab39056b-6a5b-41cd-9ccb-cccd5662180a\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrvns" Jul 28 01:27:37.724587 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.724420 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/38c8a5bd-f01b-407f-9651-8d8665b13672-root\") pod \"node-exporter-t7gkq\" (UID: \"38c8a5bd-f01b-407f-9651-8d8665b13672\") " pod="openshift-monitoring/node-exporter-t7gkq" Jul 28 01:27:37.724587 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.724453 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/38c8a5bd-f01b-407f-9651-8d8665b13672-node-exporter-accelerators-collector-config\") pod \"node-exporter-t7gkq\" (UID: \"38c8a5bd-f01b-407f-9651-8d8665b13672\") " pod="openshift-monitoring/node-exporter-t7gkq" Jul 28 01:27:37.724587 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.724486 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ab39056b-6a5b-41cd-9ccb-cccd5662180a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-vrvns\" (UID: \"ab39056b-6a5b-41cd-9ccb-cccd5662180a\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrvns" Jul 28 01:27:37.724587 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.724508 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/38c8a5bd-f01b-407f-9651-8d8665b13672-node-exporter-wtmp\") pod \"node-exporter-t7gkq\" (UID: \"38c8a5bd-f01b-407f-9651-8d8665b13672\") " pod="openshift-monitoring/node-exporter-t7gkq" Jul 28 01:27:37.724587 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.724538 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsx95\" (UniqueName: \"kubernetes.io/projected/ab39056b-6a5b-41cd-9ccb-cccd5662180a-kube-api-access-fsx95\") pod \"openshift-state-metrics-b8f49f5c6-vrvns\" (UID: \"ab39056b-6a5b-41cd-9ccb-cccd5662180a\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrvns" Jul 28 01:27:37.724587 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.724584 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38c8a5bd-f01b-407f-9651-8d8665b13672-metrics-client-ca\") pod \"node-exporter-t7gkq\" (UID: \"38c8a5bd-f01b-407f-9651-8d8665b13672\") " pod="openshift-monitoring/node-exporter-t7gkq" Jul 28 01:27:37.724892 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.724633 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/38c8a5bd-f01b-407f-9651-8d8665b13672-node-exporter-textfile\") pod \"node-exporter-t7gkq\" (UID: \"38c8a5bd-f01b-407f-9651-8d8665b13672\") " pod="openshift-monitoring/node-exporter-t7gkq" Jul 28 01:27:37.724892 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.724715 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ab39056b-6a5b-41cd-9ccb-cccd5662180a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-vrvns\" (UID: \"ab39056b-6a5b-41cd-9ccb-cccd5662180a\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrvns" Jul 28 01:27:37.724892 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.724751 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/38c8a5bd-f01b-407f-9651-8d8665b13672-node-exporter-tls\") pod \"node-exporter-t7gkq\" (UID: \"38c8a5bd-f01b-407f-9651-8d8665b13672\") " pod="openshift-monitoring/node-exporter-t7gkq" Jul 28 01:27:37.724892 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:27:37.724797 2578 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 28 01:27:37.724892 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.724796 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsq8j\" (UniqueName: \"kubernetes.io/projected/38c8a5bd-f01b-407f-9651-8d8665b13672-kube-api-access-xsq8j\") pod \"node-exporter-t7gkq\" (UID: \"38c8a5bd-f01b-407f-9651-8d8665b13672\") " pod="openshift-monitoring/node-exporter-t7gkq" Jul 28 01:27:37.724892 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:27:37.724856 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab39056b-6a5b-41cd-9ccb-cccd5662180a-openshift-state-metrics-tls podName:ab39056b-6a5b-41cd-9ccb-cccd5662180a nodeName:}" failed. No retries permitted until 2026-07-28 01:27:38.224837059 +0000 UTC m=+96.778961857 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/ab39056b-6a5b-41cd-9ccb-cccd5662180a-openshift-state-metrics-tls") pod "openshift-state-metrics-b8f49f5c6-vrvns" (UID: "ab39056b-6a5b-41cd-9ccb-cccd5662180a") : secret "openshift-state-metrics-tls" not found Jul 28 01:27:37.725137 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.724899 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/38c8a5bd-f01b-407f-9651-8d8665b13672-sys\") pod \"node-exporter-t7gkq\" (UID: \"38c8a5bd-f01b-407f-9651-8d8665b13672\") " pod="openshift-monitoring/node-exporter-t7gkq" Jul 28 01:27:37.725137 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.724925 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/38c8a5bd-f01b-407f-9651-8d8665b13672-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t7gkq\" (UID: \"38c8a5bd-f01b-407f-9651-8d8665b13672\") " pod="openshift-monitoring/node-exporter-t7gkq" Jul 28 01:27:37.725137 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.725073 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ab39056b-6a5b-41cd-9ccb-cccd5662180a-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-vrvns\" (UID: \"ab39056b-6a5b-41cd-9ccb-cccd5662180a\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrvns" Jul 28 01:27:37.726926 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.726906 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ab39056b-6a5b-41cd-9ccb-cccd5662180a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-vrvns\" (UID: \"ab39056b-6a5b-41cd-9ccb-cccd5662180a\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrvns" Jul 28 01:27:37.735598 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.735570 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsx95\" (UniqueName: \"kubernetes.io/projected/ab39056b-6a5b-41cd-9ccb-cccd5662180a-kube-api-access-fsx95\") pod \"openshift-state-metrics-b8f49f5c6-vrvns\" (UID: \"ab39056b-6a5b-41cd-9ccb-cccd5662180a\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrvns" Jul 28 01:27:37.825424 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.825386 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/38c8a5bd-f01b-407f-9651-8d8665b13672-sys\") pod \"node-exporter-t7gkq\" (UID: \"38c8a5bd-f01b-407f-9651-8d8665b13672\") " pod="openshift-monitoring/node-exporter-t7gkq" Jul 28 01:27:37.825424 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.825421 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/38c8a5bd-f01b-407f-9651-8d8665b13672-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t7gkq\" (UID: \"38c8a5bd-f01b-407f-9651-8d8665b13672\") " pod="openshift-monitoring/node-exporter-t7gkq" Jul 28 01:27:37.825650 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.825447 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/38c8a5bd-f01b-407f-9651-8d8665b13672-root\") pod \"node-exporter-t7gkq\" (UID: \"38c8a5bd-f01b-407f-9651-8d8665b13672\") " pod="openshift-monitoring/node-exporter-t7gkq" Jul 28 01:27:37.825650 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.825467 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/38c8a5bd-f01b-407f-9651-8d8665b13672-node-exporter-accelerators-collector-config\") pod \"node-exporter-t7gkq\" (UID: \"38c8a5bd-f01b-407f-9651-8d8665b13672\") " pod="openshift-monitoring/node-exporter-t7gkq" Jul 28 01:27:37.825650 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.825499 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/38c8a5bd-f01b-407f-9651-8d8665b13672-node-exporter-wtmp\") pod \"node-exporter-t7gkq\" (UID: \"38c8a5bd-f01b-407f-9651-8d8665b13672\") " pod="openshift-monitoring/node-exporter-t7gkq" Jul 28 01:27:37.825650 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.825515 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/38c8a5bd-f01b-407f-9651-8d8665b13672-root\") pod \"node-exporter-t7gkq\" (UID: \"38c8a5bd-f01b-407f-9651-8d8665b13672\") " pod="openshift-monitoring/node-exporter-t7gkq" Jul 28 01:27:37.825650 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.825512 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/38c8a5bd-f01b-407f-9651-8d8665b13672-sys\") pod \"node-exporter-t7gkq\" (UID: \"38c8a5bd-f01b-407f-9651-8d8665b13672\") " pod="openshift-monitoring/node-exporter-t7gkq" Jul 28 01:27:37.825650 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.825520 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38c8a5bd-f01b-407f-9651-8d8665b13672-metrics-client-ca\") pod \"node-exporter-t7gkq\" (UID: \"38c8a5bd-f01b-407f-9651-8d8665b13672\") " pod="openshift-monitoring/node-exporter-t7gkq" Jul 28 01:27:37.825650 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.825615 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/38c8a5bd-f01b-407f-9651-8d8665b13672-node-exporter-textfile\") pod \"node-exporter-t7gkq\" (UID: \"38c8a5bd-f01b-407f-9651-8d8665b13672\") " pod="openshift-monitoring/node-exporter-t7gkq" Jul 28 01:27:37.825928 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.825658 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/38c8a5bd-f01b-407f-9651-8d8665b13672-node-exporter-tls\") pod \"node-exporter-t7gkq\" (UID: \"38c8a5bd-f01b-407f-9651-8d8665b13672\") " pod="openshift-monitoring/node-exporter-t7gkq" Jul 28 01:27:37.825928 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.825683 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/38c8a5bd-f01b-407f-9651-8d8665b13672-node-exporter-wtmp\") pod \"node-exporter-t7gkq\" (UID: \"38c8a5bd-f01b-407f-9651-8d8665b13672\") " pod="openshift-monitoring/node-exporter-t7gkq" Jul 28 01:27:37.825928 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.825702 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsq8j\" (UniqueName: \"kubernetes.io/projected/38c8a5bd-f01b-407f-9651-8d8665b13672-kube-api-access-xsq8j\") pod \"node-exporter-t7gkq\" (UID: \"38c8a5bd-f01b-407f-9651-8d8665b13672\") " pod="openshift-monitoring/node-exporter-t7gkq" Jul 28 01:27:37.826092 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.825949 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/38c8a5bd-f01b-407f-9651-8d8665b13672-node-exporter-textfile\") pod \"node-exporter-t7gkq\" (UID: \"38c8a5bd-f01b-407f-9651-8d8665b13672\") " pod="openshift-monitoring/node-exporter-t7gkq" Jul 28 01:27:37.826206 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.826179 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/38c8a5bd-f01b-407f-9651-8d8665b13672-node-exporter-accelerators-collector-config\") pod \"node-exporter-t7gkq\" (UID: \"38c8a5bd-f01b-407f-9651-8d8665b13672\") " pod="openshift-monitoring/node-exporter-t7gkq" Jul 28 01:27:37.826245 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.826203 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38c8a5bd-f01b-407f-9651-8d8665b13672-metrics-client-ca\") pod \"node-exporter-t7gkq\" (UID: \"38c8a5bd-f01b-407f-9651-8d8665b13672\") " pod="openshift-monitoring/node-exporter-t7gkq" Jul 28 01:27:37.827816 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.827797 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/38c8a5bd-f01b-407f-9651-8d8665b13672-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t7gkq\" (UID: \"38c8a5bd-f01b-407f-9651-8d8665b13672\") " pod="openshift-monitoring/node-exporter-t7gkq" Jul 28 01:27:37.828001 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.827982 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/38c8a5bd-f01b-407f-9651-8d8665b13672-node-exporter-tls\") pod \"node-exporter-t7gkq\" (UID: \"38c8a5bd-f01b-407f-9651-8d8665b13672\") " pod="openshift-monitoring/node-exporter-t7gkq" Jul 28 01:27:37.833962 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.833936 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsq8j\" (UniqueName: \"kubernetes.io/projected/38c8a5bd-f01b-407f-9651-8d8665b13672-kube-api-access-xsq8j\") pod \"node-exporter-t7gkq\" (UID: \"38c8a5bd-f01b-407f-9651-8d8665b13672\") " pod="openshift-monitoring/node-exporter-t7gkq" Jul 28 01:27:37.955376 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.955299 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-t7gkq" Jul 28 01:27:37.958922 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:37.958895 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-757879ff5c-x2xkc"] Jul 28 01:27:37.983696 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:27:37.983671 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38c8a5bd_f01b_407f_9651_8d8665b13672.slice/crio-9d2122dfdd8675de107d8d0db94d566f674b832d4ef972da7467f94017363eed WatchSource:0}: Error finding container 9d2122dfdd8675de107d8d0db94d566f674b832d4ef972da7467f94017363eed: Status 404 returned error can't find the container with id 9d2122dfdd8675de107d8d0db94d566f674b832d4ef972da7467f94017363eed Jul 28 01:27:38.229282 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.229211 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ab39056b-6a5b-41cd-9ccb-cccd5662180a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-vrvns\" (UID: \"ab39056b-6a5b-41cd-9ccb-cccd5662180a\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrvns" Jul 28 01:27:38.231651 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.231626 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ab39056b-6a5b-41cd-9ccb-cccd5662180a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-vrvns\" (UID: \"ab39056b-6a5b-41cd-9ccb-cccd5662180a\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrvns" Jul 28 01:27:38.513533 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.513493 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrvns" Jul 28 01:27:38.563908 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.563869 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t7gkq" event={"ID":"38c8a5bd-f01b-407f-9651-8d8665b13672","Type":"ContainerStarted","Data":"9d2122dfdd8675de107d8d0db94d566f674b832d4ef972da7467f94017363eed"} Jul 28 01:27:38.606116 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.606083 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 01:27:38.629467 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.629431 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 01:27:38.629674 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.629656 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:27:38.633007 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.632974 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 28 01:27:38.633634 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.633384 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 28 01:27:38.633634 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.633453 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-pmpfw\"" Jul 28 01:27:38.634029 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.634003 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 28 01:27:38.634148 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.634131 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 28 01:27:38.634673 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.634424 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 28 01:27:38.634673 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.634441 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 28 01:27:38.634673 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.634477 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 28 01:27:38.634673 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.634429 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 28 01:27:38.634673 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.634635 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 28 01:27:38.679682 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.679526 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrvns"] Jul 28 01:27:38.734172 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.734135 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:27:38.734330 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.734172 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzt5f\" (UniqueName: \"kubernetes.io/projected/5ee77ea9-96ed-40bc-8169-f937717e4fa5-kube-api-access-vzt5f\") pod \"alertmanager-main-0\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:27:38.734923 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.734411 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ee77ea9-96ed-40bc-8169-f937717e4fa5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:27:38.734923 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.734459 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-web-config\") pod \"alertmanager-main-0\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:27:38.734923 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.734492 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:27:38.734923 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.734525 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:27:38.734923 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.734583 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:27:38.734923 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.734612 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:27:38.734923 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.734635 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5ee77ea9-96ed-40bc-8169-f937717e4fa5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:27:38.734923 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.734661 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ee77ea9-96ed-40bc-8169-f937717e4fa5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:27:38.734923 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.734704 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5ee77ea9-96ed-40bc-8169-f937717e4fa5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:27:38.734923 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.734733 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-config-volume\") pod \"alertmanager-main-0\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:27:38.734923 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.734774 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5ee77ea9-96ed-40bc-8169-f937717e4fa5-config-out\") pod \"alertmanager-main-0\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:27:38.835249 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.835173 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:27:38.835249 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.835212 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:27:38.835249 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.835239 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5ee77ea9-96ed-40bc-8169-f937717e4fa5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:27:38.835528 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.835264 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ee77ea9-96ed-40bc-8169-f937717e4fa5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:27:38.835528 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.835307 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5ee77ea9-96ed-40bc-8169-f937717e4fa5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:27:38.835528 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.835337 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-config-volume\") pod \"alertmanager-main-0\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:27:38.835528 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.835375 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5ee77ea9-96ed-40bc-8169-f937717e4fa5-config-out\") pod \"alertmanager-main-0\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:27:38.835528 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.835430 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:27:38.835528 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.835456 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzt5f\" (UniqueName: \"kubernetes.io/projected/5ee77ea9-96ed-40bc-8169-f937717e4fa5-kube-api-access-vzt5f\") pod \"alertmanager-main-0\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:27:38.835528 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.835488 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ee77ea9-96ed-40bc-8169-f937717e4fa5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:27:38.835528 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.835521 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-web-config\") pod \"alertmanager-main-0\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:27:38.835942 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.835580 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:27:38.835942 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.835613 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:27:38.835942 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.835748 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5ee77ea9-96ed-40bc-8169-f937717e4fa5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:27:38.837382 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.836582 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ee77ea9-96ed-40bc-8169-f937717e4fa5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:27:38.837382 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:27:38.836687 2578 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 28 01:27:38.837382 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.836696 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ee77ea9-96ed-40bc-8169-f937717e4fa5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:27:38.837382 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:27:38.836756 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-secret-alertmanager-main-tls podName:5ee77ea9-96ed-40bc-8169-f937717e4fa5 nodeName:}" failed. No retries permitted until 2026-07-28 01:27:39.336736906 +0000 UTC m=+97.890861682 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "5ee77ea9-96ed-40bc-8169-f937717e4fa5") : secret "alertmanager-main-tls" not found Jul 28 01:27:38.838200 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.838168 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:27:38.838656 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.838632 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:27:38.839274 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.839223 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5ee77ea9-96ed-40bc-8169-f937717e4fa5-config-out\") pod \"alertmanager-main-0\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:27:38.839274 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.839238 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:27:38.839468 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.839447 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-config-volume\") pod \"alertmanager-main-0\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:27:38.839716 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.839692 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5ee77ea9-96ed-40bc-8169-f937717e4fa5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:27:38.840051 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.840033 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:27:38.840227 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.840212 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-web-config\") pod \"alertmanager-main-0\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:27:38.848461 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:38.848417 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzt5f\" (UniqueName: \"kubernetes.io/projected/5ee77ea9-96ed-40bc-8169-f937717e4fa5-kube-api-access-vzt5f\") pod \"alertmanager-main-0\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:27:38.895644 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:27:38.895611 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab39056b_6a5b_41cd_9ccb_cccd5662180a.slice/crio-e66bccc1a739085a6ed92e3117981a6f9cac0cbfc05c61e553faef6b02938806 WatchSource:0}: Error finding container e66bccc1a739085a6ed92e3117981a6f9cac0cbfc05c61e553faef6b02938806: Status 404 returned error can't find the container with id e66bccc1a739085a6ed92e3117981a6f9cac0cbfc05c61e553faef6b02938806 Jul 28 01:27:39.340644 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:39.340612 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:27:39.343180 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:39.343156 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:27:39.542686 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:39.542643 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:27:39.569714 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:39.569682 2578 generic.go:356] "Generic (PLEG): container finished" podID="38c8a5bd-f01b-407f-9651-8d8665b13672" containerID="2f8f4dc6c246effb03aa5e0ae9a5e0a0dc4d892b45a582f00c955cae8754fda0" exitCode=0 Jul 28 01:27:39.569833 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:39.569761 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t7gkq" event={"ID":"38c8a5bd-f01b-407f-9651-8d8665b13672","Type":"ContainerDied","Data":"2f8f4dc6c246effb03aa5e0ae9a5e0a0dc4d892b45a582f00c955cae8754fda0"} Jul 28 01:27:39.571718 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:39.571697 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrvns" event={"ID":"ab39056b-6a5b-41cd-9ccb-cccd5662180a","Type":"ContainerStarted","Data":"ac432a451fc34c58dff002906e02bacc087b039f5bd6522a66e4a31517640f4c"} Jul 28 01:27:39.571801 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:39.571738 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrvns" event={"ID":"ab39056b-6a5b-41cd-9ccb-cccd5662180a","Type":"ContainerStarted","Data":"49b9ba9ed856f19056135fd7f22ffa18a8efd4ccffc5a203319297a298056f78"} Jul 28 01:27:39.571801 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:39.571758 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrvns" event={"ID":"ab39056b-6a5b-41cd-9ccb-cccd5662180a","Type":"ContainerStarted","Data":"e66bccc1a739085a6ed92e3117981a6f9cac0cbfc05c61e553faef6b02938806"} Jul 28 01:27:39.704215 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:39.704192 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 01:27:39.706905 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:27:39.706883 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ee77ea9_96ed_40bc_8169_f937717e4fa5.slice/crio-0b87a3126c1fb19d2f6d6df7579c4256725479dcae7d786db78352f64137de85 WatchSource:0}: Error finding container 0b87a3126c1fb19d2f6d6df7579c4256725479dcae7d786db78352f64137de85: Status 404 returned error can't find the container with id 0b87a3126c1fb19d2f6d6df7579c4256725479dcae7d786db78352f64137de85 Jul 28 01:27:40.578914 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:40.578818 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t7gkq" event={"ID":"38c8a5bd-f01b-407f-9651-8d8665b13672","Type":"ContainerStarted","Data":"edf23e9b9d34098d2d185a49356081deca84feac851bdbabeec4492d480e4502"} Jul 28 01:27:40.579062 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:40.578914 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t7gkq" event={"ID":"38c8a5bd-f01b-407f-9651-8d8665b13672","Type":"ContainerStarted","Data":"de4bbe9e53d9a4c8c5beb67da92d8c9123c0218e9e78f3d6dafd55f6aeabf28a"} Jul 28 01:27:40.581432 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:40.581402 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrvns" event={"ID":"ab39056b-6a5b-41cd-9ccb-cccd5662180a","Type":"ContainerStarted","Data":"0bfcc2fa2e0e8f1dbe07f7d4d1e1964799368690a973506502647fbd14715925"} Jul 28 01:27:40.582672 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:40.582644 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ee77ea9-96ed-40bc-8169-f937717e4fa5","Type":"ContainerStarted","Data":"0b87a3126c1fb19d2f6d6df7579c4256725479dcae7d786db78352f64137de85"} Jul 28 01:27:40.614135 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:40.614078 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-t7gkq" podStartSLOduration=2.33632999 podStartE2EDuration="3.614058484s" totalImagesPullingTime="1.1920993s" totalInitContainerRuntime="85.629194ms" isStatefulPod=true podCreationTimestamp="2026-07-28 01:27:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:27:40.612314769 +0000 UTC m=+99.166439566" watchObservedRunningTime="2026-07-28 01:27:40.614058484 +0000 UTC m=+99.168183284" Jul 28 01:27:40.617964 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:40.617933 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-84c8d95df8-22666"] Jul 28 01:27:40.623064 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:40.623041 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-84c8d95df8-22666" Jul 28 01:27:40.628978 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:40.628957 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 28 01:27:40.629161 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:40.629057 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 28 01:27:40.629261 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:40.629240 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-cne49fgupjq50\"" Jul 28 01:27:40.629402 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:40.629362 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-tlvv4\"" Jul 28 01:27:40.629641 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:40.629600 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 28 01:27:40.629816 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:40.629791 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 28 01:27:40.630018 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:40.629926 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 28 01:27:40.644768 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:40.644732 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-84c8d95df8-22666"] Jul 28 01:27:40.645505 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:40.645456 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrvns" podStartSLOduration=2.614663919 podStartE2EDuration="3.645441826s" totalImagesPullingTime="1.030777907s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:27:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:27:40.640828011 +0000 UTC m=+99.194952810" watchObservedRunningTime="2026-07-28 01:27:40.645441826 +0000 UTC m=+99.199566623" Jul 28 01:27:40.653579 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:40.653539 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6335a2b9-cce3-440c-a57b-cfc264e024de-metrics-client-ca\") pod \"thanos-querier-84c8d95df8-22666\" (UID: \"6335a2b9-cce3-440c-a57b-cfc264e024de\") " pod="openshift-monitoring/thanos-querier-84c8d95df8-22666" Jul 28 01:27:40.653731 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:40.653639 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6335a2b9-cce3-440c-a57b-cfc264e024de-secret-thanos-querier-tls\") pod \"thanos-querier-84c8d95df8-22666\" (UID: \"6335a2b9-cce3-440c-a57b-cfc264e024de\") " pod="openshift-monitoring/thanos-querier-84c8d95df8-22666" Jul 28 01:27:40.653731 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:40.653701 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6335a2b9-cce3-440c-a57b-cfc264e024de-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-84c8d95df8-22666\" (UID: \"6335a2b9-cce3-440c-a57b-cfc264e024de\") " pod="openshift-monitoring/thanos-querier-84c8d95df8-22666" Jul 28 01:27:40.653849 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:40.653788 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6335a2b9-cce3-440c-a57b-cfc264e024de-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-84c8d95df8-22666\" (UID: \"6335a2b9-cce3-440c-a57b-cfc264e024de\") " pod="openshift-monitoring/thanos-querier-84c8d95df8-22666" Jul 28 01:27:40.654006 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:40.653982 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8g86\" (UniqueName: \"kubernetes.io/projected/6335a2b9-cce3-440c-a57b-cfc264e024de-kube-api-access-d8g86\") pod \"thanos-querier-84c8d95df8-22666\" (UID: \"6335a2b9-cce3-440c-a57b-cfc264e024de\") " pod="openshift-monitoring/thanos-querier-84c8d95df8-22666" Jul 28 01:27:40.654077 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:40.654046 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6335a2b9-cce3-440c-a57b-cfc264e024de-secret-grpc-tls\") pod \"thanos-querier-84c8d95df8-22666\" (UID: \"6335a2b9-cce3-440c-a57b-cfc264e024de\") " pod="openshift-monitoring/thanos-querier-84c8d95df8-22666" Jul 28 01:27:40.654211 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:40.654183 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6335a2b9-cce3-440c-a57b-cfc264e024de-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-84c8d95df8-22666\" (UID: \"6335a2b9-cce3-440c-a57b-cfc264e024de\") " pod="openshift-monitoring/thanos-querier-84c8d95df8-22666" Jul 28 01:27:40.654297 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:40.654269 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6335a2b9-cce3-440c-a57b-cfc264e024de-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-84c8d95df8-22666\" (UID: \"6335a2b9-cce3-440c-a57b-cfc264e024de\") " pod="openshift-monitoring/thanos-querier-84c8d95df8-22666" Jul 28 01:27:40.755249 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:40.755205 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6335a2b9-cce3-440c-a57b-cfc264e024de-secret-thanos-querier-tls\") pod \"thanos-querier-84c8d95df8-22666\" (UID: \"6335a2b9-cce3-440c-a57b-cfc264e024de\") " pod="openshift-monitoring/thanos-querier-84c8d95df8-22666" Jul 28 01:27:40.755659 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:40.755271 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6335a2b9-cce3-440c-a57b-cfc264e024de-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-84c8d95df8-22666\" (UID: \"6335a2b9-cce3-440c-a57b-cfc264e024de\") " pod="openshift-monitoring/thanos-querier-84c8d95df8-22666" Jul 28 01:27:40.755659 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:40.755330 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6335a2b9-cce3-440c-a57b-cfc264e024de-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-84c8d95df8-22666\" (UID: \"6335a2b9-cce3-440c-a57b-cfc264e024de\") " pod="openshift-monitoring/thanos-querier-84c8d95df8-22666" Jul 28 01:27:40.755659 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:40.755425 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8g86\" (UniqueName: \"kubernetes.io/projected/6335a2b9-cce3-440c-a57b-cfc264e024de-kube-api-access-d8g86\") pod \"thanos-querier-84c8d95df8-22666\" (UID: \"6335a2b9-cce3-440c-a57b-cfc264e024de\") " pod="openshift-monitoring/thanos-querier-84c8d95df8-22666" Jul 28 01:27:40.755659 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:40.755486 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6335a2b9-cce3-440c-a57b-cfc264e024de-secret-grpc-tls\") pod \"thanos-querier-84c8d95df8-22666\" (UID: \"6335a2b9-cce3-440c-a57b-cfc264e024de\") " pod="openshift-monitoring/thanos-querier-84c8d95df8-22666" Jul 28 01:27:40.755659 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:40.755522 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6335a2b9-cce3-440c-a57b-cfc264e024de-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-84c8d95df8-22666\" (UID: \"6335a2b9-cce3-440c-a57b-cfc264e024de\") " pod="openshift-monitoring/thanos-querier-84c8d95df8-22666" Jul 28 01:27:40.755659 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:40.755588 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6335a2b9-cce3-440c-a57b-cfc264e024de-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-84c8d95df8-22666\" (UID: \"6335a2b9-cce3-440c-a57b-cfc264e024de\") " pod="openshift-monitoring/thanos-querier-84c8d95df8-22666" Jul 28 01:27:40.755659 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:40.755624 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6335a2b9-cce3-440c-a57b-cfc264e024de-metrics-client-ca\") pod \"thanos-querier-84c8d95df8-22666\" (UID: \"6335a2b9-cce3-440c-a57b-cfc264e024de\") " pod="openshift-monitoring/thanos-querier-84c8d95df8-22666" Jul 28 01:27:40.756429 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:40.756365 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6335a2b9-cce3-440c-a57b-cfc264e024de-metrics-client-ca\") pod \"thanos-querier-84c8d95df8-22666\" (UID: \"6335a2b9-cce3-440c-a57b-cfc264e024de\") " pod="openshift-monitoring/thanos-querier-84c8d95df8-22666" Jul 28 01:27:40.758480 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:40.758353 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6335a2b9-cce3-440c-a57b-cfc264e024de-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-84c8d95df8-22666\" (UID: \"6335a2b9-cce3-440c-a57b-cfc264e024de\") " pod="openshift-monitoring/thanos-querier-84c8d95df8-22666" Jul 28 01:27:40.758480 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:40.758393 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6335a2b9-cce3-440c-a57b-cfc264e024de-secret-thanos-querier-tls\") pod \"thanos-querier-84c8d95df8-22666\" (UID: \"6335a2b9-cce3-440c-a57b-cfc264e024de\") " pod="openshift-monitoring/thanos-querier-84c8d95df8-22666" Jul 28 01:27:40.758824 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:40.758787 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6335a2b9-cce3-440c-a57b-cfc264e024de-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-84c8d95df8-22666\" (UID: \"6335a2b9-cce3-440c-a57b-cfc264e024de\") " pod="openshift-monitoring/thanos-querier-84c8d95df8-22666" Jul 28 01:27:40.758950 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:40.758882 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6335a2b9-cce3-440c-a57b-cfc264e024de-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-84c8d95df8-22666\" (UID: \"6335a2b9-cce3-440c-a57b-cfc264e024de\") " pod="openshift-monitoring/thanos-querier-84c8d95df8-22666" Jul 28 01:27:40.759013 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:40.758983 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6335a2b9-cce3-440c-a57b-cfc264e024de-secret-grpc-tls\") pod \"thanos-querier-84c8d95df8-22666\" (UID: \"6335a2b9-cce3-440c-a57b-cfc264e024de\") " pod="openshift-monitoring/thanos-querier-84c8d95df8-22666" Jul 28 01:27:40.759090 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:40.759061 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6335a2b9-cce3-440c-a57b-cfc264e024de-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-84c8d95df8-22666\" (UID: \"6335a2b9-cce3-440c-a57b-cfc264e024de\") " pod="openshift-monitoring/thanos-querier-84c8d95df8-22666" Jul 28 01:27:40.765377 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:40.765333 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8g86\" (UniqueName: \"kubernetes.io/projected/6335a2b9-cce3-440c-a57b-cfc264e024de-kube-api-access-d8g86\") pod \"thanos-querier-84c8d95df8-22666\" (UID: \"6335a2b9-cce3-440c-a57b-cfc264e024de\") " pod="openshift-monitoring/thanos-querier-84c8d95df8-22666" Jul 28 01:27:40.934386 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:40.934362 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-84c8d95df8-22666" Jul 28 01:27:41.076958 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:41.076930 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-84c8d95df8-22666"] Jul 28 01:27:41.079789 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:27:41.079756 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6335a2b9_cce3_440c_a57b_cfc264e024de.slice/crio-a4f0147b746d12163aaa1069b715f6dfa9765a8f1d045aae4a4b997e2d085c37 WatchSource:0}: Error finding container a4f0147b746d12163aaa1069b715f6dfa9765a8f1d045aae4a4b997e2d085c37: Status 404 returned error can't find the container with id a4f0147b746d12163aaa1069b715f6dfa9765a8f1d045aae4a4b997e2d085c37 Jul 28 01:27:41.439235 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:41.439202 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2znt8" Jul 28 01:27:41.589802 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:41.589758 2578 generic.go:356] "Generic (PLEG): container finished" podID="5ee77ea9-96ed-40bc-8169-f937717e4fa5" containerID="d7b6dff65da4c81d0c28f380fcaa9c05b20813cdf96098212d10433718e562a0" exitCode=0 Jul 28 01:27:41.589966 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:41.589838 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ee77ea9-96ed-40bc-8169-f937717e4fa5","Type":"ContainerDied","Data":"d7b6dff65da4c81d0c28f380fcaa9c05b20813cdf96098212d10433718e562a0"} Jul 28 01:27:41.591355 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:41.591324 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84c8d95df8-22666" event={"ID":"6335a2b9-cce3-440c-a57b-cfc264e024de","Type":"ContainerStarted","Data":"a4f0147b746d12163aaa1069b715f6dfa9765a8f1d045aae4a4b997e2d085c37"} Jul 28 01:27:42.323538 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:42.323503 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-cr46m"] Jul 28 01:27:42.326279 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:42.326251 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-cr46m" Jul 28 01:27:42.328829 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:42.328804 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 28 01:27:42.328943 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:42.328916 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-fsr6f\"" Jul 28 01:27:42.337618 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:42.337595 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-cr46m"] Jul 28 01:27:42.370504 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:42.370468 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/339cd32a-6e07-4d34-a4ee-e72122365e49-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-cr46m\" (UID: \"339cd32a-6e07-4d34-a4ee-e72122365e49\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-cr46m" Jul 28 01:27:42.370715 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:42.370521 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jwc9\" (UniqueName: \"kubernetes.io/projected/339cd32a-6e07-4d34-a4ee-e72122365e49-kube-api-access-9jwc9\") pod \"monitoring-plugin-68b7fc9574-cr46m\" (UID: \"339cd32a-6e07-4d34-a4ee-e72122365e49\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-cr46m" Jul 28 01:27:42.471404 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:42.471365 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/339cd32a-6e07-4d34-a4ee-e72122365e49-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-cr46m\" (UID: \"339cd32a-6e07-4d34-a4ee-e72122365e49\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-cr46m" Jul 28 01:27:42.471615 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:42.471418 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jwc9\" (UniqueName: \"kubernetes.io/projected/339cd32a-6e07-4d34-a4ee-e72122365e49-kube-api-access-9jwc9\") pod \"monitoring-plugin-68b7fc9574-cr46m\" (UID: \"339cd32a-6e07-4d34-a4ee-e72122365e49\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-cr46m" Jul 28 01:27:42.471615 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:27:42.471530 2578 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 28 01:27:42.471756 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:27:42.471629 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/339cd32a-6e07-4d34-a4ee-e72122365e49-monitoring-plugin-cert podName:339cd32a-6e07-4d34-a4ee-e72122365e49 nodeName:}" failed. No retries permitted until 2026-07-28 01:27:42.971604309 +0000 UTC m=+101.525729099 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/339cd32a-6e07-4d34-a4ee-e72122365e49-monitoring-plugin-cert") pod "monitoring-plugin-68b7fc9574-cr46m" (UID: "339cd32a-6e07-4d34-a4ee-e72122365e49") : secret "monitoring-plugin-cert" not found Jul 28 01:27:42.482965 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:42.482908 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jwc9\" (UniqueName: \"kubernetes.io/projected/339cd32a-6e07-4d34-a4ee-e72122365e49-kube-api-access-9jwc9\") pod \"monitoring-plugin-68b7fc9574-cr46m\" (UID: \"339cd32a-6e07-4d34-a4ee-e72122365e49\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-cr46m" Jul 28 01:27:42.975828 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:42.975792 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/339cd32a-6e07-4d34-a4ee-e72122365e49-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-cr46m\" (UID: \"339cd32a-6e07-4d34-a4ee-e72122365e49\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-cr46m" Jul 28 01:27:42.975980 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:27:42.975941 2578 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 28 01:27:42.976025 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:27:42.976016 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/339cd32a-6e07-4d34-a4ee-e72122365e49-monitoring-plugin-cert podName:339cd32a-6e07-4d34-a4ee-e72122365e49 nodeName:}" failed. No retries permitted until 2026-07-28 01:27:43.97599222 +0000 UTC m=+102.530116996 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/339cd32a-6e07-4d34-a4ee-e72122365e49-monitoring-plugin-cert") pod "monitoring-plugin-68b7fc9574-cr46m" (UID: "339cd32a-6e07-4d34-a4ee-e72122365e49") : secret "monitoring-plugin-cert" not found Jul 28 01:27:43.602711 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:43.602680 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84c8d95df8-22666" event={"ID":"6335a2b9-cce3-440c-a57b-cfc264e024de","Type":"ContainerStarted","Data":"f1f1acf9ee86a3844ed79cbd3d05ab1f060100bd84c0665bbb8e9d4cc5f20568"} Jul 28 01:27:43.603131 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:43.602760 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84c8d95df8-22666" event={"ID":"6335a2b9-cce3-440c-a57b-cfc264e024de","Type":"ContainerStarted","Data":"8c87f89bc6e8d4b8b31e10f4036ec90c490203cd3bc9cea7d92b6cf8b169214f"} Jul 28 01:27:43.603131 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:43.602799 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84c8d95df8-22666" event={"ID":"6335a2b9-cce3-440c-a57b-cfc264e024de","Type":"ContainerStarted","Data":"8cffd400134167e09a3cc59b10dbc86c9bb36e6044f04aa0b99acdb90a485413"} Jul 28 01:27:43.606306 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:43.606276 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ee77ea9-96ed-40bc-8169-f937717e4fa5","Type":"ContainerStarted","Data":"0e06f08cb7fc2a37e3c0f8483a60b414bad7352d567e0b17df09d0ee25eed895"} Jul 28 01:27:43.606444 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:43.606358 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ee77ea9-96ed-40bc-8169-f937717e4fa5","Type":"ContainerStarted","Data":"58ae96a0f7fecb3a5e246cbee267e45082830b63cbb0e60bef67e013ae80a069"} Jul 28 01:27:43.606444 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:43.606403 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ee77ea9-96ed-40bc-8169-f937717e4fa5","Type":"ContainerStarted","Data":"3fd45d513856778bc013daafa3566a777ee75087d7ef42d1307e7a5848fa5a6e"} Jul 28 01:27:43.606444 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:43.606440 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ee77ea9-96ed-40bc-8169-f937717e4fa5","Type":"ContainerStarted","Data":"81baa42daa247159ef19313af55b8637910318b62da81e6fa62f499a232b5062"} Jul 28 01:27:43.606594 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:43.606480 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ee77ea9-96ed-40bc-8169-f937717e4fa5","Type":"ContainerStarted","Data":"8d443e0d2dcb90422cedf500a702235a98e4b46ca9acfa7fb86b0df364b3ee57"} Jul 28 01:27:43.985350 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:43.985323 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/339cd32a-6e07-4d34-a4ee-e72122365e49-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-cr46m\" (UID: \"339cd32a-6e07-4d34-a4ee-e72122365e49\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-cr46m" Jul 28 01:27:43.987442 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:43.987418 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/339cd32a-6e07-4d34-a4ee-e72122365e49-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-cr46m\" (UID: \"339cd32a-6e07-4d34-a4ee-e72122365e49\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-cr46m" Jul 28 01:27:44.139601 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:44.139575 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-cr46m" Jul 28 01:27:44.263652 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:44.263572 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-cr46m"] Jul 28 01:27:44.266499 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:27:44.266475 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod339cd32a_6e07_4d34_a4ee_e72122365e49.slice/crio-dae0b80a2b2ed103f95bbf1dd15afa5f61da43e35663d31ef923f8be6b5e6cf2 WatchSource:0}: Error finding container dae0b80a2b2ed103f95bbf1dd15afa5f61da43e35663d31ef923f8be6b5e6cf2: Status 404 returned error can't find the container with id dae0b80a2b2ed103f95bbf1dd15afa5f61da43e35663d31ef923f8be6b5e6cf2 Jul 28 01:27:44.613304 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:44.613271 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ee77ea9-96ed-40bc-8169-f937717e4fa5","Type":"ContainerStarted","Data":"770b7f5bf8166a25ad1ac58f4440d07c850d8fff652792226757105ca257422c"} Jul 28 01:27:44.614533 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:44.614507 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-cr46m" event={"ID":"339cd32a-6e07-4d34-a4ee-e72122365e49","Type":"ContainerStarted","Data":"dae0b80a2b2ed103f95bbf1dd15afa5f61da43e35663d31ef923f8be6b5e6cf2"} Jul 28 01:27:44.617266 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:44.617241 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84c8d95df8-22666" event={"ID":"6335a2b9-cce3-440c-a57b-cfc264e024de","Type":"ContainerStarted","Data":"cdf8134e2fed00956f90d876d3faea4e30fcea39f41f1d6db819298cc61a26f6"} Jul 28 01:27:44.617344 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:44.617303 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84c8d95df8-22666" event={"ID":"6335a2b9-cce3-440c-a57b-cfc264e024de","Type":"ContainerStarted","Data":"6a52da133e07858167ac2a8741361c41a96aea438df34b11f2979f9e131ff381"} Jul 28 01:27:44.617399 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:44.617346 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84c8d95df8-22666" event={"ID":"6335a2b9-cce3-440c-a57b-cfc264e024de","Type":"ContainerStarted","Data":"fad4d689a8478c05b28975a1abdf02f4de4a900316745ac948e621fc96d652c6"} Jul 28 01:27:44.617399 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:44.617388 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-84c8d95df8-22666" Jul 28 01:27:44.696095 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:44.696045 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.458260872 podStartE2EDuration="6.696027186s" totalImagesPullingTime="3.220984786s" totalInitContainerRuntime="16.781528ms" isStatefulPod=false podCreationTimestamp="2026-07-28 01:27:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:27:44.695024982 +0000 UTC m=+103.249149780" watchObservedRunningTime="2026-07-28 01:27:44.696027186 +0000 UTC m=+103.250151985" Jul 28 01:27:44.731601 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:44.731521 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-84c8d95df8-22666" podStartSLOduration=2.087168097 podStartE2EDuration="4.731500995s" totalImagesPullingTime="2.644332898s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:27:40 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:27:44.730878858 +0000 UTC m=+103.285003687" watchObservedRunningTime="2026-07-28 01:27:44.731500995 +0000 UTC m=+103.285625794" Jul 28 01:27:45.623748 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:45.623665 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-cr46m" event={"ID":"339cd32a-6e07-4d34-a4ee-e72122365e49","Type":"ContainerStarted","Data":"364047321b48fcc54136bc5680d12b8d031b32a1e19d85c274c87fd2f55ff5d6"} Jul 28 01:27:45.641409 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:45.641362 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-cr46m" podStartSLOduration=2.601823815 podStartE2EDuration="3.64134568s" totalImagesPullingTime="1.039521865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:27:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:27:45.639082277 +0000 UTC m=+104.193207066" watchObservedRunningTime="2026-07-28 01:27:45.64134568 +0000 UTC m=+104.195470481" Jul 28 01:27:46.628706 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:46.628670 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-cr46m" Jul 28 01:27:46.633755 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:46.633733 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-cr46m" Jul 28 01:27:50.630320 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:50.630289 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-84c8d95df8-22666" Jul 28 01:27:53.026415 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:53.026381 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-64c4b96fd9-8vd5q"] Jul 28 01:27:53.077598 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:53.077565 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64c4b96fd9-8vd5q"] Jul 28 01:27:53.077755 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:53.077624 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64c4b96fd9-8vd5q" Jul 28 01:27:53.164401 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:53.164373 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a-trusted-ca-bundle\") pod \"console-64c4b96fd9-8vd5q\" (UID: \"6df465b5-b8f4-4b86-bbe1-c75dedf24d7a\") " pod="openshift-console/console-64c4b96fd9-8vd5q" Jul 28 01:27:53.164526 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:53.164408 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a-console-config\") pod \"console-64c4b96fd9-8vd5q\" (UID: \"6df465b5-b8f4-4b86-bbe1-c75dedf24d7a\") " pod="openshift-console/console-64c4b96fd9-8vd5q" Jul 28 01:27:53.164526 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:53.164430 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrgbz\" (UniqueName: \"kubernetes.io/projected/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a-kube-api-access-hrgbz\") pod \"console-64c4b96fd9-8vd5q\" (UID: \"6df465b5-b8f4-4b86-bbe1-c75dedf24d7a\") " pod="openshift-console/console-64c4b96fd9-8vd5q" Jul 28 01:27:53.164526 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:53.164489 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a-console-oauth-config\") pod \"console-64c4b96fd9-8vd5q\" (UID: \"6df465b5-b8f4-4b86-bbe1-c75dedf24d7a\") " pod="openshift-console/console-64c4b96fd9-8vd5q" Jul 28 01:27:53.164662 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:53.164564 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a-console-serving-cert\") pod \"console-64c4b96fd9-8vd5q\" (UID: \"6df465b5-b8f4-4b86-bbe1-c75dedf24d7a\") " pod="openshift-console/console-64c4b96fd9-8vd5q" Jul 28 01:27:53.164662 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:53.164600 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a-oauth-serving-cert\") pod \"console-64c4b96fd9-8vd5q\" (UID: \"6df465b5-b8f4-4b86-bbe1-c75dedf24d7a\") " pod="openshift-console/console-64c4b96fd9-8vd5q" Jul 28 01:27:53.164662 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:53.164627 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a-service-ca\") pod \"console-64c4b96fd9-8vd5q\" (UID: \"6df465b5-b8f4-4b86-bbe1-c75dedf24d7a\") " pod="openshift-console/console-64c4b96fd9-8vd5q" Jul 28 01:27:53.265943 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:53.265915 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a-trusted-ca-bundle\") pod \"console-64c4b96fd9-8vd5q\" (UID: \"6df465b5-b8f4-4b86-bbe1-c75dedf24d7a\") " pod="openshift-console/console-64c4b96fd9-8vd5q" Jul 28 01:27:53.266106 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:53.265947 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a-console-config\") pod \"console-64c4b96fd9-8vd5q\" (UID: \"6df465b5-b8f4-4b86-bbe1-c75dedf24d7a\") " pod="openshift-console/console-64c4b96fd9-8vd5q" Jul 28 01:27:53.266106 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:53.265973 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrgbz\" (UniqueName: \"kubernetes.io/projected/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a-kube-api-access-hrgbz\") pod \"console-64c4b96fd9-8vd5q\" (UID: \"6df465b5-b8f4-4b86-bbe1-c75dedf24d7a\") " pod="openshift-console/console-64c4b96fd9-8vd5q" Jul 28 01:27:53.266106 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:53.265995 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a-console-oauth-config\") pod \"console-64c4b96fd9-8vd5q\" (UID: \"6df465b5-b8f4-4b86-bbe1-c75dedf24d7a\") " pod="openshift-console/console-64c4b96fd9-8vd5q" Jul 28 01:27:53.266106 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:53.266019 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a-console-serving-cert\") pod \"console-64c4b96fd9-8vd5q\" (UID: \"6df465b5-b8f4-4b86-bbe1-c75dedf24d7a\") " pod="openshift-console/console-64c4b96fd9-8vd5q" Jul 28 01:27:53.266106 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:53.266040 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a-oauth-serving-cert\") pod \"console-64c4b96fd9-8vd5q\" (UID: \"6df465b5-b8f4-4b86-bbe1-c75dedf24d7a\") " pod="openshift-console/console-64c4b96fd9-8vd5q" Jul 28 01:27:53.266106 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:53.266065 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a-service-ca\") pod \"console-64c4b96fd9-8vd5q\" (UID: \"6df465b5-b8f4-4b86-bbe1-c75dedf24d7a\") " pod="openshift-console/console-64c4b96fd9-8vd5q" Jul 28 01:27:53.266815 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:53.266790 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a-service-ca\") pod \"console-64c4b96fd9-8vd5q\" (UID: \"6df465b5-b8f4-4b86-bbe1-c75dedf24d7a\") " pod="openshift-console/console-64c4b96fd9-8vd5q" Jul 28 01:27:53.266936 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:53.266791 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a-console-config\") pod \"console-64c4b96fd9-8vd5q\" (UID: \"6df465b5-b8f4-4b86-bbe1-c75dedf24d7a\") " pod="openshift-console/console-64c4b96fd9-8vd5q" Jul 28 01:27:53.266936 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:53.266872 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a-oauth-serving-cert\") pod \"console-64c4b96fd9-8vd5q\" (UID: \"6df465b5-b8f4-4b86-bbe1-c75dedf24d7a\") " pod="openshift-console/console-64c4b96fd9-8vd5q" Jul 28 01:27:53.267016 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:53.266967 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a-trusted-ca-bundle\") pod \"console-64c4b96fd9-8vd5q\" (UID: \"6df465b5-b8f4-4b86-bbe1-c75dedf24d7a\") " pod="openshift-console/console-64c4b96fd9-8vd5q" Jul 28 01:27:53.268353 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:53.268326 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a-console-oauth-config\") pod \"console-64c4b96fd9-8vd5q\" (UID: \"6df465b5-b8f4-4b86-bbe1-c75dedf24d7a\") " pod="openshift-console/console-64c4b96fd9-8vd5q" Jul 28 01:27:53.268441 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:53.268422 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a-console-serving-cert\") pod \"console-64c4b96fd9-8vd5q\" (UID: \"6df465b5-b8f4-4b86-bbe1-c75dedf24d7a\") " pod="openshift-console/console-64c4b96fd9-8vd5q" Jul 28 01:27:53.274501 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:53.274481 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrgbz\" (UniqueName: \"kubernetes.io/projected/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a-kube-api-access-hrgbz\") pod \"console-64c4b96fd9-8vd5q\" (UID: \"6df465b5-b8f4-4b86-bbe1-c75dedf24d7a\") " pod="openshift-console/console-64c4b96fd9-8vd5q" Jul 28 01:27:53.386810 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:53.386747 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64c4b96fd9-8vd5q" Jul 28 01:27:53.526371 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:53.526347 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64c4b96fd9-8vd5q"] Jul 28 01:27:53.528529 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:27:53.528487 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6df465b5_b8f4_4b86_bbe1_c75dedf24d7a.slice/crio-29984f0d77174277c84fc79903d7c509a5df6bf2c77669f1ec9fff8d577208af WatchSource:0}: Error finding container 29984f0d77174277c84fc79903d7c509a5df6bf2c77669f1ec9fff8d577208af: Status 404 returned error can't find the container with id 29984f0d77174277c84fc79903d7c509a5df6bf2c77669f1ec9fff8d577208af Jul 28 01:27:53.662842 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:53.662764 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64c4b96fd9-8vd5q" event={"ID":"6df465b5-b8f4-4b86-bbe1-c75dedf24d7a","Type":"ContainerStarted","Data":"1d88edfee93797f6d9070e745b029d1b9fce121d903e91b2870ad33309047cb1"} Jul 28 01:27:53.662842 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:53.662821 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64c4b96fd9-8vd5q" event={"ID":"6df465b5-b8f4-4b86-bbe1-c75dedf24d7a","Type":"ContainerStarted","Data":"29984f0d77174277c84fc79903d7c509a5df6bf2c77669f1ec9fff8d577208af"} Jul 28 01:27:53.684593 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:53.684528 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-64c4b96fd9-8vd5q" podStartSLOduration=0.68451324 podStartE2EDuration="684.51324ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:27:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:27:53.682602156 +0000 UTC m=+112.236726954" watchObservedRunningTime="2026-07-28 01:27:53.68451324 +0000 UTC m=+112.238638077" Jul 28 01:27:57.680431 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:57.680404 2578 generic.go:356] "Generic (PLEG): container finished" podID="b4e84366-546a-4004-b614-6b3c35c61dfe" containerID="d696346e26724c48f017cca5624800af92888ee5cfb70e030136efd63d2c606c" exitCode=0 Jul 28 01:27:57.680776 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:57.680456 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-m7p9h" event={"ID":"b4e84366-546a-4004-b614-6b3c35c61dfe","Type":"ContainerDied","Data":"d696346e26724c48f017cca5624800af92888ee5cfb70e030136efd63d2c606c"} Jul 28 01:27:57.680776 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:57.680745 2578 scope.go:117] "RemoveContainer" containerID="d696346e26724c48f017cca5624800af92888ee5cfb70e030136efd63d2c606c" Jul 28 01:27:58.686078 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:27:58.686046 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-m7p9h" event={"ID":"b4e84366-546a-4004-b614-6b3c35c61dfe","Type":"ContainerStarted","Data":"6fbbc3b352fb8ca941dd78d0d9fd453c161ba61fd4a611f926673d076f00aaab"} Jul 28 01:28:02.977772 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:02.977708 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-757879ff5c-x2xkc" podUID="31992241-f826-45d3-a26a-679d5d82465c" containerName="registry" containerID="cri-o://242629c75af41eed1e5ba5b158952524e9c68c1568129a60711b91b9ae59361c" gracePeriod=30 Jul 28 01:28:03.229230 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:03.229164 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-757879ff5c-x2xkc" Jul 28 01:28:03.349182 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:03.349155 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/31992241-f826-45d3-a26a-679d5d82465c-trusted-ca\") pod \"31992241-f826-45d3-a26a-679d5d82465c\" (UID: \"31992241-f826-45d3-a26a-679d5d82465c\") " Jul 28 01:28:03.349320 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:03.349205 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/31992241-f826-45d3-a26a-679d5d82465c-installation-pull-secrets\") pod \"31992241-f826-45d3-a26a-679d5d82465c\" (UID: \"31992241-f826-45d3-a26a-679d5d82465c\") " Jul 28 01:28:03.349320 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:03.349226 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/31992241-f826-45d3-a26a-679d5d82465c-ca-trust-extracted\") pod \"31992241-f826-45d3-a26a-679d5d82465c\" (UID: \"31992241-f826-45d3-a26a-679d5d82465c\") " Jul 28 01:28:03.349320 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:03.349253 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/31992241-f826-45d3-a26a-679d5d82465c-image-registry-private-configuration\") pod \"31992241-f826-45d3-a26a-679d5d82465c\" (UID: \"31992241-f826-45d3-a26a-679d5d82465c\") " Jul 28 01:28:03.349320 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:03.349310 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/31992241-f826-45d3-a26a-679d5d82465c-bound-sa-token\") pod \"31992241-f826-45d3-a26a-679d5d82465c\" (UID: \"31992241-f826-45d3-a26a-679d5d82465c\") " Jul 28 01:28:03.349499 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:03.349337 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k7vdf\" (UniqueName: \"kubernetes.io/projected/31992241-f826-45d3-a26a-679d5d82465c-kube-api-access-k7vdf\") pod \"31992241-f826-45d3-a26a-679d5d82465c\" (UID: \"31992241-f826-45d3-a26a-679d5d82465c\") " Jul 28 01:28:03.349499 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:03.349376 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/31992241-f826-45d3-a26a-679d5d82465c-registry-certificates\") pod \"31992241-f826-45d3-a26a-679d5d82465c\" (UID: \"31992241-f826-45d3-a26a-679d5d82465c\") " Jul 28 01:28:03.349499 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:03.349431 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/31992241-f826-45d3-a26a-679d5d82465c-registry-tls\") pod \"31992241-f826-45d3-a26a-679d5d82465c\" (UID: \"31992241-f826-45d3-a26a-679d5d82465c\") " Jul 28 01:28:03.349684 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:03.349610 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31992241-f826-45d3-a26a-679d5d82465c-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "31992241-f826-45d3-a26a-679d5d82465c" (UID: "31992241-f826-45d3-a26a-679d5d82465c"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 01:28:03.349773 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:03.349741 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/31992241-f826-45d3-a26a-679d5d82465c-trusted-ca\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:28:03.350184 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:03.350126 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31992241-f826-45d3-a26a-679d5d82465c-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "31992241-f826-45d3-a26a-679d5d82465c" (UID: "31992241-f826-45d3-a26a-679d5d82465c"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 01:28:03.351788 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:03.351746 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31992241-f826-45d3-a26a-679d5d82465c-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "31992241-f826-45d3-a26a-679d5d82465c" (UID: "31992241-f826-45d3-a26a-679d5d82465c"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 01:28:03.351788 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:03.351781 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31992241-f826-45d3-a26a-679d5d82465c-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "31992241-f826-45d3-a26a-679d5d82465c" (UID: "31992241-f826-45d3-a26a-679d5d82465c"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 01:28:03.351917 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:03.351890 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31992241-f826-45d3-a26a-679d5d82465c-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "31992241-f826-45d3-a26a-679d5d82465c" (UID: "31992241-f826-45d3-a26a-679d5d82465c"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 01:28:03.351966 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:03.351937 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31992241-f826-45d3-a26a-679d5d82465c-kube-api-access-k7vdf" (OuterVolumeSpecName: "kube-api-access-k7vdf") pod "31992241-f826-45d3-a26a-679d5d82465c" (UID: "31992241-f826-45d3-a26a-679d5d82465c"). InnerVolumeSpecName "kube-api-access-k7vdf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 01:28:03.352081 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:03.352066 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31992241-f826-45d3-a26a-679d5d82465c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "31992241-f826-45d3-a26a-679d5d82465c" (UID: "31992241-f826-45d3-a26a-679d5d82465c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 01:28:03.357741 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:03.357720 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31992241-f826-45d3-a26a-679d5d82465c-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "31992241-f826-45d3-a26a-679d5d82465c" (UID: "31992241-f826-45d3-a26a-679d5d82465c"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 01:28:03.387376 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:03.387355 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-64c4b96fd9-8vd5q" Jul 28 01:28:03.387508 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:03.387493 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-64c4b96fd9-8vd5q" Jul 28 01:28:03.392016 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:03.391998 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-64c4b96fd9-8vd5q" Jul 28 01:28:03.450274 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:03.450254 2578 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/31992241-f826-45d3-a26a-679d5d82465c-image-registry-private-configuration\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:28:03.450378 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:03.450277 2578 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/31992241-f826-45d3-a26a-679d5d82465c-bound-sa-token\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:28:03.450378 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:03.450288 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7vdf\" (UniqueName: \"kubernetes.io/projected/31992241-f826-45d3-a26a-679d5d82465c-kube-api-access-k7vdf\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:28:03.450378 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:03.450298 2578 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/31992241-f826-45d3-a26a-679d5d82465c-registry-certificates\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:28:03.450378 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:03.450307 2578 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/31992241-f826-45d3-a26a-679d5d82465c-registry-tls\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:28:03.450378 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:03.450316 2578 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/31992241-f826-45d3-a26a-679d5d82465c-installation-pull-secrets\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:28:03.450378 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:03.450324 2578 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/31992241-f826-45d3-a26a-679d5d82465c-ca-trust-extracted\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:28:03.706920 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:03.706891 2578 generic.go:356] "Generic (PLEG): container finished" podID="31992241-f826-45d3-a26a-679d5d82465c" containerID="242629c75af41eed1e5ba5b158952524e9c68c1568129a60711b91b9ae59361c" exitCode=0 Jul 28 01:28:03.707055 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:03.706938 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-757879ff5c-x2xkc" Jul 28 01:28:03.707055 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:03.706976 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-757879ff5c-x2xkc" event={"ID":"31992241-f826-45d3-a26a-679d5d82465c","Type":"ContainerDied","Data":"242629c75af41eed1e5ba5b158952524e9c68c1568129a60711b91b9ae59361c"} Jul 28 01:28:03.707055 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:03.707020 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-757879ff5c-x2xkc" event={"ID":"31992241-f826-45d3-a26a-679d5d82465c","Type":"ContainerDied","Data":"8b4da04ae3d2569e2751500eefaf54c7f84bcb423f60ac4937cb2be7460d2149"} Jul 28 01:28:03.707055 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:03.707045 2578 scope.go:117] "RemoveContainer" containerID="242629c75af41eed1e5ba5b158952524e9c68c1568129a60711b91b9ae59361c" Jul 28 01:28:03.711248 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:03.711227 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-64c4b96fd9-8vd5q" Jul 28 01:28:03.716075 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:03.716058 2578 scope.go:117] "RemoveContainer" containerID="242629c75af41eed1e5ba5b158952524e9c68c1568129a60711b91b9ae59361c" Jul 28 01:28:03.716329 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:28:03.716305 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"242629c75af41eed1e5ba5b158952524e9c68c1568129a60711b91b9ae59361c\": container with ID starting with 242629c75af41eed1e5ba5b158952524e9c68c1568129a60711b91b9ae59361c not found: ID does not exist" containerID="242629c75af41eed1e5ba5b158952524e9c68c1568129a60711b91b9ae59361c" Jul 28 01:28:03.716389 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:03.716335 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"242629c75af41eed1e5ba5b158952524e9c68c1568129a60711b91b9ae59361c"} err="failed to get container status \"242629c75af41eed1e5ba5b158952524e9c68c1568129a60711b91b9ae59361c\": rpc error: code = NotFound desc = could not find container \"242629c75af41eed1e5ba5b158952524e9c68c1568129a60711b91b9ae59361c\": container with ID starting with 242629c75af41eed1e5ba5b158952524e9c68c1568129a60711b91b9ae59361c not found: ID does not exist" Jul 28 01:28:03.744763 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:03.744744 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-757879ff5c-x2xkc"] Jul 28 01:28:03.748839 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:03.748818 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-757879ff5c-x2xkc"] Jul 28 01:28:03.759497 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:03.759475 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57bb8cdcd4-8l59f"] Jul 28 01:28:04.009220 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:04.009189 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31992241-f826-45d3-a26a-679d5d82465c" path="/var/lib/kubelet/pods/31992241-f826-45d3-a26a-679d5d82465c/volumes" Jul 28 01:28:11.740690 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:11.740659 2578 generic.go:356] "Generic (PLEG): container finished" podID="18e29850-170d-4a22-b890-95237b45246d" containerID="85115ddad03ce821d6c8694458d557811d29c5b8b52af002330528ccc3ad7c5a" exitCode=0 Jul 28 01:28:11.741078 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:11.740725 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-q8ldl" event={"ID":"18e29850-170d-4a22-b890-95237b45246d","Type":"ContainerDied","Data":"85115ddad03ce821d6c8694458d557811d29c5b8b52af002330528ccc3ad7c5a"} Jul 28 01:28:11.741078 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:11.741017 2578 scope.go:117] "RemoveContainer" containerID="85115ddad03ce821d6c8694458d557811d29c5b8b52af002330528ccc3ad7c5a" Jul 28 01:28:12.746461 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:12.746424 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-q8ldl" event={"ID":"18e29850-170d-4a22-b890-95237b45246d","Type":"ContainerStarted","Data":"1e71fb970709cf5238c526970b6e628a119e5249f381c011d270990005a754ab"} Jul 28 01:28:17.773693 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:17.773654 2578 generic.go:356] "Generic (PLEG): container finished" podID="afab8f75-0a1a-43a3-9a98-be26e28ab36f" containerID="7fc70f009a95d7ad0140b60eecbb66f6b6082374780112020192a807031ca676" exitCode=0 Jul 28 01:28:17.774070 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:17.773722 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-phhjx" event={"ID":"afab8f75-0a1a-43a3-9a98-be26e28ab36f","Type":"ContainerDied","Data":"7fc70f009a95d7ad0140b60eecbb66f6b6082374780112020192a807031ca676"} Jul 28 01:28:17.774125 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:17.774083 2578 scope.go:117] "RemoveContainer" containerID="7fc70f009a95d7ad0140b60eecbb66f6b6082374780112020192a807031ca676" Jul 28 01:28:18.779625 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:18.779592 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-phhjx" event={"ID":"afab8f75-0a1a-43a3-9a98-be26e28ab36f","Type":"ContainerStarted","Data":"78bc93388464ee23b759cec58264bc26aa5785e9629805cfd0fe7f2ce9a008e6"} Jul 28 01:28:28.780789 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:28.780753 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-57bb8cdcd4-8l59f" podUID="1b43f6b8-dabc-40de-944b-25145570ed23" containerName="console" containerID="cri-o://527260c0f0a2b01c91231fe7e4d9b80496895a685b44effc91d3b2928ec35187" gracePeriod=15 Jul 28 01:28:29.013501 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:29.013480 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57bb8cdcd4-8l59f" Jul 28 01:28:29.057038 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:29.056979 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1b43f6b8-dabc-40de-944b-25145570ed23-service-ca\") pod \"1b43f6b8-dabc-40de-944b-25145570ed23\" (UID: \"1b43f6b8-dabc-40de-944b-25145570ed23\") " Jul 28 01:28:29.057038 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:29.057025 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vx2x8\" (UniqueName: \"kubernetes.io/projected/1b43f6b8-dabc-40de-944b-25145570ed23-kube-api-access-vx2x8\") pod \"1b43f6b8-dabc-40de-944b-25145570ed23\" (UID: \"1b43f6b8-dabc-40de-944b-25145570ed23\") " Jul 28 01:28:29.057214 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:29.057055 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1b43f6b8-dabc-40de-944b-25145570ed23-console-oauth-config\") pod \"1b43f6b8-dabc-40de-944b-25145570ed23\" (UID: \"1b43f6b8-dabc-40de-944b-25145570ed23\") " Jul 28 01:28:29.057214 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:29.057117 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b43f6b8-dabc-40de-944b-25145570ed23-trusted-ca-bundle\") pod \"1b43f6b8-dabc-40de-944b-25145570ed23\" (UID: \"1b43f6b8-dabc-40de-944b-25145570ed23\") " Jul 28 01:28:29.057214 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:29.057165 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b43f6b8-dabc-40de-944b-25145570ed23-console-serving-cert\") pod \"1b43f6b8-dabc-40de-944b-25145570ed23\" (UID: \"1b43f6b8-dabc-40de-944b-25145570ed23\") " Jul 28 01:28:29.057214 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:29.057197 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1b43f6b8-dabc-40de-944b-25145570ed23-oauth-serving-cert\") pod \"1b43f6b8-dabc-40de-944b-25145570ed23\" (UID: \"1b43f6b8-dabc-40de-944b-25145570ed23\") " Jul 28 01:28:29.057420 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:29.057227 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1b43f6b8-dabc-40de-944b-25145570ed23-console-config\") pod \"1b43f6b8-dabc-40de-944b-25145570ed23\" (UID: \"1b43f6b8-dabc-40de-944b-25145570ed23\") " Jul 28 01:28:29.057420 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:29.057335 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b43f6b8-dabc-40de-944b-25145570ed23-service-ca" (OuterVolumeSpecName: "service-ca") pod "1b43f6b8-dabc-40de-944b-25145570ed23" (UID: "1b43f6b8-dabc-40de-944b-25145570ed23"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 01:28:29.057660 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:29.057622 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b43f6b8-dabc-40de-944b-25145570ed23-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1b43f6b8-dabc-40de-944b-25145570ed23" (UID: "1b43f6b8-dabc-40de-944b-25145570ed23"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 01:28:29.057761 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:29.057677 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b43f6b8-dabc-40de-944b-25145570ed23-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1b43f6b8-dabc-40de-944b-25145570ed23" (UID: "1b43f6b8-dabc-40de-944b-25145570ed23"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 01:28:29.057761 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:29.057736 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b43f6b8-dabc-40de-944b-25145570ed23-console-config" (OuterVolumeSpecName: "console-config") pod "1b43f6b8-dabc-40de-944b-25145570ed23" (UID: "1b43f6b8-dabc-40de-944b-25145570ed23"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 01:28:29.057874 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:29.057819 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b43f6b8-dabc-40de-944b-25145570ed23-trusted-ca-bundle\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:28:29.057874 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:29.057835 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1b43f6b8-dabc-40de-944b-25145570ed23-oauth-serving-cert\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:28:29.057874 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:29.057851 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1b43f6b8-dabc-40de-944b-25145570ed23-console-config\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:28:29.057874 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:29.057864 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1b43f6b8-dabc-40de-944b-25145570ed23-service-ca\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:28:29.059317 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:29.059288 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b43f6b8-dabc-40de-944b-25145570ed23-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1b43f6b8-dabc-40de-944b-25145570ed23" (UID: "1b43f6b8-dabc-40de-944b-25145570ed23"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 01:28:29.059519 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:29.059503 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b43f6b8-dabc-40de-944b-25145570ed23-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1b43f6b8-dabc-40de-944b-25145570ed23" (UID: "1b43f6b8-dabc-40de-944b-25145570ed23"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 01:28:29.059617 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:29.059557 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b43f6b8-dabc-40de-944b-25145570ed23-kube-api-access-vx2x8" (OuterVolumeSpecName: "kube-api-access-vx2x8") pod "1b43f6b8-dabc-40de-944b-25145570ed23" (UID: "1b43f6b8-dabc-40de-944b-25145570ed23"). InnerVolumeSpecName "kube-api-access-vx2x8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 01:28:29.158376 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:29.158353 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1b43f6b8-dabc-40de-944b-25145570ed23-console-oauth-config\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:28:29.158376 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:29.158374 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b43f6b8-dabc-40de-944b-25145570ed23-console-serving-cert\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:28:29.158504 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:29.158383 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vx2x8\" (UniqueName: \"kubernetes.io/projected/1b43f6b8-dabc-40de-944b-25145570ed23-kube-api-access-vx2x8\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:28:29.828736 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:29.828701 2578 generic.go:356] "Generic (PLEG): container finished" podID="1b43f6b8-dabc-40de-944b-25145570ed23" containerID="527260c0f0a2b01c91231fe7e4d9b80496895a685b44effc91d3b2928ec35187" exitCode=0 Jul 28 01:28:29.829129 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:29.828749 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57bb8cdcd4-8l59f" Jul 28 01:28:29.829129 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:29.828760 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57bb8cdcd4-8l59f" event={"ID":"1b43f6b8-dabc-40de-944b-25145570ed23","Type":"ContainerDied","Data":"527260c0f0a2b01c91231fe7e4d9b80496895a685b44effc91d3b2928ec35187"} Jul 28 01:28:29.829129 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:29.828809 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57bb8cdcd4-8l59f" event={"ID":"1b43f6b8-dabc-40de-944b-25145570ed23","Type":"ContainerDied","Data":"be5cd286066cae1f139bb01ed38f08088891277bda55272f1587b810614e3275"} Jul 28 01:28:29.829129 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:29.828836 2578 scope.go:117] "RemoveContainer" containerID="527260c0f0a2b01c91231fe7e4d9b80496895a685b44effc91d3b2928ec35187" Jul 28 01:28:29.837820 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:29.837798 2578 scope.go:117] "RemoveContainer" containerID="527260c0f0a2b01c91231fe7e4d9b80496895a685b44effc91d3b2928ec35187" Jul 28 01:28:29.838087 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:28:29.838067 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"527260c0f0a2b01c91231fe7e4d9b80496895a685b44effc91d3b2928ec35187\": container with ID starting with 527260c0f0a2b01c91231fe7e4d9b80496895a685b44effc91d3b2928ec35187 not found: ID does not exist" containerID="527260c0f0a2b01c91231fe7e4d9b80496895a685b44effc91d3b2928ec35187" Jul 28 01:28:29.838131 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:29.838096 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"527260c0f0a2b01c91231fe7e4d9b80496895a685b44effc91d3b2928ec35187"} err="failed to get container status \"527260c0f0a2b01c91231fe7e4d9b80496895a685b44effc91d3b2928ec35187\": rpc error: code = NotFound desc = could not find container \"527260c0f0a2b01c91231fe7e4d9b80496895a685b44effc91d3b2928ec35187\": container with ID starting with 527260c0f0a2b01c91231fe7e4d9b80496895a685b44effc91d3b2928ec35187 not found: ID does not exist" Jul 28 01:28:29.850017 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:29.849994 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57bb8cdcd4-8l59f"] Jul 28 01:28:29.853731 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:29.853712 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-57bb8cdcd4-8l59f"] Jul 28 01:28:30.010172 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:30.010137 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b43f6b8-dabc-40de-944b-25145570ed23" path="/var/lib/kubelet/pods/1b43f6b8-dabc-40de-944b-25145570ed23/volumes" Jul 28 01:28:57.986802 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:57.986723 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 01:28:57.987319 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:57.986986 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5ee77ea9-96ed-40bc-8169-f937717e4fa5" containerName="alertmanager" containerID="cri-o://8d443e0d2dcb90422cedf500a702235a98e4b46ca9acfa7fb86b0df364b3ee57" gracePeriod=120 Jul 28 01:28:57.987319 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:57.987020 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5ee77ea9-96ed-40bc-8169-f937717e4fa5" containerName="kube-rbac-proxy-metric" containerID="cri-o://0e06f08cb7fc2a37e3c0f8483a60b414bad7352d567e0b17df09d0ee25eed895" gracePeriod=120 Jul 28 01:28:57.987319 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:57.987032 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5ee77ea9-96ed-40bc-8169-f937717e4fa5" containerName="kube-rbac-proxy" containerID="cri-o://58ae96a0f7fecb3a5e246cbee267e45082830b63cbb0e60bef67e013ae80a069" gracePeriod=120 Jul 28 01:28:57.987319 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:57.987051 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5ee77ea9-96ed-40bc-8169-f937717e4fa5" containerName="kube-rbac-proxy-web" containerID="cri-o://3fd45d513856778bc013daafa3566a777ee75087d7ef42d1307e7a5848fa5a6e" gracePeriod=120 Jul 28 01:28:57.987319 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:57.987064 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5ee77ea9-96ed-40bc-8169-f937717e4fa5" containerName="config-reloader" containerID="cri-o://81baa42daa247159ef19313af55b8637910318b62da81e6fa62f499a232b5062" gracePeriod=120 Jul 28 01:28:57.987319 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:57.987006 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5ee77ea9-96ed-40bc-8169-f937717e4fa5" containerName="prom-label-proxy" containerID="cri-o://770b7f5bf8166a25ad1ac58f4440d07c850d8fff652792226757105ca257422c" gracePeriod=120 Jul 28 01:28:58.950994 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:58.950964 2578 generic.go:356] "Generic (PLEG): container finished" podID="5ee77ea9-96ed-40bc-8169-f937717e4fa5" containerID="770b7f5bf8166a25ad1ac58f4440d07c850d8fff652792226757105ca257422c" exitCode=0 Jul 28 01:28:58.950994 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:58.950988 2578 generic.go:356] "Generic (PLEG): container finished" podID="5ee77ea9-96ed-40bc-8169-f937717e4fa5" containerID="58ae96a0f7fecb3a5e246cbee267e45082830b63cbb0e60bef67e013ae80a069" exitCode=0 Jul 28 01:28:58.950994 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:58.950995 2578 generic.go:356] "Generic (PLEG): container finished" podID="5ee77ea9-96ed-40bc-8169-f937717e4fa5" containerID="81baa42daa247159ef19313af55b8637910318b62da81e6fa62f499a232b5062" exitCode=0 Jul 28 01:28:58.950994 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:58.951000 2578 generic.go:356] "Generic (PLEG): container finished" podID="5ee77ea9-96ed-40bc-8169-f937717e4fa5" containerID="8d443e0d2dcb90422cedf500a702235a98e4b46ca9acfa7fb86b0df364b3ee57" exitCode=0 Jul 28 01:28:58.951312 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:58.951041 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ee77ea9-96ed-40bc-8169-f937717e4fa5","Type":"ContainerDied","Data":"770b7f5bf8166a25ad1ac58f4440d07c850d8fff652792226757105ca257422c"} Jul 28 01:28:58.951312 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:58.951107 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ee77ea9-96ed-40bc-8169-f937717e4fa5","Type":"ContainerDied","Data":"58ae96a0f7fecb3a5e246cbee267e45082830b63cbb0e60bef67e013ae80a069"} Jul 28 01:28:58.951312 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:58.951135 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ee77ea9-96ed-40bc-8169-f937717e4fa5","Type":"ContainerDied","Data":"81baa42daa247159ef19313af55b8637910318b62da81e6fa62f499a232b5062"} Jul 28 01:28:58.951312 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:58.951159 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ee77ea9-96ed-40bc-8169-f937717e4fa5","Type":"ContainerDied","Data":"8d443e0d2dcb90422cedf500a702235a98e4b46ca9acfa7fb86b0df364b3ee57"} Jul 28 01:28:59.223528 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.223503 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:28:59.305635 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.305606 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vzt5f\" (UniqueName: \"kubernetes.io/projected/5ee77ea9-96ed-40bc-8169-f937717e4fa5-kube-api-access-vzt5f\") pod \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " Jul 28 01:28:59.305790 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.305661 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5ee77ea9-96ed-40bc-8169-f937717e4fa5-config-out\") pod \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " Jul 28 01:28:59.305790 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.305695 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-secret-alertmanager-kube-rbac-proxy-web\") pod \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " Jul 28 01:28:59.305790 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.305717 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-secret-alertmanager-main-tls\") pod \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " Jul 28 01:28:59.305790 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.305739 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-config-volume\") pod \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " Jul 28 01:28:59.305790 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.305765 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5ee77ea9-96ed-40bc-8169-f937717e4fa5-tls-assets\") pod \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " Jul 28 01:28:59.306058 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.305822 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-cluster-tls-config\") pod \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " Jul 28 01:28:59.306058 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.305859 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-web-config\") pod \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " Jul 28 01:28:59.306058 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.305918 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-secret-alertmanager-kube-rbac-proxy\") pod \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " Jul 28 01:28:59.306058 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.305948 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ee77ea9-96ed-40bc-8169-f937717e4fa5-alertmanager-trusted-ca-bundle\") pod \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " Jul 28 01:28:59.306058 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.305980 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " Jul 28 01:28:59.306058 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.306009 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5ee77ea9-96ed-40bc-8169-f937717e4fa5-alertmanager-main-db\") pod \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " Jul 28 01:28:59.306058 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.306033 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ee77ea9-96ed-40bc-8169-f937717e4fa5-metrics-client-ca\") pod \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\" (UID: \"5ee77ea9-96ed-40bc-8169-f937717e4fa5\") " Jul 28 01:28:59.306697 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.306427 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ee77ea9-96ed-40bc-8169-f937717e4fa5-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "5ee77ea9-96ed-40bc-8169-f937717e4fa5" (UID: "5ee77ea9-96ed-40bc-8169-f937717e4fa5"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 01:28:59.306697 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.306629 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ee77ea9-96ed-40bc-8169-f937717e4fa5-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "5ee77ea9-96ed-40bc-8169-f937717e4fa5" (UID: "5ee77ea9-96ed-40bc-8169-f937717e4fa5"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 01:28:59.307007 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.306983 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ee77ea9-96ed-40bc-8169-f937717e4fa5-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "5ee77ea9-96ed-40bc-8169-f937717e4fa5" (UID: "5ee77ea9-96ed-40bc-8169-f937717e4fa5"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 01:28:59.309066 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.309044 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ee77ea9-96ed-40bc-8169-f937717e4fa5-kube-api-access-vzt5f" (OuterVolumeSpecName: "kube-api-access-vzt5f") pod "5ee77ea9-96ed-40bc-8169-f937717e4fa5" (UID: "5ee77ea9-96ed-40bc-8169-f937717e4fa5"). InnerVolumeSpecName "kube-api-access-vzt5f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 01:28:59.309565 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.309512 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ee77ea9-96ed-40bc-8169-f937717e4fa5-config-out" (OuterVolumeSpecName: "config-out") pod "5ee77ea9-96ed-40bc-8169-f937717e4fa5" (UID: "5ee77ea9-96ed-40bc-8169-f937717e4fa5"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 01:28:59.309565 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.309518 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ee77ea9-96ed-40bc-8169-f937717e4fa5-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "5ee77ea9-96ed-40bc-8169-f937717e4fa5" (UID: "5ee77ea9-96ed-40bc-8169-f937717e4fa5"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 01:28:59.310175 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.310152 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "5ee77ea9-96ed-40bc-8169-f937717e4fa5" (UID: "5ee77ea9-96ed-40bc-8169-f937717e4fa5"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 01:28:59.310778 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.310705 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "5ee77ea9-96ed-40bc-8169-f937717e4fa5" (UID: "5ee77ea9-96ed-40bc-8169-f937717e4fa5"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 01:28:59.310778 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.310723 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "5ee77ea9-96ed-40bc-8169-f937717e4fa5" (UID: "5ee77ea9-96ed-40bc-8169-f937717e4fa5"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 01:28:59.310778 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.310739 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-config-volume" (OuterVolumeSpecName: "config-volume") pod "5ee77ea9-96ed-40bc-8169-f937717e4fa5" (UID: "5ee77ea9-96ed-40bc-8169-f937717e4fa5"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 01:28:59.310778 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.310767 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "5ee77ea9-96ed-40bc-8169-f937717e4fa5" (UID: "5ee77ea9-96ed-40bc-8169-f937717e4fa5"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 01:28:59.317487 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.317464 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "5ee77ea9-96ed-40bc-8169-f937717e4fa5" (UID: "5ee77ea9-96ed-40bc-8169-f937717e4fa5"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 01:28:59.324095 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.324071 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-web-config" (OuterVolumeSpecName: "web-config") pod "5ee77ea9-96ed-40bc-8169-f937717e4fa5" (UID: "5ee77ea9-96ed-40bc-8169-f937717e4fa5"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 01:28:59.406763 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.406724 2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:28:59.406763 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.406758 2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-secret-alertmanager-main-tls\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:28:59.406763 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.406770 2578 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-config-volume\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:28:59.406763 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.406778 2578 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5ee77ea9-96ed-40bc-8169-f937717e4fa5-tls-assets\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:28:59.406964 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.406786 2578 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-cluster-tls-config\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:28:59.406964 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.406794 2578 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-web-config\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:28:59.406964 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.406802 2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:28:59.406964 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.406810 2578 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ee77ea9-96ed-40bc-8169-f937717e4fa5-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:28:59.406964 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.406818 2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5ee77ea9-96ed-40bc-8169-f937717e4fa5-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:28:59.406964 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.406827 2578 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5ee77ea9-96ed-40bc-8169-f937717e4fa5-alertmanager-main-db\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:28:59.406964 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.406835 2578 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ee77ea9-96ed-40bc-8169-f937717e4fa5-metrics-client-ca\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:28:59.406964 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.406843 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzt5f\" (UniqueName: \"kubernetes.io/projected/5ee77ea9-96ed-40bc-8169-f937717e4fa5-kube-api-access-vzt5f\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:28:59.406964 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.406851 2578 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5ee77ea9-96ed-40bc-8169-f937717e4fa5-config-out\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:28:59.958319 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.958286 2578 generic.go:356] "Generic (PLEG): container finished" podID="5ee77ea9-96ed-40bc-8169-f937717e4fa5" containerID="0e06f08cb7fc2a37e3c0f8483a60b414bad7352d567e0b17df09d0ee25eed895" exitCode=0 Jul 28 01:28:59.958319 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.958312 2578 generic.go:356] "Generic (PLEG): container finished" podID="5ee77ea9-96ed-40bc-8169-f937717e4fa5" containerID="3fd45d513856778bc013daafa3566a777ee75087d7ef42d1307e7a5848fa5a6e" exitCode=0 Jul 28 01:28:59.958486 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.958348 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ee77ea9-96ed-40bc-8169-f937717e4fa5","Type":"ContainerDied","Data":"0e06f08cb7fc2a37e3c0f8483a60b414bad7352d567e0b17df09d0ee25eed895"} Jul 28 01:28:59.958486 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.958359 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:28:59.958486 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.958405 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ee77ea9-96ed-40bc-8169-f937717e4fa5","Type":"ContainerDied","Data":"3fd45d513856778bc013daafa3566a777ee75087d7ef42d1307e7a5848fa5a6e"} Jul 28 01:28:59.958486 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.958432 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ee77ea9-96ed-40bc-8169-f937717e4fa5","Type":"ContainerDied","Data":"0b87a3126c1fb19d2f6d6df7579c4256725479dcae7d786db78352f64137de85"} Jul 28 01:28:59.958486 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.958464 2578 scope.go:117] "RemoveContainer" containerID="770b7f5bf8166a25ad1ac58f4440d07c850d8fff652792226757105ca257422c" Jul 28 01:28:59.967198 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.967177 2578 scope.go:117] "RemoveContainer" containerID="0e06f08cb7fc2a37e3c0f8483a60b414bad7352d567e0b17df09d0ee25eed895" Jul 28 01:28:59.974034 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.974019 2578 scope.go:117] "RemoveContainer" containerID="58ae96a0f7fecb3a5e246cbee267e45082830b63cbb0e60bef67e013ae80a069" Jul 28 01:28:59.980896 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.980877 2578 scope.go:117] "RemoveContainer" containerID="3fd45d513856778bc013daafa3566a777ee75087d7ef42d1307e7a5848fa5a6e" Jul 28 01:28:59.981053 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.981037 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 01:28:59.985983 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.985961 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 01:28:59.988053 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.988038 2578 scope.go:117] "RemoveContainer" containerID="81baa42daa247159ef19313af55b8637910318b62da81e6fa62f499a232b5062" Jul 28 01:28:59.994507 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:28:59.994487 2578 scope.go:117] "RemoveContainer" containerID="8d443e0d2dcb90422cedf500a702235a98e4b46ca9acfa7fb86b0df364b3ee57" Jul 28 01:29:00.001346 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.001329 2578 scope.go:117] "RemoveContainer" containerID="d7b6dff65da4c81d0c28f380fcaa9c05b20813cdf96098212d10433718e562a0" Jul 28 01:29:00.008739 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.008725 2578 scope.go:117] "RemoveContainer" containerID="770b7f5bf8166a25ad1ac58f4440d07c850d8fff652792226757105ca257422c" Jul 28 01:29:00.008987 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:29:00.008963 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"770b7f5bf8166a25ad1ac58f4440d07c850d8fff652792226757105ca257422c\": container with ID starting with 770b7f5bf8166a25ad1ac58f4440d07c850d8fff652792226757105ca257422c not found: ID does not exist" containerID="770b7f5bf8166a25ad1ac58f4440d07c850d8fff652792226757105ca257422c" Jul 28 01:29:00.009045 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.008999 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"770b7f5bf8166a25ad1ac58f4440d07c850d8fff652792226757105ca257422c"} err="failed to get container status \"770b7f5bf8166a25ad1ac58f4440d07c850d8fff652792226757105ca257422c\": rpc error: code = NotFound desc = could not find container \"770b7f5bf8166a25ad1ac58f4440d07c850d8fff652792226757105ca257422c\": container with ID starting with 770b7f5bf8166a25ad1ac58f4440d07c850d8fff652792226757105ca257422c not found: ID does not exist" Jul 28 01:29:00.009045 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.009022 2578 scope.go:117] "RemoveContainer" containerID="0e06f08cb7fc2a37e3c0f8483a60b414bad7352d567e0b17df09d0ee25eed895" Jul 28 01:29:00.009295 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:29:00.009276 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e06f08cb7fc2a37e3c0f8483a60b414bad7352d567e0b17df09d0ee25eed895\": container with ID starting with 0e06f08cb7fc2a37e3c0f8483a60b414bad7352d567e0b17df09d0ee25eed895 not found: ID does not exist" containerID="0e06f08cb7fc2a37e3c0f8483a60b414bad7352d567e0b17df09d0ee25eed895" Jul 28 01:29:00.009383 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.009301 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e06f08cb7fc2a37e3c0f8483a60b414bad7352d567e0b17df09d0ee25eed895"} err="failed to get container status \"0e06f08cb7fc2a37e3c0f8483a60b414bad7352d567e0b17df09d0ee25eed895\": rpc error: code = NotFound desc = could not find container \"0e06f08cb7fc2a37e3c0f8483a60b414bad7352d567e0b17df09d0ee25eed895\": container with ID starting with 0e06f08cb7fc2a37e3c0f8483a60b414bad7352d567e0b17df09d0ee25eed895 not found: ID does not exist" Jul 28 01:29:00.009383 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.009317 2578 scope.go:117] "RemoveContainer" containerID="58ae96a0f7fecb3a5e246cbee267e45082830b63cbb0e60bef67e013ae80a069" Jul 28 01:29:00.009566 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:29:00.009532 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58ae96a0f7fecb3a5e246cbee267e45082830b63cbb0e60bef67e013ae80a069\": container with ID starting with 58ae96a0f7fecb3a5e246cbee267e45082830b63cbb0e60bef67e013ae80a069 not found: ID does not exist" containerID="58ae96a0f7fecb3a5e246cbee267e45082830b63cbb0e60bef67e013ae80a069" Jul 28 01:29:00.009633 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.009575 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58ae96a0f7fecb3a5e246cbee267e45082830b63cbb0e60bef67e013ae80a069"} err="failed to get container status \"58ae96a0f7fecb3a5e246cbee267e45082830b63cbb0e60bef67e013ae80a069\": rpc error: code = NotFound desc = could not find container \"58ae96a0f7fecb3a5e246cbee267e45082830b63cbb0e60bef67e013ae80a069\": container with ID starting with 58ae96a0f7fecb3a5e246cbee267e45082830b63cbb0e60bef67e013ae80a069 not found: ID does not exist" Jul 28 01:29:00.009633 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.009595 2578 scope.go:117] "RemoveContainer" containerID="3fd45d513856778bc013daafa3566a777ee75087d7ef42d1307e7a5848fa5a6e" Jul 28 01:29:00.009882 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:29:00.009866 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fd45d513856778bc013daafa3566a777ee75087d7ef42d1307e7a5848fa5a6e\": container with ID starting with 3fd45d513856778bc013daafa3566a777ee75087d7ef42d1307e7a5848fa5a6e not found: ID does not exist" containerID="3fd45d513856778bc013daafa3566a777ee75087d7ef42d1307e7a5848fa5a6e" Jul 28 01:29:00.009923 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.009885 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ee77ea9-96ed-40bc-8169-f937717e4fa5" path="/var/lib/kubelet/pods/5ee77ea9-96ed-40bc-8169-f937717e4fa5/volumes" Jul 28 01:29:00.009994 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.009887 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fd45d513856778bc013daafa3566a777ee75087d7ef42d1307e7a5848fa5a6e"} err="failed to get container status \"3fd45d513856778bc013daafa3566a777ee75087d7ef42d1307e7a5848fa5a6e\": rpc error: code = NotFound desc = could not find container \"3fd45d513856778bc013daafa3566a777ee75087d7ef42d1307e7a5848fa5a6e\": container with ID starting with 3fd45d513856778bc013daafa3566a777ee75087d7ef42d1307e7a5848fa5a6e not found: ID does not exist" Jul 28 01:29:00.009994 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.009968 2578 scope.go:117] "RemoveContainer" containerID="81baa42daa247159ef19313af55b8637910318b62da81e6fa62f499a232b5062" Jul 28 01:29:00.010269 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:29:00.010220 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81baa42daa247159ef19313af55b8637910318b62da81e6fa62f499a232b5062\": container with ID starting with 81baa42daa247159ef19313af55b8637910318b62da81e6fa62f499a232b5062 not found: ID does not exist" containerID="81baa42daa247159ef19313af55b8637910318b62da81e6fa62f499a232b5062" Jul 28 01:29:00.010394 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.010266 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81baa42daa247159ef19313af55b8637910318b62da81e6fa62f499a232b5062"} err="failed to get container status \"81baa42daa247159ef19313af55b8637910318b62da81e6fa62f499a232b5062\": rpc error: code = NotFound desc = could not find container \"81baa42daa247159ef19313af55b8637910318b62da81e6fa62f499a232b5062\": container with ID starting with 81baa42daa247159ef19313af55b8637910318b62da81e6fa62f499a232b5062 not found: ID does not exist" Jul 28 01:29:00.010394 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.010286 2578 scope.go:117] "RemoveContainer" containerID="8d443e0d2dcb90422cedf500a702235a98e4b46ca9acfa7fb86b0df364b3ee57" Jul 28 01:29:00.010986 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:29:00.010965 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d443e0d2dcb90422cedf500a702235a98e4b46ca9acfa7fb86b0df364b3ee57\": container with ID starting with 8d443e0d2dcb90422cedf500a702235a98e4b46ca9acfa7fb86b0df364b3ee57 not found: ID does not exist" containerID="8d443e0d2dcb90422cedf500a702235a98e4b46ca9acfa7fb86b0df364b3ee57" Jul 28 01:29:00.011101 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.010994 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d443e0d2dcb90422cedf500a702235a98e4b46ca9acfa7fb86b0df364b3ee57"} err="failed to get container status \"8d443e0d2dcb90422cedf500a702235a98e4b46ca9acfa7fb86b0df364b3ee57\": rpc error: code = NotFound desc = could not find container \"8d443e0d2dcb90422cedf500a702235a98e4b46ca9acfa7fb86b0df364b3ee57\": container with ID starting with 8d443e0d2dcb90422cedf500a702235a98e4b46ca9acfa7fb86b0df364b3ee57 not found: ID does not exist" Jul 28 01:29:00.011101 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.011015 2578 scope.go:117] "RemoveContainer" containerID="d7b6dff65da4c81d0c28f380fcaa9c05b20813cdf96098212d10433718e562a0" Jul 28 01:29:00.011286 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:29:00.011269 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7b6dff65da4c81d0c28f380fcaa9c05b20813cdf96098212d10433718e562a0\": container with ID starting with d7b6dff65da4c81d0c28f380fcaa9c05b20813cdf96098212d10433718e562a0 not found: ID does not exist" containerID="d7b6dff65da4c81d0c28f380fcaa9c05b20813cdf96098212d10433718e562a0" Jul 28 01:29:00.011348 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.011295 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7b6dff65da4c81d0c28f380fcaa9c05b20813cdf96098212d10433718e562a0"} err="failed to get container status \"d7b6dff65da4c81d0c28f380fcaa9c05b20813cdf96098212d10433718e562a0\": rpc error: code = NotFound desc = could not find container \"d7b6dff65da4c81d0c28f380fcaa9c05b20813cdf96098212d10433718e562a0\": container with ID starting with d7b6dff65da4c81d0c28f380fcaa9c05b20813cdf96098212d10433718e562a0 not found: ID does not exist" Jul 28 01:29:00.011348 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.011314 2578 scope.go:117] "RemoveContainer" containerID="770b7f5bf8166a25ad1ac58f4440d07c850d8fff652792226757105ca257422c" Jul 28 01:29:00.011607 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.011585 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"770b7f5bf8166a25ad1ac58f4440d07c850d8fff652792226757105ca257422c"} err="failed to get container status \"770b7f5bf8166a25ad1ac58f4440d07c850d8fff652792226757105ca257422c\": rpc error: code = NotFound desc = could not find container \"770b7f5bf8166a25ad1ac58f4440d07c850d8fff652792226757105ca257422c\": container with ID starting with 770b7f5bf8166a25ad1ac58f4440d07c850d8fff652792226757105ca257422c not found: ID does not exist" Jul 28 01:29:00.011607 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.011606 2578 scope.go:117] "RemoveContainer" containerID="0e06f08cb7fc2a37e3c0f8483a60b414bad7352d567e0b17df09d0ee25eed895" Jul 28 01:29:00.011844 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.011820 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e06f08cb7fc2a37e3c0f8483a60b414bad7352d567e0b17df09d0ee25eed895"} err="failed to get container status \"0e06f08cb7fc2a37e3c0f8483a60b414bad7352d567e0b17df09d0ee25eed895\": rpc error: code = NotFound desc = could not find container \"0e06f08cb7fc2a37e3c0f8483a60b414bad7352d567e0b17df09d0ee25eed895\": container with ID starting with 0e06f08cb7fc2a37e3c0f8483a60b414bad7352d567e0b17df09d0ee25eed895 not found: ID does not exist" Jul 28 01:29:00.011895 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.011845 2578 scope.go:117] "RemoveContainer" containerID="58ae96a0f7fecb3a5e246cbee267e45082830b63cbb0e60bef67e013ae80a069" Jul 28 01:29:00.012078 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.012058 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58ae96a0f7fecb3a5e246cbee267e45082830b63cbb0e60bef67e013ae80a069"} err="failed to get container status \"58ae96a0f7fecb3a5e246cbee267e45082830b63cbb0e60bef67e013ae80a069\": rpc error: code = NotFound desc = could not find container \"58ae96a0f7fecb3a5e246cbee267e45082830b63cbb0e60bef67e013ae80a069\": container with ID starting with 58ae96a0f7fecb3a5e246cbee267e45082830b63cbb0e60bef67e013ae80a069 not found: ID does not exist" Jul 28 01:29:00.012078 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.012078 2578 scope.go:117] "RemoveContainer" containerID="3fd45d513856778bc013daafa3566a777ee75087d7ef42d1307e7a5848fa5a6e" Jul 28 01:29:00.012329 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.012311 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fd45d513856778bc013daafa3566a777ee75087d7ef42d1307e7a5848fa5a6e"} err="failed to get container status \"3fd45d513856778bc013daafa3566a777ee75087d7ef42d1307e7a5848fa5a6e\": rpc error: code = NotFound desc = could not find container \"3fd45d513856778bc013daafa3566a777ee75087d7ef42d1307e7a5848fa5a6e\": container with ID starting with 3fd45d513856778bc013daafa3566a777ee75087d7ef42d1307e7a5848fa5a6e not found: ID does not exist" Jul 28 01:29:00.012376 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.012332 2578 scope.go:117] "RemoveContainer" containerID="81baa42daa247159ef19313af55b8637910318b62da81e6fa62f499a232b5062" Jul 28 01:29:00.012424 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.012375 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 01:29:00.012583 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.012565 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81baa42daa247159ef19313af55b8637910318b62da81e6fa62f499a232b5062"} err="failed to get container status \"81baa42daa247159ef19313af55b8637910318b62da81e6fa62f499a232b5062\": rpc error: code = NotFound desc = could not find container \"81baa42daa247159ef19313af55b8637910318b62da81e6fa62f499a232b5062\": container with ID starting with 81baa42daa247159ef19313af55b8637910318b62da81e6fa62f499a232b5062 not found: ID does not exist" Jul 28 01:29:00.012641 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.012583 2578 scope.go:117] "RemoveContainer" containerID="8d443e0d2dcb90422cedf500a702235a98e4b46ca9acfa7fb86b0df364b3ee57" Jul 28 01:29:00.012698 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.012686 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ee77ea9-96ed-40bc-8169-f937717e4fa5" containerName="kube-rbac-proxy-metric" Jul 28 01:29:00.012732 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.012700 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ee77ea9-96ed-40bc-8169-f937717e4fa5" containerName="kube-rbac-proxy-metric" Jul 28 01:29:00.012732 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.012711 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b43f6b8-dabc-40de-944b-25145570ed23" containerName="console" Jul 28 01:29:00.012732 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.012717 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b43f6b8-dabc-40de-944b-25145570ed23" containerName="console" Jul 28 01:29:00.012732 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.012726 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ee77ea9-96ed-40bc-8169-f937717e4fa5" containerName="alertmanager" Jul 28 01:29:00.012858 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.012735 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ee77ea9-96ed-40bc-8169-f937717e4fa5" containerName="alertmanager" Jul 28 01:29:00.012858 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.012744 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ee77ea9-96ed-40bc-8169-f937717e4fa5" containerName="init-config-reloader" Jul 28 01:29:00.012858 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.012750 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ee77ea9-96ed-40bc-8169-f937717e4fa5" containerName="init-config-reloader" Jul 28 01:29:00.012858 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.012764 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ee77ea9-96ed-40bc-8169-f937717e4fa5" containerName="config-reloader" Jul 28 01:29:00.012858 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.012770 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ee77ea9-96ed-40bc-8169-f937717e4fa5" containerName="config-reloader" Jul 28 01:29:00.012858 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.012776 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ee77ea9-96ed-40bc-8169-f937717e4fa5" containerName="kube-rbac-proxy" Jul 28 01:29:00.012858 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.012781 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ee77ea9-96ed-40bc-8169-f937717e4fa5" containerName="kube-rbac-proxy" Jul 28 01:29:00.012858 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.012782 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d443e0d2dcb90422cedf500a702235a98e4b46ca9acfa7fb86b0df364b3ee57"} err="failed to get container status \"8d443e0d2dcb90422cedf500a702235a98e4b46ca9acfa7fb86b0df364b3ee57\": rpc error: code = NotFound desc = could not find container \"8d443e0d2dcb90422cedf500a702235a98e4b46ca9acfa7fb86b0df364b3ee57\": container with ID starting with 8d443e0d2dcb90422cedf500a702235a98e4b46ca9acfa7fb86b0df364b3ee57 not found: ID does not exist" Jul 28 01:29:00.012858 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.012796 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ee77ea9-96ed-40bc-8169-f937717e4fa5" containerName="prom-label-proxy" Jul 28 01:29:00.012858 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.012799 2578 scope.go:117] "RemoveContainer" containerID="d7b6dff65da4c81d0c28f380fcaa9c05b20813cdf96098212d10433718e562a0" Jul 28 01:29:00.012858 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.012801 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ee77ea9-96ed-40bc-8169-f937717e4fa5" containerName="prom-label-proxy" Jul 28 01:29:00.012858 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.012852 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31992241-f826-45d3-a26a-679d5d82465c" containerName="registry" Jul 28 01:29:00.012858 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.012860 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="31992241-f826-45d3-a26a-679d5d82465c" containerName="registry" Jul 28 01:29:00.013250 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.012869 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ee77ea9-96ed-40bc-8169-f937717e4fa5" containerName="kube-rbac-proxy-web" Jul 28 01:29:00.013250 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.012875 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ee77ea9-96ed-40bc-8169-f937717e4fa5" containerName="kube-rbac-proxy-web" Jul 28 01:29:00.013250 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.012942 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="31992241-f826-45d3-a26a-679d5d82465c" containerName="registry" Jul 28 01:29:00.013250 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.012950 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ee77ea9-96ed-40bc-8169-f937717e4fa5" containerName="alertmanager" Jul 28 01:29:00.013250 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.012961 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ee77ea9-96ed-40bc-8169-f937717e4fa5" containerName="kube-rbac-proxy-metric" Jul 28 01:29:00.013250 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.012968 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ee77ea9-96ed-40bc-8169-f937717e4fa5" containerName="kube-rbac-proxy-web" Jul 28 01:29:00.013250 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.012979 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ee77ea9-96ed-40bc-8169-f937717e4fa5" containerName="config-reloader" Jul 28 01:29:00.013250 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.012988 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b43f6b8-dabc-40de-944b-25145570ed23" containerName="console" Jul 28 01:29:00.013250 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.012994 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ee77ea9-96ed-40bc-8169-f937717e4fa5" containerName="prom-label-proxy" Jul 28 01:29:00.013250 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.013000 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ee77ea9-96ed-40bc-8169-f937717e4fa5" containerName="kube-rbac-proxy" Jul 28 01:29:00.013250 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.013023 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7b6dff65da4c81d0c28f380fcaa9c05b20813cdf96098212d10433718e562a0"} err="failed to get container status \"d7b6dff65da4c81d0c28f380fcaa9c05b20813cdf96098212d10433718e562a0\": rpc error: code = NotFound desc = could not find container \"d7b6dff65da4c81d0c28f380fcaa9c05b20813cdf96098212d10433718e562a0\": container with ID starting with d7b6dff65da4c81d0c28f380fcaa9c05b20813cdf96098212d10433718e562a0 not found: ID does not exist" Jul 28 01:29:00.019836 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.019816 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:29:00.022493 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.022473 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 28 01:29:00.022493 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.022475 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 28 01:29:00.022672 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.022530 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 28 01:29:00.022672 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.022621 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 28 01:29:00.022960 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.022942 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 28 01:29:00.023022 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.022948 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-pmpfw\"" Jul 28 01:29:00.023088 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.023047 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 28 01:29:00.023088 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.023070 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 28 01:29:00.023088 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.023076 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 28 01:29:00.029487 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.029470 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 28 01:29:00.037202 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.037184 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 01:29:00.109857 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.109830 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d9edc30d-4554-40ed-96a0-6bbeb0bf82b4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:29:00.109949 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.109868 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d9edc30d-4554-40ed-96a0-6bbeb0bf82b4-config-out\") pod \"alertmanager-main-0\" (UID: \"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:29:00.109949 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.109885 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d9edc30d-4554-40ed-96a0-6bbeb0bf82b4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:29:00.110027 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.110009 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d9edc30d-4554-40ed-96a0-6bbeb0bf82b4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:29:00.110099 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.110086 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d9edc30d-4554-40ed-96a0-6bbeb0bf82b4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:29:00.110131 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.110113 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bmdj\" (UniqueName: \"kubernetes.io/projected/d9edc30d-4554-40ed-96a0-6bbeb0bf82b4-kube-api-access-8bmdj\") pod \"alertmanager-main-0\" (UID: \"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:29:00.110257 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.110228 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d9edc30d-4554-40ed-96a0-6bbeb0bf82b4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:29:00.110319 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.110291 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d9edc30d-4554-40ed-96a0-6bbeb0bf82b4-web-config\") pod \"alertmanager-main-0\" (UID: \"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:29:00.110375 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.110348 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d9edc30d-4554-40ed-96a0-6bbeb0bf82b4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:29:00.110433 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.110382 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d9edc30d-4554-40ed-96a0-6bbeb0bf82b4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:29:00.110433 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.110416 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d9edc30d-4554-40ed-96a0-6bbeb0bf82b4-config-volume\") pod \"alertmanager-main-0\" (UID: \"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:29:00.110521 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.110433 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d9edc30d-4554-40ed-96a0-6bbeb0bf82b4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:29:00.110521 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.110493 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d9edc30d-4554-40ed-96a0-6bbeb0bf82b4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:29:00.211392 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.211298 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d9edc30d-4554-40ed-96a0-6bbeb0bf82b4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:29:00.211392 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.211342 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d9edc30d-4554-40ed-96a0-6bbeb0bf82b4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:29:00.211392 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.211364 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d9edc30d-4554-40ed-96a0-6bbeb0bf82b4-config-out\") pod \"alertmanager-main-0\" (UID: \"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:29:00.211720 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.211474 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d9edc30d-4554-40ed-96a0-6bbeb0bf82b4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:29:00.211720 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.211518 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d9edc30d-4554-40ed-96a0-6bbeb0bf82b4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:29:00.211720 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.211589 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d9edc30d-4554-40ed-96a0-6bbeb0bf82b4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:29:00.211720 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.211623 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bmdj\" (UniqueName: \"kubernetes.io/projected/d9edc30d-4554-40ed-96a0-6bbeb0bf82b4-kube-api-access-8bmdj\") pod \"alertmanager-main-0\" (UID: \"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:29:00.211720 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.211654 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d9edc30d-4554-40ed-96a0-6bbeb0bf82b4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:29:00.211720 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.211680 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d9edc30d-4554-40ed-96a0-6bbeb0bf82b4-web-config\") pod \"alertmanager-main-0\" (UID: \"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:29:00.212008 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.211732 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d9edc30d-4554-40ed-96a0-6bbeb0bf82b4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:29:00.212008 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.211764 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d9edc30d-4554-40ed-96a0-6bbeb0bf82b4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:29:00.212008 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.211790 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d9edc30d-4554-40ed-96a0-6bbeb0bf82b4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:29:00.212008 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.211807 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d9edc30d-4554-40ed-96a0-6bbeb0bf82b4-config-volume\") pod \"alertmanager-main-0\" (UID: \"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:29:00.212008 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.211848 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d9edc30d-4554-40ed-96a0-6bbeb0bf82b4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:29:00.212835 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.212644 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d9edc30d-4554-40ed-96a0-6bbeb0bf82b4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:29:00.212835 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.212771 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d9edc30d-4554-40ed-96a0-6bbeb0bf82b4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:29:00.213957 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.213932 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d9edc30d-4554-40ed-96a0-6bbeb0bf82b4-config-out\") pod \"alertmanager-main-0\" (UID: \"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:29:00.214409 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.214389 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d9edc30d-4554-40ed-96a0-6bbeb0bf82b4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:29:00.214840 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.214815 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d9edc30d-4554-40ed-96a0-6bbeb0bf82b4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:29:00.214965 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.214945 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d9edc30d-4554-40ed-96a0-6bbeb0bf82b4-web-config\") pod \"alertmanager-main-0\" (UID: \"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:29:00.215076 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.215055 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d9edc30d-4554-40ed-96a0-6bbeb0bf82b4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:29:00.215141 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.215110 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d9edc30d-4554-40ed-96a0-6bbeb0bf82b4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:29:00.215531 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.215510 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d9edc30d-4554-40ed-96a0-6bbeb0bf82b4-config-volume\") pod \"alertmanager-main-0\" (UID: \"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:29:00.215661 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.215644 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d9edc30d-4554-40ed-96a0-6bbeb0bf82b4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:29:00.215938 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.215922 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d9edc30d-4554-40ed-96a0-6bbeb0bf82b4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:29:00.220956 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.220939 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bmdj\" (UniqueName: \"kubernetes.io/projected/d9edc30d-4554-40ed-96a0-6bbeb0bf82b4-kube-api-access-8bmdj\") pod \"alertmanager-main-0\" (UID: \"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:29:00.329363 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.329340 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 01:29:00.461951 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.461879 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 01:29:00.464805 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:29:00.464782 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9edc30d_4554_40ed_96a0_6bbeb0bf82b4.slice/crio-42df7c6654921b96474d827ddc75df723df0e3bef59fb83b360a24d3310447ce WatchSource:0}: Error finding container 42df7c6654921b96474d827ddc75df723df0e3bef59fb83b360a24d3310447ce: Status 404 returned error can't find the container with id 42df7c6654921b96474d827ddc75df723df0e3bef59fb83b360a24d3310447ce Jul 28 01:29:00.964497 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.964465 2578 generic.go:356] "Generic (PLEG): container finished" podID="d9edc30d-4554-40ed-96a0-6bbeb0bf82b4" containerID="e29ffb0f69a0a86d16d10687abfed0537bad4a959b00d62f5a89d28b039668ff" exitCode=0 Jul 28 01:29:00.964664 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.964576 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4","Type":"ContainerDied","Data":"e29ffb0f69a0a86d16d10687abfed0537bad4a959b00d62f5a89d28b039668ff"} Jul 28 01:29:00.964664 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:00.964654 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4","Type":"ContainerStarted","Data":"42df7c6654921b96474d827ddc75df723df0e3bef59fb83b360a24d3310447ce"} Jul 28 01:29:01.583751 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:01.583722 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5d8557d54f-wn4vp"] Jul 28 01:29:01.585951 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:01.585929 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d8557d54f-wn4vp" Jul 28 01:29:01.604139 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:01.604114 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d8557d54f-wn4vp"] Jul 28 01:29:01.722125 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:01.722092 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d-console-config\") pod \"console-5d8557d54f-wn4vp\" (UID: \"a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d\") " pod="openshift-console/console-5d8557d54f-wn4vp" Jul 28 01:29:01.722125 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:01.722125 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d-service-ca\") pod \"console-5d8557d54f-wn4vp\" (UID: \"a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d\") " pod="openshift-console/console-5d8557d54f-wn4vp" Jul 28 01:29:01.722335 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:01.722150 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d-oauth-serving-cert\") pod \"console-5d8557d54f-wn4vp\" (UID: \"a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d\") " pod="openshift-console/console-5d8557d54f-wn4vp" Jul 28 01:29:01.722335 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:01.722281 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lp2t\" (UniqueName: \"kubernetes.io/projected/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d-kube-api-access-6lp2t\") pod \"console-5d8557d54f-wn4vp\" (UID: \"a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d\") " pod="openshift-console/console-5d8557d54f-wn4vp" Jul 28 01:29:01.722335 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:01.722330 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d-console-serving-cert\") pod \"console-5d8557d54f-wn4vp\" (UID: \"a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d\") " pod="openshift-console/console-5d8557d54f-wn4vp" Jul 28 01:29:01.722481 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:01.722362 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d-console-oauth-config\") pod \"console-5d8557d54f-wn4vp\" (UID: \"a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d\") " pod="openshift-console/console-5d8557d54f-wn4vp" Jul 28 01:29:01.722481 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:01.722404 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d-trusted-ca-bundle\") pod \"console-5d8557d54f-wn4vp\" (UID: \"a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d\") " pod="openshift-console/console-5d8557d54f-wn4vp" Jul 28 01:29:01.823662 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:01.823633 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lp2t\" (UniqueName: \"kubernetes.io/projected/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d-kube-api-access-6lp2t\") pod \"console-5d8557d54f-wn4vp\" (UID: \"a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d\") " pod="openshift-console/console-5d8557d54f-wn4vp" Jul 28 01:29:01.823747 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:01.823695 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d-console-serving-cert\") pod \"console-5d8557d54f-wn4vp\" (UID: \"a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d\") " pod="openshift-console/console-5d8557d54f-wn4vp" Jul 28 01:29:01.823747 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:01.823721 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d-console-oauth-config\") pod \"console-5d8557d54f-wn4vp\" (UID: \"a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d\") " pod="openshift-console/console-5d8557d54f-wn4vp" Jul 28 01:29:01.823747 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:01.823743 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d-trusted-ca-bundle\") pod \"console-5d8557d54f-wn4vp\" (UID: \"a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d\") " pod="openshift-console/console-5d8557d54f-wn4vp" Jul 28 01:29:01.823856 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:01.823783 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d-console-config\") pod \"console-5d8557d54f-wn4vp\" (UID: \"a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d\") " pod="openshift-console/console-5d8557d54f-wn4vp" Jul 28 01:29:01.823856 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:01.823806 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d-service-ca\") pod \"console-5d8557d54f-wn4vp\" (UID: \"a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d\") " pod="openshift-console/console-5d8557d54f-wn4vp" Jul 28 01:29:01.823856 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:01.823827 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d-oauth-serving-cert\") pod \"console-5d8557d54f-wn4vp\" (UID: \"a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d\") " pod="openshift-console/console-5d8557d54f-wn4vp" Jul 28 01:29:01.824460 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:01.824441 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d-oauth-serving-cert\") pod \"console-5d8557d54f-wn4vp\" (UID: \"a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d\") " pod="openshift-console/console-5d8557d54f-wn4vp" Jul 28 01:29:01.824460 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:01.824451 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d-console-config\") pod \"console-5d8557d54f-wn4vp\" (UID: \"a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d\") " pod="openshift-console/console-5d8557d54f-wn4vp" Jul 28 01:29:01.824692 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:01.824677 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d-service-ca\") pod \"console-5d8557d54f-wn4vp\" (UID: \"a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d\") " pod="openshift-console/console-5d8557d54f-wn4vp" Jul 28 01:29:01.824830 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:01.824809 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d-trusted-ca-bundle\") pod \"console-5d8557d54f-wn4vp\" (UID: \"a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d\") " pod="openshift-console/console-5d8557d54f-wn4vp" Jul 28 01:29:01.826013 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:01.825988 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d-console-serving-cert\") pod \"console-5d8557d54f-wn4vp\" (UID: \"a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d\") " pod="openshift-console/console-5d8557d54f-wn4vp" Jul 28 01:29:01.826147 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:01.826128 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d-console-oauth-config\") pod \"console-5d8557d54f-wn4vp\" (UID: \"a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d\") " pod="openshift-console/console-5d8557d54f-wn4vp" Jul 28 01:29:01.831262 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:01.831240 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lp2t\" (UniqueName: \"kubernetes.io/projected/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d-kube-api-access-6lp2t\") pod \"console-5d8557d54f-wn4vp\" (UID: \"a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d\") " pod="openshift-console/console-5d8557d54f-wn4vp" Jul 28 01:29:01.896698 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:01.896625 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d8557d54f-wn4vp" Jul 28 01:29:01.973117 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:01.973087 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4","Type":"ContainerStarted","Data":"66555307a9bcc23d86f4be8dabfe40119a98b46dda82acacb48d3929d0c92347"} Jul 28 01:29:01.973266 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:01.973194 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4","Type":"ContainerStarted","Data":"2b47712483cc499270890d0bef76e27dbba61517425f25f6cec4b65e036560d8"} Jul 28 01:29:01.973266 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:01.973243 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4","Type":"ContainerStarted","Data":"2e0c6ed6e89759cabd7ee79b18e0a656ac6ec744240b514711cfb8476f7c3d05"} Jul 28 01:29:01.973393 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:01.973281 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4","Type":"ContainerStarted","Data":"7a865a3eaf87abf9fb0a4830a94c3efe8740f8ff7ced9056ae622966060831d2"} Jul 28 01:29:01.973393 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:01.973325 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4","Type":"ContainerStarted","Data":"779f6cf6f795d13da7334bc39098814a91245f79442ee2af3aab58db1829f315"} Jul 28 01:29:01.973393 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:01.973379 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d9edc30d-4554-40ed-96a0-6bbeb0bf82b4","Type":"ContainerStarted","Data":"5ab0d449d84ef0fb2e2ceee10a1aa5f9bc98b0c841fdcf1ea06a2992073bc724"} Jul 28 01:29:02.002720 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:02.002668 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.987640624 podStartE2EDuration="3.002645672s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.005048ms" isStatefulPod=false podCreationTimestamp="2026-07-28 01:28:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:29:02.000615605 +0000 UTC m=+180.554740405" watchObservedRunningTime="2026-07-28 01:29:02.002645672 +0000 UTC m=+180.556770474" Jul 28 01:29:02.032483 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:02.032462 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d8557d54f-wn4vp"] Jul 28 01:29:02.035756 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:29:02.035733 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9457bfc_9a76_42e5_bfd0_86d3b6bdbf8d.slice/crio-a6f1ffbca4fdace333627662afd3db13968577ef0bc1d04d7642df06144d491e WatchSource:0}: Error finding container a6f1ffbca4fdace333627662afd3db13968577ef0bc1d04d7642df06144d491e: Status 404 returned error can't find the container with id a6f1ffbca4fdace333627662afd3db13968577ef0bc1d04d7642df06144d491e Jul 28 01:29:02.978755 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:02.978713 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d8557d54f-wn4vp" event={"ID":"a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d","Type":"ContainerStarted","Data":"3d05038ee4498de98498516a5cca23fed142a3a30472c08619e793f5847f38a3"} Jul 28 01:29:02.979105 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:02.978776 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d8557d54f-wn4vp" event={"ID":"a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d","Type":"ContainerStarted","Data":"a6f1ffbca4fdace333627662afd3db13968577ef0bc1d04d7642df06144d491e"} Jul 28 01:29:02.997262 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:02.997195 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5d8557d54f-wn4vp" podStartSLOduration=1.997180063 podStartE2EDuration="1.997180063s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:29:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:29:02.995782178 +0000 UTC m=+181.549906977" watchObservedRunningTime="2026-07-28 01:29:02.997180063 +0000 UTC m=+181.551304861" Jul 28 01:29:11.897377 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:11.897347 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5d8557d54f-wn4vp" Jul 28 01:29:11.897754 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:11.897418 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5d8557d54f-wn4vp" Jul 28 01:29:11.903331 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:11.903308 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5d8557d54f-wn4vp" Jul 28 01:29:12.016077 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:12.016051 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5d8557d54f-wn4vp" Jul 28 01:29:12.063931 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:12.063899 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64c4b96fd9-8vd5q"] Jul 28 01:29:37.090324 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:37.090285 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-64c4b96fd9-8vd5q" podUID="6df465b5-b8f4-4b86-bbe1-c75dedf24d7a" containerName="console" containerID="cri-o://1d88edfee93797f6d9070e745b029d1b9fce121d903e91b2870ad33309047cb1" gracePeriod=15 Jul 28 01:29:37.321362 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:37.321339 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64c4b96fd9-8vd5q" Jul 28 01:29:37.387481 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:37.387416 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a-console-oauth-config\") pod \"6df465b5-b8f4-4b86-bbe1-c75dedf24d7a\" (UID: \"6df465b5-b8f4-4b86-bbe1-c75dedf24d7a\") " Jul 28 01:29:37.387481 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:37.387446 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a-service-ca\") pod \"6df465b5-b8f4-4b86-bbe1-c75dedf24d7a\" (UID: \"6df465b5-b8f4-4b86-bbe1-c75dedf24d7a\") " Jul 28 01:29:37.387481 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:37.387471 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a-oauth-serving-cert\") pod \"6df465b5-b8f4-4b86-bbe1-c75dedf24d7a\" (UID: \"6df465b5-b8f4-4b86-bbe1-c75dedf24d7a\") " Jul 28 01:29:37.387746 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:37.387511 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hrgbz\" (UniqueName: \"kubernetes.io/projected/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a-kube-api-access-hrgbz\") pod \"6df465b5-b8f4-4b86-bbe1-c75dedf24d7a\" (UID: \"6df465b5-b8f4-4b86-bbe1-c75dedf24d7a\") " Jul 28 01:29:37.387746 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:37.387585 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a-console-config\") pod \"6df465b5-b8f4-4b86-bbe1-c75dedf24d7a\" (UID: \"6df465b5-b8f4-4b86-bbe1-c75dedf24d7a\") " Jul 28 01:29:37.387746 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:37.387630 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a-console-serving-cert\") pod \"6df465b5-b8f4-4b86-bbe1-c75dedf24d7a\" (UID: \"6df465b5-b8f4-4b86-bbe1-c75dedf24d7a\") " Jul 28 01:29:37.387746 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:37.387657 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a-trusted-ca-bundle\") pod \"6df465b5-b8f4-4b86-bbe1-c75dedf24d7a\" (UID: \"6df465b5-b8f4-4b86-bbe1-c75dedf24d7a\") " Jul 28 01:29:37.387944 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:37.387920 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6df465b5-b8f4-4b86-bbe1-c75dedf24d7a" (UID: "6df465b5-b8f4-4b86-bbe1-c75dedf24d7a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 01:29:37.387998 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:37.387934 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a-service-ca" (OuterVolumeSpecName: "service-ca") pod "6df465b5-b8f4-4b86-bbe1-c75dedf24d7a" (UID: "6df465b5-b8f4-4b86-bbe1-c75dedf24d7a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 01:29:37.387998 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:37.387954 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a-console-config" (OuterVolumeSpecName: "console-config") pod "6df465b5-b8f4-4b86-bbe1-c75dedf24d7a" (UID: "6df465b5-b8f4-4b86-bbe1-c75dedf24d7a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 01:29:37.388217 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:37.388190 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6df465b5-b8f4-4b86-bbe1-c75dedf24d7a" (UID: "6df465b5-b8f4-4b86-bbe1-c75dedf24d7a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 01:29:37.389590 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:37.389570 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6df465b5-b8f4-4b86-bbe1-c75dedf24d7a" (UID: "6df465b5-b8f4-4b86-bbe1-c75dedf24d7a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 01:29:37.389768 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:37.389752 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6df465b5-b8f4-4b86-bbe1-c75dedf24d7a" (UID: "6df465b5-b8f4-4b86-bbe1-c75dedf24d7a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 01:29:37.389950 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:37.389929 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a-kube-api-access-hrgbz" (OuterVolumeSpecName: "kube-api-access-hrgbz") pod "6df465b5-b8f4-4b86-bbe1-c75dedf24d7a" (UID: "6df465b5-b8f4-4b86-bbe1-c75dedf24d7a"). InnerVolumeSpecName "kube-api-access-hrgbz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 01:29:37.489032 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:37.489012 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a-console-config\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:29:37.489032 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:37.489032 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a-console-serving-cert\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:29:37.489149 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:37.489041 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a-trusted-ca-bundle\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:29:37.489149 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:37.489051 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a-console-oauth-config\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:29:37.489149 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:37.489060 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a-service-ca\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:29:37.489149 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:37.489069 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a-oauth-serving-cert\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:29:37.489149 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:37.489077 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hrgbz\" (UniqueName: \"kubernetes.io/projected/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a-kube-api-access-hrgbz\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:29:38.115876 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:38.115842 2578 generic.go:356] "Generic (PLEG): container finished" podID="6df465b5-b8f4-4b86-bbe1-c75dedf24d7a" containerID="1d88edfee93797f6d9070e745b029d1b9fce121d903e91b2870ad33309047cb1" exitCode=0 Jul 28 01:29:38.116327 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:38.115888 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64c4b96fd9-8vd5q" Jul 28 01:29:38.116327 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:38.115931 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64c4b96fd9-8vd5q" event={"ID":"6df465b5-b8f4-4b86-bbe1-c75dedf24d7a","Type":"ContainerDied","Data":"1d88edfee93797f6d9070e745b029d1b9fce121d903e91b2870ad33309047cb1"} Jul 28 01:29:38.116327 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:38.115991 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64c4b96fd9-8vd5q" event={"ID":"6df465b5-b8f4-4b86-bbe1-c75dedf24d7a","Type":"ContainerDied","Data":"29984f0d77174277c84fc79903d7c509a5df6bf2c77669f1ec9fff8d577208af"} Jul 28 01:29:38.116327 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:38.116023 2578 scope.go:117] "RemoveContainer" containerID="1d88edfee93797f6d9070e745b029d1b9fce121d903e91b2870ad33309047cb1" Jul 28 01:29:38.124446 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:38.124424 2578 scope.go:117] "RemoveContainer" containerID="1d88edfee93797f6d9070e745b029d1b9fce121d903e91b2870ad33309047cb1" Jul 28 01:29:38.124737 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:29:38.124709 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d88edfee93797f6d9070e745b029d1b9fce121d903e91b2870ad33309047cb1\": container with ID starting with 1d88edfee93797f6d9070e745b029d1b9fce121d903e91b2870ad33309047cb1 not found: ID does not exist" containerID="1d88edfee93797f6d9070e745b029d1b9fce121d903e91b2870ad33309047cb1" Jul 28 01:29:38.124826 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:38.124734 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d88edfee93797f6d9070e745b029d1b9fce121d903e91b2870ad33309047cb1"} err="failed to get container status \"1d88edfee93797f6d9070e745b029d1b9fce121d903e91b2870ad33309047cb1\": rpc error: code = NotFound desc = could not find container \"1d88edfee93797f6d9070e745b029d1b9fce121d903e91b2870ad33309047cb1\": container with ID starting with 1d88edfee93797f6d9070e745b029d1b9fce121d903e91b2870ad33309047cb1 not found: ID does not exist" Jul 28 01:29:38.132377 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:38.132357 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64c4b96fd9-8vd5q"] Jul 28 01:29:38.137905 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:38.137885 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-64c4b96fd9-8vd5q"] Jul 28 01:29:40.010000 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:29:40.009964 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6df465b5-b8f4-4b86-bbe1-c75dedf24d7a" path="/var/lib/kubelet/pods/6df465b5-b8f4-4b86-bbe1-c75dedf24d7a/volumes" Jul 28 01:31:01.806282 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:01.806253 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-k2wlm_3503116a-1398-42ea-ac86-743f5f51e989/console-operator/1.log" Jul 28 01:31:01.812823 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:01.812800 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-k2wlm_3503116a-1398-42ea-ac86-743f5f51e989/console-operator/1.log" Jul 28 01:31:01.817290 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:01.817272 2578 kubelet.go:1648] "Image garbage collection succeeded" Jul 28 01:31:14.267505 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:14.267473 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5fnftj"] Jul 28 01:31:14.268789 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:14.267827 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6df465b5-b8f4-4b86-bbe1-c75dedf24d7a" containerName="console" Jul 28 01:31:14.268789 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:14.267840 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6df465b5-b8f4-4b86-bbe1-c75dedf24d7a" containerName="console" Jul 28 01:31:14.268789 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:14.267904 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6df465b5-b8f4-4b86-bbe1-c75dedf24d7a" containerName="console" Jul 28 01:31:14.269864 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:14.269850 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5fnftj" Jul 28 01:31:14.272377 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:14.272353 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 28 01:31:14.272491 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:14.272454 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 28 01:31:14.273262 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:14.273245 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6xzww\"" Jul 28 01:31:14.278584 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:14.278564 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5fnftj"] Jul 28 01:31:14.348779 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:14.348755 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f59ab34d-49a5-4242-8ae4-0c739c2d4a5b-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5fnftj\" (UID: \"f59ab34d-49a5-4242-8ae4-0c739c2d4a5b\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5fnftj" Jul 28 01:31:14.348908 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:14.348799 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f59ab34d-49a5-4242-8ae4-0c739c2d4a5b-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5fnftj\" (UID: \"f59ab34d-49a5-4242-8ae4-0c739c2d4a5b\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5fnftj" Jul 28 01:31:14.348908 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:14.348827 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nt9nb\" (UniqueName: \"kubernetes.io/projected/f59ab34d-49a5-4242-8ae4-0c739c2d4a5b-kube-api-access-nt9nb\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5fnftj\" (UID: \"f59ab34d-49a5-4242-8ae4-0c739c2d4a5b\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5fnftj" Jul 28 01:31:14.450026 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:14.449997 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f59ab34d-49a5-4242-8ae4-0c739c2d4a5b-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5fnftj\" (UID: \"f59ab34d-49a5-4242-8ae4-0c739c2d4a5b\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5fnftj" Jul 28 01:31:14.450150 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:14.450040 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f59ab34d-49a5-4242-8ae4-0c739c2d4a5b-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5fnftj\" (UID: \"f59ab34d-49a5-4242-8ae4-0c739c2d4a5b\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5fnftj" Jul 28 01:31:14.450150 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:14.450067 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nt9nb\" (UniqueName: \"kubernetes.io/projected/f59ab34d-49a5-4242-8ae4-0c739c2d4a5b-kube-api-access-nt9nb\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5fnftj\" (UID: \"f59ab34d-49a5-4242-8ae4-0c739c2d4a5b\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5fnftj" Jul 28 01:31:14.450404 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:14.450385 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f59ab34d-49a5-4242-8ae4-0c739c2d4a5b-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5fnftj\" (UID: \"f59ab34d-49a5-4242-8ae4-0c739c2d4a5b\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5fnftj" Jul 28 01:31:14.450440 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:14.450409 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f59ab34d-49a5-4242-8ae4-0c739c2d4a5b-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5fnftj\" (UID: \"f59ab34d-49a5-4242-8ae4-0c739c2d4a5b\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5fnftj" Jul 28 01:31:14.458247 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:14.458226 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nt9nb\" (UniqueName: \"kubernetes.io/projected/f59ab34d-49a5-4242-8ae4-0c739c2d4a5b-kube-api-access-nt9nb\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5fnftj\" (UID: \"f59ab34d-49a5-4242-8ae4-0c739c2d4a5b\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5fnftj" Jul 28 01:31:14.579909 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:14.579833 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5fnftj" Jul 28 01:31:14.703155 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:14.703094 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5fnftj"] Jul 28 01:31:14.705569 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:31:14.705530 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf59ab34d_49a5_4242_8ae4_0c739c2d4a5b.slice/crio-a9dfd3344bcf002e75a6126519722c69e87a605c6d660a2f1f99e4e5f096b467 WatchSource:0}: Error finding container a9dfd3344bcf002e75a6126519722c69e87a605c6d660a2f1f99e4e5f096b467: Status 404 returned error can't find the container with id a9dfd3344bcf002e75a6126519722c69e87a605c6d660a2f1f99e4e5f096b467 Jul 28 01:31:14.707885 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:14.707869 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 01:31:15.502414 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:15.502383 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5fnftj" event={"ID":"f59ab34d-49a5-4242-8ae4-0c739c2d4a5b","Type":"ContainerStarted","Data":"a9dfd3344bcf002e75a6126519722c69e87a605c6d660a2f1f99e4e5f096b467"} Jul 28 01:31:21.529628 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:21.529597 2578 generic.go:356] "Generic (PLEG): container finished" podID="f59ab34d-49a5-4242-8ae4-0c739c2d4a5b" containerID="6e593fdd11130a60ae2d84334dae188e995f4317dacbded0854581825b9c2d2e" exitCode=0 Jul 28 01:31:21.530055 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:21.529691 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5fnftj" event={"ID":"f59ab34d-49a5-4242-8ae4-0c739c2d4a5b","Type":"ContainerDied","Data":"6e593fdd11130a60ae2d84334dae188e995f4317dacbded0854581825b9c2d2e"} Jul 28 01:31:24.545936 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:24.545898 2578 generic.go:356] "Generic (PLEG): container finished" podID="f59ab34d-49a5-4242-8ae4-0c739c2d4a5b" containerID="759b20aa50851080d6c2adfa0b5d6292952bea76413bc52e8dc37fc32c8c6797" exitCode=0 Jul 28 01:31:24.546384 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:24.545989 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5fnftj" event={"ID":"f59ab34d-49a5-4242-8ae4-0c739c2d4a5b","Type":"ContainerDied","Data":"759b20aa50851080d6c2adfa0b5d6292952bea76413bc52e8dc37fc32c8c6797"} Jul 28 01:31:31.583917 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:31.583875 2578 generic.go:356] "Generic (PLEG): container finished" podID="f59ab34d-49a5-4242-8ae4-0c739c2d4a5b" containerID="9469a9d86ebf50399af36bf7bb7cb527c68361791346d9153daa914cf233e38e" exitCode=0 Jul 28 01:31:31.584360 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:31.583965 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5fnftj" event={"ID":"f59ab34d-49a5-4242-8ae4-0c739c2d4a5b","Type":"ContainerDied","Data":"9469a9d86ebf50399af36bf7bb7cb527c68361791346d9153daa914cf233e38e"} Jul 28 01:31:32.709304 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:32.709277 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5fnftj" Jul 28 01:31:32.713418 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:32.713399 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f59ab34d-49a5-4242-8ae4-0c739c2d4a5b-bundle\") pod \"f59ab34d-49a5-4242-8ae4-0c739c2d4a5b\" (UID: \"f59ab34d-49a5-4242-8ae4-0c739c2d4a5b\") " Jul 28 01:31:32.713500 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:32.713449 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f59ab34d-49a5-4242-8ae4-0c739c2d4a5b-util\") pod \"f59ab34d-49a5-4242-8ae4-0c739c2d4a5b\" (UID: \"f59ab34d-49a5-4242-8ae4-0c739c2d4a5b\") " Jul 28 01:31:32.713500 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:32.713476 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nt9nb\" (UniqueName: \"kubernetes.io/projected/f59ab34d-49a5-4242-8ae4-0c739c2d4a5b-kube-api-access-nt9nb\") pod \"f59ab34d-49a5-4242-8ae4-0c739c2d4a5b\" (UID: \"f59ab34d-49a5-4242-8ae4-0c739c2d4a5b\") " Jul 28 01:31:32.714025 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:32.714003 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f59ab34d-49a5-4242-8ae4-0c739c2d4a5b-bundle" (OuterVolumeSpecName: "bundle") pod "f59ab34d-49a5-4242-8ae4-0c739c2d4a5b" (UID: "f59ab34d-49a5-4242-8ae4-0c739c2d4a5b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 01:31:32.715593 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:32.715566 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f59ab34d-49a5-4242-8ae4-0c739c2d4a5b-kube-api-access-nt9nb" (OuterVolumeSpecName: "kube-api-access-nt9nb") pod "f59ab34d-49a5-4242-8ae4-0c739c2d4a5b" (UID: "f59ab34d-49a5-4242-8ae4-0c739c2d4a5b"). InnerVolumeSpecName "kube-api-access-nt9nb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 01:31:32.717654 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:32.717634 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f59ab34d-49a5-4242-8ae4-0c739c2d4a5b-util" (OuterVolumeSpecName: "util") pod "f59ab34d-49a5-4242-8ae4-0c739c2d4a5b" (UID: "f59ab34d-49a5-4242-8ae4-0c739c2d4a5b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 01:31:32.814757 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:32.814722 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f59ab34d-49a5-4242-8ae4-0c739c2d4a5b-bundle\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:31:32.814757 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:32.814750 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f59ab34d-49a5-4242-8ae4-0c739c2d4a5b-util\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:31:32.814757 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:32.814761 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nt9nb\" (UniqueName: \"kubernetes.io/projected/f59ab34d-49a5-4242-8ae4-0c739c2d4a5b-kube-api-access-nt9nb\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:31:33.593519 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:33.593488 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5fnftj" Jul 28 01:31:33.593519 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:33.593505 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5fnftj" event={"ID":"f59ab34d-49a5-4242-8ae4-0c739c2d4a5b","Type":"ContainerDied","Data":"a9dfd3344bcf002e75a6126519722c69e87a605c6d660a2f1f99e4e5f096b467"} Jul 28 01:31:33.593737 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:33.593565 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a9dfd3344bcf002e75a6126519722c69e87a605c6d660a2f1f99e4e5f096b467" Jul 28 01:31:37.864153 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:37.864126 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-5q8nf"] Jul 28 01:31:37.870017 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:37.865095 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f59ab34d-49a5-4242-8ae4-0c739c2d4a5b" containerName="util" Jul 28 01:31:37.870017 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:37.865120 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f59ab34d-49a5-4242-8ae4-0c739c2d4a5b" containerName="util" Jul 28 01:31:37.870017 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:37.865150 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f59ab34d-49a5-4242-8ae4-0c739c2d4a5b" containerName="pull" Jul 28 01:31:37.870017 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:37.865179 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f59ab34d-49a5-4242-8ae4-0c739c2d4a5b" containerName="pull" Jul 28 01:31:37.870017 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:37.865202 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f59ab34d-49a5-4242-8ae4-0c739c2d4a5b" containerName="extract" Jul 28 01:31:37.870017 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:37.865211 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f59ab34d-49a5-4242-8ae4-0c739c2d4a5b" containerName="extract" Jul 28 01:31:37.870017 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:37.865375 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f59ab34d-49a5-4242-8ae4-0c739c2d4a5b" containerName="extract" Jul 28 01:31:37.908534 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:37.908507 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-5q8nf"] Jul 28 01:31:37.908667 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:37.908618 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-5q8nf" Jul 28 01:31:37.911552 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:37.911518 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-mmgdz\"" Jul 28 01:31:37.912093 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:37.912072 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\"" Jul 28 01:31:37.912217 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:37.912099 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\"" Jul 28 01:31:37.912217 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:37.912100 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\"" Jul 28 01:31:37.959104 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:37.959081 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/663cc2bf-d9ef-49bd-818a-fbcd559e7d03-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-5q8nf\" (UID: \"663cc2bf-d9ef-49bd-818a-fbcd559e7d03\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-5q8nf" Jul 28 01:31:37.959208 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:37.959167 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7r2r\" (UniqueName: \"kubernetes.io/projected/663cc2bf-d9ef-49bd-818a-fbcd559e7d03-kube-api-access-j7r2r\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-5q8nf\" (UID: \"663cc2bf-d9ef-49bd-818a-fbcd559e7d03\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-5q8nf" Jul 28 01:31:38.060303 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:38.060273 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7r2r\" (UniqueName: \"kubernetes.io/projected/663cc2bf-d9ef-49bd-818a-fbcd559e7d03-kube-api-access-j7r2r\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-5q8nf\" (UID: \"663cc2bf-d9ef-49bd-818a-fbcd559e7d03\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-5q8nf" Jul 28 01:31:38.060418 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:38.060310 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/663cc2bf-d9ef-49bd-818a-fbcd559e7d03-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-5q8nf\" (UID: \"663cc2bf-d9ef-49bd-818a-fbcd559e7d03\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-5q8nf" Jul 28 01:31:38.062575 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:38.062555 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/663cc2bf-d9ef-49bd-818a-fbcd559e7d03-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-5q8nf\" (UID: \"663cc2bf-d9ef-49bd-818a-fbcd559e7d03\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-5q8nf" Jul 28 01:31:38.070030 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:38.070008 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7r2r\" (UniqueName: \"kubernetes.io/projected/663cc2bf-d9ef-49bd-818a-fbcd559e7d03-kube-api-access-j7r2r\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-5q8nf\" (UID: \"663cc2bf-d9ef-49bd-818a-fbcd559e7d03\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-5q8nf" Jul 28 01:31:38.219337 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:38.219264 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-5q8nf" Jul 28 01:31:38.367243 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:38.367205 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-5q8nf"] Jul 28 01:31:38.371058 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:31:38.371030 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod663cc2bf_d9ef_49bd_818a_fbcd559e7d03.slice/crio-0787447e17de1cc7ef42ce269a532975fd03b3c27b02d7ba18486ea1d7b88e08 WatchSource:0}: Error finding container 0787447e17de1cc7ef42ce269a532975fd03b3c27b02d7ba18486ea1d7b88e08: Status 404 returned error can't find the container with id 0787447e17de1cc7ef42ce269a532975fd03b3c27b02d7ba18486ea1d7b88e08 Jul 28 01:31:38.615899 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:38.615864 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-5q8nf" event={"ID":"663cc2bf-d9ef-49bd-818a-fbcd559e7d03","Type":"ContainerStarted","Data":"0787447e17de1cc7ef42ce269a532975fd03b3c27b02d7ba18486ea1d7b88e08"} Jul 28 01:31:41.633509 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:41.633472 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-5q8nf" event={"ID":"663cc2bf-d9ef-49bd-818a-fbcd559e7d03","Type":"ContainerStarted","Data":"4eee7091f3540a01992a7bc9eb67c44869bc1f1cf78bd2730d1963d4a50b31a6"} Jul 28 01:31:41.633934 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:41.633596 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-5q8nf" Jul 28 01:31:41.665918 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:41.665828 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-5q8nf" podStartSLOduration=1.627516392 podStartE2EDuration="4.66581354s" totalImagesPullingTime="3.038297148s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:31:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:31:41.662957864 +0000 UTC m=+340.217082664" watchObservedRunningTime="2026-07-28 01:31:41.66581354 +0000 UTC m=+340.219938337" Jul 28 01:31:42.255290 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:42.255257 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-77f648d694-zd5sn"] Jul 28 01:31:42.257466 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:42.257449 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-77f648d694-zd5sn" Jul 28 01:31:42.259967 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:42.259945 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\"" Jul 28 01:31:42.260064 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:42.260025 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\"" Jul 28 01:31:42.260124 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:42.260094 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-fzprb\"" Jul 28 01:31:42.269585 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:42.269562 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-77f648d694-zd5sn"] Jul 28 01:31:42.300725 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:42.300703 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/e1bc512e-f171-4243-902e-8329975473a4-cabundle0\") pod \"keda-operator-77f648d694-zd5sn\" (UID: \"e1bc512e-f171-4243-902e-8329975473a4\") " pod="openshift-keda/keda-operator-77f648d694-zd5sn" Jul 28 01:31:42.300825 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:42.300733 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dps6q\" (UniqueName: \"kubernetes.io/projected/e1bc512e-f171-4243-902e-8329975473a4-kube-api-access-dps6q\") pod \"keda-operator-77f648d694-zd5sn\" (UID: \"e1bc512e-f171-4243-902e-8329975473a4\") " pod="openshift-keda/keda-operator-77f648d694-zd5sn" Jul 28 01:31:42.300825 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:42.300770 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/e1bc512e-f171-4243-902e-8329975473a4-certificates\") pod \"keda-operator-77f648d694-zd5sn\" (UID: \"e1bc512e-f171-4243-902e-8329975473a4\") " pod="openshift-keda/keda-operator-77f648d694-zd5sn" Jul 28 01:31:42.401302 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:42.401267 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/e1bc512e-f171-4243-902e-8329975473a4-certificates\") pod \"keda-operator-77f648d694-zd5sn\" (UID: \"e1bc512e-f171-4243-902e-8329975473a4\") " pod="openshift-keda/keda-operator-77f648d694-zd5sn" Jul 28 01:31:42.401486 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:42.401324 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/e1bc512e-f171-4243-902e-8329975473a4-cabundle0\") pod \"keda-operator-77f648d694-zd5sn\" (UID: \"e1bc512e-f171-4243-902e-8329975473a4\") " pod="openshift-keda/keda-operator-77f648d694-zd5sn" Jul 28 01:31:42.401486 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:42.401347 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dps6q\" (UniqueName: \"kubernetes.io/projected/e1bc512e-f171-4243-902e-8329975473a4-kube-api-access-dps6q\") pod \"keda-operator-77f648d694-zd5sn\" (UID: \"e1bc512e-f171-4243-902e-8329975473a4\") " pod="openshift-keda/keda-operator-77f648d694-zd5sn" Jul 28 01:31:42.401486 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:31:42.401409 2578 projected.go:266] Couldn't get secret openshift-keda/keda-operator-certs: secret "keda-operator-certs" not found Jul 28 01:31:42.401486 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:31:42.401431 2578 secret.go:281] references non-existent secret key: ca.crt Jul 28 01:31:42.401486 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:31:42.401439 2578 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt Jul 28 01:31:42.401486 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:31:42.401450 2578 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-77f648d694-zd5sn: [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt] Jul 28 01:31:42.401801 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:31:42.401507 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e1bc512e-f171-4243-902e-8329975473a4-certificates podName:e1bc512e-f171-4243-902e-8329975473a4 nodeName:}" failed. No retries permitted until 2026-07-28 01:31:42.901492491 +0000 UTC m=+341.455617267 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/e1bc512e-f171-4243-902e-8329975473a4-certificates") pod "keda-operator-77f648d694-zd5sn" (UID: "e1bc512e-f171-4243-902e-8329975473a4") : [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt] Jul 28 01:31:42.402039 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:42.402017 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/e1bc512e-f171-4243-902e-8329975473a4-cabundle0\") pod \"keda-operator-77f648d694-zd5sn\" (UID: \"e1bc512e-f171-4243-902e-8329975473a4\") " pod="openshift-keda/keda-operator-77f648d694-zd5sn" Jul 28 01:31:42.412587 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:42.412562 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dps6q\" (UniqueName: \"kubernetes.io/projected/e1bc512e-f171-4243-902e-8329975473a4-kube-api-access-dps6q\") pod \"keda-operator-77f648d694-zd5sn\" (UID: \"e1bc512e-f171-4243-902e-8329975473a4\") " pod="openshift-keda/keda-operator-77f648d694-zd5sn" Jul 28 01:31:42.471627 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:42.471598 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-55557df8c-n4m9l"] Jul 28 01:31:42.473950 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:42.473935 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-55557df8c-n4m9l" Jul 28 01:31:42.476890 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:42.476866 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\"" Jul 28 01:31:42.487627 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:42.487607 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-55557df8c-n4m9l"] Jul 28 01:31:42.502052 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:42.502024 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/d4edbd8a-0a2c-44d4-b570-0220176a76b0-certificates\") pod \"keda-metrics-apiserver-55557df8c-n4m9l\" (UID: \"d4edbd8a-0a2c-44d4-b570-0220176a76b0\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-n4m9l" Jul 28 01:31:42.502171 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:42.502106 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/d4edbd8a-0a2c-44d4-b570-0220176a76b0-temp-vol\") pod \"keda-metrics-apiserver-55557df8c-n4m9l\" (UID: \"d4edbd8a-0a2c-44d4-b570-0220176a76b0\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-n4m9l" Jul 28 01:31:42.502171 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:42.502160 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qwfb\" (UniqueName: \"kubernetes.io/projected/d4edbd8a-0a2c-44d4-b570-0220176a76b0-kube-api-access-9qwfb\") pod \"keda-metrics-apiserver-55557df8c-n4m9l\" (UID: \"d4edbd8a-0a2c-44d4-b570-0220176a76b0\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-n4m9l" Jul 28 01:31:42.602624 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:42.602560 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/d4edbd8a-0a2c-44d4-b570-0220176a76b0-certificates\") pod \"keda-metrics-apiserver-55557df8c-n4m9l\" (UID: \"d4edbd8a-0a2c-44d4-b570-0220176a76b0\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-n4m9l" Jul 28 01:31:42.602624 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:42.602618 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/d4edbd8a-0a2c-44d4-b570-0220176a76b0-temp-vol\") pod \"keda-metrics-apiserver-55557df8c-n4m9l\" (UID: \"d4edbd8a-0a2c-44d4-b570-0220176a76b0\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-n4m9l" Jul 28 01:31:42.602772 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:42.602650 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qwfb\" (UniqueName: \"kubernetes.io/projected/d4edbd8a-0a2c-44d4-b570-0220176a76b0-kube-api-access-9qwfb\") pod \"keda-metrics-apiserver-55557df8c-n4m9l\" (UID: \"d4edbd8a-0a2c-44d4-b570-0220176a76b0\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-n4m9l" Jul 28 01:31:42.602772 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:31:42.602711 2578 secret.go:281] references non-existent secret key: tls.crt Jul 28 01:31:42.602772 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:31:42.602732 2578 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt Jul 28 01:31:42.602772 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:31:42.602752 2578 projected.go:266] Couldn't get secret openshift-keda/keda-metrics-apiserver-certs: secret "keda-metrics-apiserver-certs" not found Jul 28 01:31:42.602904 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:31:42.602780 2578 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-55557df8c-n4m9l: [references non-existent secret key: tls.crt, secret "keda-metrics-apiserver-certs" not found] Jul 28 01:31:42.602904 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:31:42.602842 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4edbd8a-0a2c-44d4-b570-0220176a76b0-certificates podName:d4edbd8a-0a2c-44d4-b570-0220176a76b0 nodeName:}" failed. No retries permitted until 2026-07-28 01:31:43.102823832 +0000 UTC m=+341.656948611 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/d4edbd8a-0a2c-44d4-b570-0220176a76b0-certificates") pod "keda-metrics-apiserver-55557df8c-n4m9l" (UID: "d4edbd8a-0a2c-44d4-b570-0220176a76b0") : [references non-existent secret key: tls.crt, secret "keda-metrics-apiserver-certs" not found] Jul 28 01:31:42.603062 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:42.603044 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/d4edbd8a-0a2c-44d4-b570-0220176a76b0-temp-vol\") pod \"keda-metrics-apiserver-55557df8c-n4m9l\" (UID: \"d4edbd8a-0a2c-44d4-b570-0220176a76b0\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-n4m9l" Jul 28 01:31:42.611017 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:42.610990 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qwfb\" (UniqueName: \"kubernetes.io/projected/d4edbd8a-0a2c-44d4-b570-0220176a76b0-kube-api-access-9qwfb\") pod \"keda-metrics-apiserver-55557df8c-n4m9l\" (UID: \"d4edbd8a-0a2c-44d4-b570-0220176a76b0\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-n4m9l" Jul 28 01:31:42.905103 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:42.905011 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/e1bc512e-f171-4243-902e-8329975473a4-certificates\") pod \"keda-operator-77f648d694-zd5sn\" (UID: \"e1bc512e-f171-4243-902e-8329975473a4\") " pod="openshift-keda/keda-operator-77f648d694-zd5sn" Jul 28 01:31:42.907649 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:42.907623 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/e1bc512e-f171-4243-902e-8329975473a4-certificates\") pod \"keda-operator-77f648d694-zd5sn\" (UID: \"e1bc512e-f171-4243-902e-8329975473a4\") " pod="openshift-keda/keda-operator-77f648d694-zd5sn" Jul 28 01:31:42.996332 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:42.996302 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-9rq8r"] Jul 28 01:31:42.999317 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:42.999303 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-56f9798bb4-9rq8r" Jul 28 01:31:43.002004 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:43.001984 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\"" Jul 28 01:31:43.012209 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:43.012185 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-9rq8r"] Jul 28 01:31:43.106216 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:43.106187 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pv7q\" (UniqueName: \"kubernetes.io/projected/3fdaeb02-de6d-4cbe-9ddd-d79af5ce49a7-kube-api-access-4pv7q\") pod \"keda-admission-56f9798bb4-9rq8r\" (UID: \"3fdaeb02-de6d-4cbe-9ddd-d79af5ce49a7\") " pod="openshift-keda/keda-admission-56f9798bb4-9rq8r" Jul 28 01:31:43.106389 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:43.106264 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/d4edbd8a-0a2c-44d4-b570-0220176a76b0-certificates\") pod \"keda-metrics-apiserver-55557df8c-n4m9l\" (UID: \"d4edbd8a-0a2c-44d4-b570-0220176a76b0\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-n4m9l" Jul 28 01:31:43.106389 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:43.106312 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/3fdaeb02-de6d-4cbe-9ddd-d79af5ce49a7-certificates\") pod \"keda-admission-56f9798bb4-9rq8r\" (UID: \"3fdaeb02-de6d-4cbe-9ddd-d79af5ce49a7\") " pod="openshift-keda/keda-admission-56f9798bb4-9rq8r" Jul 28 01:31:43.108756 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:43.108733 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/d4edbd8a-0a2c-44d4-b570-0220176a76b0-certificates\") pod \"keda-metrics-apiserver-55557df8c-n4m9l\" (UID: \"d4edbd8a-0a2c-44d4-b570-0220176a76b0\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-n4m9l" Jul 28 01:31:43.168007 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:43.167939 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-77f648d694-zd5sn" Jul 28 01:31:43.207944 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:43.207914 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pv7q\" (UniqueName: \"kubernetes.io/projected/3fdaeb02-de6d-4cbe-9ddd-d79af5ce49a7-kube-api-access-4pv7q\") pod \"keda-admission-56f9798bb4-9rq8r\" (UID: \"3fdaeb02-de6d-4cbe-9ddd-d79af5ce49a7\") " pod="openshift-keda/keda-admission-56f9798bb4-9rq8r" Jul 28 01:31:43.208103 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:43.208002 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/3fdaeb02-de6d-4cbe-9ddd-d79af5ce49a7-certificates\") pod \"keda-admission-56f9798bb4-9rq8r\" (UID: \"3fdaeb02-de6d-4cbe-9ddd-d79af5ce49a7\") " pod="openshift-keda/keda-admission-56f9798bb4-9rq8r" Jul 28 01:31:43.210921 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:43.210895 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/3fdaeb02-de6d-4cbe-9ddd-d79af5ce49a7-certificates\") pod \"keda-admission-56f9798bb4-9rq8r\" (UID: \"3fdaeb02-de6d-4cbe-9ddd-d79af5ce49a7\") " pod="openshift-keda/keda-admission-56f9798bb4-9rq8r" Jul 28 01:31:43.220962 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:43.220297 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pv7q\" (UniqueName: \"kubernetes.io/projected/3fdaeb02-de6d-4cbe-9ddd-d79af5ce49a7-kube-api-access-4pv7q\") pod \"keda-admission-56f9798bb4-9rq8r\" (UID: \"3fdaeb02-de6d-4cbe-9ddd-d79af5ce49a7\") " pod="openshift-keda/keda-admission-56f9798bb4-9rq8r" Jul 28 01:31:43.309086 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:43.309051 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-56f9798bb4-9rq8r" Jul 28 01:31:43.311669 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:43.311646 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-77f648d694-zd5sn"] Jul 28 01:31:43.314215 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:31:43.314188 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1bc512e_f171_4243_902e_8329975473a4.slice/crio-6a9cc30d5d20b79a7dc8eaff623d1d5c6c9769b6dd182e6fde6373dfc7c43b38 WatchSource:0}: Error finding container 6a9cc30d5d20b79a7dc8eaff623d1d5c6c9769b6dd182e6fde6373dfc7c43b38: Status 404 returned error can't find the container with id 6a9cc30d5d20b79a7dc8eaff623d1d5c6c9769b6dd182e6fde6373dfc7c43b38 Jul 28 01:31:43.384857 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:43.384828 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-55557df8c-n4m9l" Jul 28 01:31:43.438956 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:43.438927 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-9rq8r"] Jul 28 01:31:43.442861 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:31:43.442831 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fdaeb02_de6d_4cbe_9ddd_d79af5ce49a7.slice/crio-64de95a57102ba0f3e68795d74197efe89a7e38418c20406cee9a26c8520234a WatchSource:0}: Error finding container 64de95a57102ba0f3e68795d74197efe89a7e38418c20406cee9a26c8520234a: Status 404 returned error can't find the container with id 64de95a57102ba0f3e68795d74197efe89a7e38418c20406cee9a26c8520234a Jul 28 01:31:43.519374 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:43.519352 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-55557df8c-n4m9l"] Jul 28 01:31:43.521278 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:31:43.521256 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4edbd8a_0a2c_44d4_b570_0220176a76b0.slice/crio-29bac88d0d158a26438f64ec292facd847af7f92782435a9aaa25116d260a971 WatchSource:0}: Error finding container 29bac88d0d158a26438f64ec292facd847af7f92782435a9aaa25116d260a971: Status 404 returned error can't find the container with id 29bac88d0d158a26438f64ec292facd847af7f92782435a9aaa25116d260a971 Jul 28 01:31:43.643956 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:43.643927 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-56f9798bb4-9rq8r" event={"ID":"3fdaeb02-de6d-4cbe-9ddd-d79af5ce49a7","Type":"ContainerStarted","Data":"64de95a57102ba0f3e68795d74197efe89a7e38418c20406cee9a26c8520234a"} Jul 28 01:31:43.645029 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:43.645008 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-55557df8c-n4m9l" event={"ID":"d4edbd8a-0a2c-44d4-b570-0220176a76b0","Type":"ContainerStarted","Data":"29bac88d0d158a26438f64ec292facd847af7f92782435a9aaa25116d260a971"} Jul 28 01:31:43.646150 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:43.646127 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-77f648d694-zd5sn" event={"ID":"e1bc512e-f171-4243-902e-8329975473a4","Type":"ContainerStarted","Data":"6a9cc30d5d20b79a7dc8eaff623d1d5c6c9769b6dd182e6fde6373dfc7c43b38"} Jul 28 01:31:45.659180 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:45.659139 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-56f9798bb4-9rq8r" event={"ID":"3fdaeb02-de6d-4cbe-9ddd-d79af5ce49a7","Type":"ContainerStarted","Data":"c784256b62c0f3757cb42d0f5642daf352f96830a6dd54c38f8cb25751b13db3"} Jul 28 01:31:45.659655 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:45.659306 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-56f9798bb4-9rq8r" Jul 28 01:31:45.678979 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:45.678924 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-56f9798bb4-9rq8r" podStartSLOduration=2.161666004 podStartE2EDuration="3.678905073s" totalImagesPullingTime="1.517239069s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:31:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:31:45.675822179 +0000 UTC m=+344.229946977" watchObservedRunningTime="2026-07-28 01:31:45.678905073 +0000 UTC m=+344.233029871" Jul 28 01:31:47.670316 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:47.670273 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-55557df8c-n4m9l" event={"ID":"d4edbd8a-0a2c-44d4-b570-0220176a76b0","Type":"ContainerStarted","Data":"136773e5d0186271dfc3e82310e18f798a883fa5e98da9337881516c6860ae99"} Jul 28 01:31:47.670771 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:47.670377 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-55557df8c-n4m9l" Jul 28 01:31:47.672002 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:47.671975 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-77f648d694-zd5sn" event={"ID":"e1bc512e-f171-4243-902e-8329975473a4","Type":"ContainerStarted","Data":"b0f10a55be010030b7b81397d3227277880e6d9ec50d02385e2954db248e028e"} Jul 28 01:31:47.672122 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:47.672035 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-77f648d694-zd5sn" Jul 28 01:31:47.690283 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:47.690222 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-55557df8c-n4m9l" podStartSLOduration=1.639068798 podStartE2EDuration="5.690200547s" totalImagesPullingTime="4.051131749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:31:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:31:47.688123884 +0000 UTC m=+346.242248705" watchObservedRunningTime="2026-07-28 01:31:47.690200547 +0000 UTC m=+346.244325346" Jul 28 01:31:47.705184 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:47.705132 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-operator-77f648d694-zd5sn" podStartSLOduration=1.444067214 podStartE2EDuration="5.705117118s" totalImagesPullingTime="4.261049904s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:31:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:31:47.703904455 +0000 UTC m=+346.258029253" watchObservedRunningTime="2026-07-28 01:31:47.705117118 +0000 UTC m=+346.259241915" Jul 28 01:31:58.681613 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:31:58.681523 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-55557df8c-n4m9l" Jul 28 01:32:02.640458 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:32:02.640429 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-5q8nf" Jul 28 01:32:06.666316 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:32:06.666282 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-56f9798bb4-9rq8r" Jul 28 01:32:08.679225 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:32:08.679194 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-77f648d694-zd5sn" Jul 28 01:32:57.773363 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:32:57.773325 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-798f5678bc-wm5rv"] Jul 28 01:32:57.775873 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:32:57.775852 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-798f5678bc-wm5rv" Jul 28 01:32:57.788388 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:32:57.788362 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-798f5678bc-wm5rv"] Jul 28 01:32:57.928594 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:32:57.928563 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f661790-6ef9-4a48-9ea1-40e2e42a76d2-console-config\") pod \"console-798f5678bc-wm5rv\" (UID: \"1f661790-6ef9-4a48-9ea1-40e2e42a76d2\") " pod="openshift-console/console-798f5678bc-wm5rv" Jul 28 01:32:57.928594 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:32:57.928608 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f661790-6ef9-4a48-9ea1-40e2e42a76d2-service-ca\") pod \"console-798f5678bc-wm5rv\" (UID: \"1f661790-6ef9-4a48-9ea1-40e2e42a76d2\") " pod="openshift-console/console-798f5678bc-wm5rv" Jul 28 01:32:57.928869 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:32:57.928640 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f661790-6ef9-4a48-9ea1-40e2e42a76d2-console-oauth-config\") pod \"console-798f5678bc-wm5rv\" (UID: \"1f661790-6ef9-4a48-9ea1-40e2e42a76d2\") " pod="openshift-console/console-798f5678bc-wm5rv" Jul 28 01:32:57.928869 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:32:57.928664 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f661790-6ef9-4a48-9ea1-40e2e42a76d2-oauth-serving-cert\") pod \"console-798f5678bc-wm5rv\" (UID: \"1f661790-6ef9-4a48-9ea1-40e2e42a76d2\") " pod="openshift-console/console-798f5678bc-wm5rv" Jul 28 01:32:57.928869 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:32:57.928703 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f661790-6ef9-4a48-9ea1-40e2e42a76d2-console-serving-cert\") pod \"console-798f5678bc-wm5rv\" (UID: \"1f661790-6ef9-4a48-9ea1-40e2e42a76d2\") " pod="openshift-console/console-798f5678bc-wm5rv" Jul 28 01:32:57.928869 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:32:57.928717 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f661790-6ef9-4a48-9ea1-40e2e42a76d2-trusted-ca-bundle\") pod \"console-798f5678bc-wm5rv\" (UID: \"1f661790-6ef9-4a48-9ea1-40e2e42a76d2\") " pod="openshift-console/console-798f5678bc-wm5rv" Jul 28 01:32:57.928869 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:32:57.928775 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grfqw\" (UniqueName: \"kubernetes.io/projected/1f661790-6ef9-4a48-9ea1-40e2e42a76d2-kube-api-access-grfqw\") pod \"console-798f5678bc-wm5rv\" (UID: \"1f661790-6ef9-4a48-9ea1-40e2e42a76d2\") " pod="openshift-console/console-798f5678bc-wm5rv" Jul 28 01:32:58.029320 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:32:58.029249 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f661790-6ef9-4a48-9ea1-40e2e42a76d2-console-serving-cert\") pod \"console-798f5678bc-wm5rv\" (UID: \"1f661790-6ef9-4a48-9ea1-40e2e42a76d2\") " pod="openshift-console/console-798f5678bc-wm5rv" Jul 28 01:32:58.029320 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:32:58.029279 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f661790-6ef9-4a48-9ea1-40e2e42a76d2-trusted-ca-bundle\") pod \"console-798f5678bc-wm5rv\" (UID: \"1f661790-6ef9-4a48-9ea1-40e2e42a76d2\") " pod="openshift-console/console-798f5678bc-wm5rv" Jul 28 01:32:58.029579 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:32:58.029551 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grfqw\" (UniqueName: \"kubernetes.io/projected/1f661790-6ef9-4a48-9ea1-40e2e42a76d2-kube-api-access-grfqw\") pod \"console-798f5678bc-wm5rv\" (UID: \"1f661790-6ef9-4a48-9ea1-40e2e42a76d2\") " pod="openshift-console/console-798f5678bc-wm5rv" Jul 28 01:32:58.029740 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:32:58.029625 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f661790-6ef9-4a48-9ea1-40e2e42a76d2-console-config\") pod \"console-798f5678bc-wm5rv\" (UID: \"1f661790-6ef9-4a48-9ea1-40e2e42a76d2\") " pod="openshift-console/console-798f5678bc-wm5rv" Jul 28 01:32:58.029740 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:32:58.029655 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f661790-6ef9-4a48-9ea1-40e2e42a76d2-service-ca\") pod \"console-798f5678bc-wm5rv\" (UID: \"1f661790-6ef9-4a48-9ea1-40e2e42a76d2\") " pod="openshift-console/console-798f5678bc-wm5rv" Jul 28 01:32:58.029740 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:32:58.029694 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f661790-6ef9-4a48-9ea1-40e2e42a76d2-console-oauth-config\") pod \"console-798f5678bc-wm5rv\" (UID: \"1f661790-6ef9-4a48-9ea1-40e2e42a76d2\") " pod="openshift-console/console-798f5678bc-wm5rv" Jul 28 01:32:58.029740 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:32:58.029716 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f661790-6ef9-4a48-9ea1-40e2e42a76d2-oauth-serving-cert\") pod \"console-798f5678bc-wm5rv\" (UID: \"1f661790-6ef9-4a48-9ea1-40e2e42a76d2\") " pod="openshift-console/console-798f5678bc-wm5rv" Jul 28 01:32:58.030223 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:32:58.030198 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f661790-6ef9-4a48-9ea1-40e2e42a76d2-trusted-ca-bundle\") pod \"console-798f5678bc-wm5rv\" (UID: \"1f661790-6ef9-4a48-9ea1-40e2e42a76d2\") " pod="openshift-console/console-798f5678bc-wm5rv" Jul 28 01:32:58.030323 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:32:58.030287 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f661790-6ef9-4a48-9ea1-40e2e42a76d2-console-config\") pod \"console-798f5678bc-wm5rv\" (UID: \"1f661790-6ef9-4a48-9ea1-40e2e42a76d2\") " pod="openshift-console/console-798f5678bc-wm5rv" Jul 28 01:32:58.030384 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:32:58.030370 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f661790-6ef9-4a48-9ea1-40e2e42a76d2-oauth-serving-cert\") pod \"console-798f5678bc-wm5rv\" (UID: \"1f661790-6ef9-4a48-9ea1-40e2e42a76d2\") " pod="openshift-console/console-798f5678bc-wm5rv" Jul 28 01:32:58.030445 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:32:58.030371 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f661790-6ef9-4a48-9ea1-40e2e42a76d2-service-ca\") pod \"console-798f5678bc-wm5rv\" (UID: \"1f661790-6ef9-4a48-9ea1-40e2e42a76d2\") " pod="openshift-console/console-798f5678bc-wm5rv" Jul 28 01:32:58.031895 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:32:58.031876 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f661790-6ef9-4a48-9ea1-40e2e42a76d2-console-serving-cert\") pod \"console-798f5678bc-wm5rv\" (UID: \"1f661790-6ef9-4a48-9ea1-40e2e42a76d2\") " pod="openshift-console/console-798f5678bc-wm5rv" Jul 28 01:32:58.032130 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:32:58.032114 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f661790-6ef9-4a48-9ea1-40e2e42a76d2-console-oauth-config\") pod \"console-798f5678bc-wm5rv\" (UID: \"1f661790-6ef9-4a48-9ea1-40e2e42a76d2\") " pod="openshift-console/console-798f5678bc-wm5rv" Jul 28 01:32:58.039022 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:32:58.038994 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grfqw\" (UniqueName: \"kubernetes.io/projected/1f661790-6ef9-4a48-9ea1-40e2e42a76d2-kube-api-access-grfqw\") pod \"console-798f5678bc-wm5rv\" (UID: \"1f661790-6ef9-4a48-9ea1-40e2e42a76d2\") " pod="openshift-console/console-798f5678bc-wm5rv" Jul 28 01:32:58.088995 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:32:58.088968 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-798f5678bc-wm5rv" Jul 28 01:32:58.220453 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:32:58.220396 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-798f5678bc-wm5rv"] Jul 28 01:32:58.223309 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:32:58.223283 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f661790_6ef9_4a48_9ea1_40e2e42a76d2.slice/crio-441733b963843c2b2f0429f5fca1865a6bb6393ae02880aa7fab08e85437226c WatchSource:0}: Error finding container 441733b963843c2b2f0429f5fca1865a6bb6393ae02880aa7fab08e85437226c: Status 404 returned error can't find the container with id 441733b963843c2b2f0429f5fca1865a6bb6393ae02880aa7fab08e85437226c Jul 28 01:32:58.994657 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:32:58.994620 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-798f5678bc-wm5rv" event={"ID":"1f661790-6ef9-4a48-9ea1-40e2e42a76d2","Type":"ContainerStarted","Data":"7c7fc35e8d76f313034b1c0d2f4f3feba2cfbaa20d7cac932ee30a73d695ae10"} Jul 28 01:32:58.994657 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:32:58.994664 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-798f5678bc-wm5rv" event={"ID":"1f661790-6ef9-4a48-9ea1-40e2e42a76d2","Type":"ContainerStarted","Data":"441733b963843c2b2f0429f5fca1865a6bb6393ae02880aa7fab08e85437226c"} Jul 28 01:32:59.016434 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:32:59.016391 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-798f5678bc-wm5rv" podStartSLOduration=2.016378621 podStartE2EDuration="2.016378621s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:32:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:32:59.014338853 +0000 UTC m=+417.568463654" watchObservedRunningTime="2026-07-28 01:32:59.016378621 +0000 UTC m=+417.570503419" Jul 28 01:33:00.049596 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:00.049563 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-64568bcd49-5jv79"] Jul 28 01:33:00.052187 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:00.052170 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-5jv79" Jul 28 01:33:00.054526 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:00.054508 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\"" Jul 28 01:33:00.055510 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:00.055492 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-spmss\"" Jul 28 01:33:00.055605 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:00.055524 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Jul 28 01:33:00.056409 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:00.056388 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Jul 28 01:33:00.062801 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:00.062780 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-5jv79"] Jul 28 01:33:00.147144 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:00.147115 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9xk8\" (UniqueName: \"kubernetes.io/projected/4da4e640-b010-44d0-b78d-e6eb94c03309-kube-api-access-d9xk8\") pod \"seaweedfs-64568bcd49-5jv79\" (UID: \"4da4e640-b010-44d0-b78d-e6eb94c03309\") " pod="kserve/seaweedfs-64568bcd49-5jv79" Jul 28 01:33:00.147309 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:00.147210 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4da4e640-b010-44d0-b78d-e6eb94c03309-data\") pod \"seaweedfs-64568bcd49-5jv79\" (UID: \"4da4e640-b010-44d0-b78d-e6eb94c03309\") " pod="kserve/seaweedfs-64568bcd49-5jv79" Jul 28 01:33:00.248280 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:00.248252 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9xk8\" (UniqueName: \"kubernetes.io/projected/4da4e640-b010-44d0-b78d-e6eb94c03309-kube-api-access-d9xk8\") pod \"seaweedfs-64568bcd49-5jv79\" (UID: \"4da4e640-b010-44d0-b78d-e6eb94c03309\") " pod="kserve/seaweedfs-64568bcd49-5jv79" Jul 28 01:33:00.248280 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:00.248283 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4da4e640-b010-44d0-b78d-e6eb94c03309-data\") pod \"seaweedfs-64568bcd49-5jv79\" (UID: \"4da4e640-b010-44d0-b78d-e6eb94c03309\") " pod="kserve/seaweedfs-64568bcd49-5jv79" Jul 28 01:33:00.248617 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:00.248602 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4da4e640-b010-44d0-b78d-e6eb94c03309-data\") pod \"seaweedfs-64568bcd49-5jv79\" (UID: \"4da4e640-b010-44d0-b78d-e6eb94c03309\") " pod="kserve/seaweedfs-64568bcd49-5jv79" Jul 28 01:33:00.257236 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:00.257216 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9xk8\" (UniqueName: \"kubernetes.io/projected/4da4e640-b010-44d0-b78d-e6eb94c03309-kube-api-access-d9xk8\") pod \"seaweedfs-64568bcd49-5jv79\" (UID: \"4da4e640-b010-44d0-b78d-e6eb94c03309\") " pod="kserve/seaweedfs-64568bcd49-5jv79" Jul 28 01:33:00.361478 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:00.361417 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-5jv79" Jul 28 01:33:00.492194 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:00.492172 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-5jv79"] Jul 28 01:33:00.494091 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:33:00.494058 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4da4e640_b010_44d0_b78d_e6eb94c03309.slice/crio-1ea2b40b11cc933b559b1b4e10971b0ddf5263771bf8563d2dd8ab477344029f WatchSource:0}: Error finding container 1ea2b40b11cc933b559b1b4e10971b0ddf5263771bf8563d2dd8ab477344029f: Status 404 returned error can't find the container with id 1ea2b40b11cc933b559b1b4e10971b0ddf5263771bf8563d2dd8ab477344029f Jul 28 01:33:01.008728 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:01.008696 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-5jv79" event={"ID":"4da4e640-b010-44d0-b78d-e6eb94c03309","Type":"ContainerStarted","Data":"1ea2b40b11cc933b559b1b4e10971b0ddf5263771bf8563d2dd8ab477344029f"} Jul 28 01:33:03.477957 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:03.477935 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\"" Jul 28 01:33:04.024480 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:04.024447 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-5jv79" event={"ID":"4da4e640-b010-44d0-b78d-e6eb94c03309","Type":"ContainerStarted","Data":"8a049315fbfe667cc8dbbef700a7b7cc8d5b2eb074313227b30eaba213f2febd"} Jul 28 01:33:04.024652 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:04.024591 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/seaweedfs-64568bcd49-5jv79" Jul 28 01:33:04.042274 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:04.042230 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-64568bcd49-5jv79" podStartSLOduration=1.062420918 podStartE2EDuration="4.042215613s" totalImagesPullingTime="2.979794695s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:33:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:33:04.040626087 +0000 UTC m=+422.594750885" watchObservedRunningTime="2026-07-28 01:33:04.042215613 +0000 UTC m=+422.596340411" Jul 28 01:33:08.089149 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:08.089113 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-798f5678bc-wm5rv" Jul 28 01:33:08.089524 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:08.089170 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-798f5678bc-wm5rv" Jul 28 01:33:08.094030 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:08.094008 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-798f5678bc-wm5rv" Jul 28 01:33:09.055092 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:09.055066 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-798f5678bc-wm5rv" Jul 28 01:33:09.106313 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:09.106281 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d8557d54f-wn4vp"] Jul 28 01:33:10.031486 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:10.031458 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/seaweedfs-64568bcd49-5jv79" Jul 28 01:33:34.133026 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:34.132920 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5d8557d54f-wn4vp" podUID="a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d" containerName="console" containerID="cri-o://3d05038ee4498de98498516a5cca23fed142a3a30472c08619e793f5847f38a3" gracePeriod=15 Jul 28 01:33:34.392258 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:34.392199 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d8557d54f-wn4vp" Jul 28 01:33:34.436893 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:34.436861 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d-console-serving-cert\") pod \"a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d\" (UID: \"a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d\") " Jul 28 01:33:34.436893 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:34.436899 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d-trusted-ca-bundle\") pod \"a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d\" (UID: \"a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d\") " Jul 28 01:33:34.437169 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:34.436960 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d-console-config\") pod \"a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d\" (UID: \"a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d\") " Jul 28 01:33:34.437169 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:34.437000 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d-service-ca\") pod \"a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d\" (UID: \"a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d\") " Jul 28 01:33:34.437169 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:34.437117 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d-console-oauth-config\") pod \"a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d\" (UID: \"a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d\") " Jul 28 01:33:34.437414 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:34.437176 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d-oauth-serving-cert\") pod \"a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d\" (UID: \"a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d\") " Jul 28 01:33:34.437414 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:34.437234 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6lp2t\" (UniqueName: \"kubernetes.io/projected/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d-kube-api-access-6lp2t\") pod \"a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d\" (UID: \"a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d\") " Jul 28 01:33:34.437414 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:34.437388 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d-console-config" (OuterVolumeSpecName: "console-config") pod "a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d" (UID: "a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 01:33:34.437581 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:34.437396 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d" (UID: "a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 01:33:34.437664 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:34.437647 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d-trusted-ca-bundle\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:33:34.437720 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:34.437669 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d-console-config\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:33:34.437720 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:34.437666 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d" (UID: "a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 01:33:34.437830 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:34.437710 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d-service-ca" (OuterVolumeSpecName: "service-ca") pod "a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d" (UID: "a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 01:33:34.439448 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:34.439423 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d-kube-api-access-6lp2t" (OuterVolumeSpecName: "kube-api-access-6lp2t") pod "a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d" (UID: "a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d"). InnerVolumeSpecName "kube-api-access-6lp2t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 01:33:34.439590 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:34.439422 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d" (UID: "a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 01:33:34.439590 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:34.439448 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d" (UID: "a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 01:33:34.538274 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:34.538243 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d-service-ca\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:33:34.538274 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:34.538269 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d-console-oauth-config\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:33:34.538274 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:34.538278 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d-oauth-serving-cert\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:33:34.538582 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:34.538286 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6lp2t\" (UniqueName: \"kubernetes.io/projected/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d-kube-api-access-6lp2t\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:33:34.538582 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:34.538296 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d-console-serving-cert\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:33:35.170260 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:35.170224 2578 generic.go:356] "Generic (PLEG): container finished" podID="a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d" containerID="3d05038ee4498de98498516a5cca23fed142a3a30472c08619e793f5847f38a3" exitCode=0 Jul 28 01:33:35.170756 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:35.170268 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d8557d54f-wn4vp" Jul 28 01:33:35.170756 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:35.170314 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d8557d54f-wn4vp" event={"ID":"a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d","Type":"ContainerDied","Data":"3d05038ee4498de98498516a5cca23fed142a3a30472c08619e793f5847f38a3"} Jul 28 01:33:35.170756 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:35.170372 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d8557d54f-wn4vp" event={"ID":"a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d","Type":"ContainerDied","Data":"a6f1ffbca4fdace333627662afd3db13968577ef0bc1d04d7642df06144d491e"} Jul 28 01:33:35.170756 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:35.170403 2578 scope.go:117] "RemoveContainer" containerID="3d05038ee4498de98498516a5cca23fed142a3a30472c08619e793f5847f38a3" Jul 28 01:33:35.179342 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:35.179327 2578 scope.go:117] "RemoveContainer" containerID="3d05038ee4498de98498516a5cca23fed142a3a30472c08619e793f5847f38a3" Jul 28 01:33:35.179612 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:33:35.179595 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d05038ee4498de98498516a5cca23fed142a3a30472c08619e793f5847f38a3\": container with ID starting with 3d05038ee4498de98498516a5cca23fed142a3a30472c08619e793f5847f38a3 not found: ID does not exist" containerID="3d05038ee4498de98498516a5cca23fed142a3a30472c08619e793f5847f38a3" Jul 28 01:33:35.179677 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:35.179622 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d05038ee4498de98498516a5cca23fed142a3a30472c08619e793f5847f38a3"} err="failed to get container status \"3d05038ee4498de98498516a5cca23fed142a3a30472c08619e793f5847f38a3\": rpc error: code = NotFound desc = could not find container \"3d05038ee4498de98498516a5cca23fed142a3a30472c08619e793f5847f38a3\": container with ID starting with 3d05038ee4498de98498516a5cca23fed142a3a30472c08619e793f5847f38a3 not found: ID does not exist" Jul 28 01:33:35.190428 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:35.190407 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d8557d54f-wn4vp"] Jul 28 01:33:35.193822 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:35.193805 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5d8557d54f-wn4vp"] Jul 28 01:33:36.010142 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:33:36.010108 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d" path="/var/lib/kubelet/pods/a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d/volumes" Jul 28 01:34:18.979256 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:34:18.979223 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/model-serving-api-7b954ffc95-n97lj"] Jul 28 01:34:18.979832 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:34:18.979793 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d" containerName="console" Jul 28 01:34:18.979832 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:34:18.979814 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d" containerName="console" Jul 28 01:34:18.979952 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:34:18.979925 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9457bfc-9a76-42e5-bfd0-86d3b6bdbf8d" containerName="console" Jul 28 01:34:18.981896 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:34:18.981876 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-7b954ffc95-n97lj" Jul 28 01:34:18.984361 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:34:18.984340 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-tls\"" Jul 28 01:34:18.984461 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:34:18.984393 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-dockercfg-txjrb\"" Jul 28 01:34:18.993076 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:34:18.993058 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-7b954ffc95-n97lj"] Jul 28 01:34:18.996909 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:34:18.996892 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-75d9cdbddf-zfd9w"] Jul 28 01:34:18.999048 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:34:18.999035 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-75d9cdbddf-zfd9w" Jul 28 01:34:19.001262 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:34:19.001222 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-gk9mz\"" Jul 28 01:34:19.001365 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:34:19.001315 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\"" Jul 28 01:34:19.010720 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:34:19.010703 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-75d9cdbddf-zfd9w"] Jul 28 01:34:19.084171 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:34:19.084144 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8ae9bd87-7366-4663-bb27-883eca661d42-cert\") pod \"odh-model-controller-75d9cdbddf-zfd9w\" (UID: \"8ae9bd87-7366-4663-bb27-883eca661d42\") " pod="kserve/odh-model-controller-75d9cdbddf-zfd9w" Jul 28 01:34:19.084283 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:34:19.084181 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ae618a2-5cd5-40e0-a75e-8158448063dc-tls-certs\") pod \"model-serving-api-7b954ffc95-n97lj\" (UID: \"7ae618a2-5cd5-40e0-a75e-8158448063dc\") " pod="kserve/model-serving-api-7b954ffc95-n97lj" Jul 28 01:34:19.084283 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:34:19.084198 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hj5qq\" (UniqueName: \"kubernetes.io/projected/7ae618a2-5cd5-40e0-a75e-8158448063dc-kube-api-access-hj5qq\") pod \"model-serving-api-7b954ffc95-n97lj\" (UID: \"7ae618a2-5cd5-40e0-a75e-8158448063dc\") " pod="kserve/model-serving-api-7b954ffc95-n97lj" Jul 28 01:34:19.084283 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:34:19.084276 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqj4g\" (UniqueName: \"kubernetes.io/projected/8ae9bd87-7366-4663-bb27-883eca661d42-kube-api-access-wqj4g\") pod \"odh-model-controller-75d9cdbddf-zfd9w\" (UID: \"8ae9bd87-7366-4663-bb27-883eca661d42\") " pod="kserve/odh-model-controller-75d9cdbddf-zfd9w" Jul 28 01:34:19.185159 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:34:19.185136 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8ae9bd87-7366-4663-bb27-883eca661d42-cert\") pod \"odh-model-controller-75d9cdbddf-zfd9w\" (UID: \"8ae9bd87-7366-4663-bb27-883eca661d42\") " pod="kserve/odh-model-controller-75d9cdbddf-zfd9w" Jul 28 01:34:19.185257 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:34:19.185167 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ae618a2-5cd5-40e0-a75e-8158448063dc-tls-certs\") pod \"model-serving-api-7b954ffc95-n97lj\" (UID: \"7ae618a2-5cd5-40e0-a75e-8158448063dc\") " pod="kserve/model-serving-api-7b954ffc95-n97lj" Jul 28 01:34:19.185257 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:34:19.185185 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hj5qq\" (UniqueName: \"kubernetes.io/projected/7ae618a2-5cd5-40e0-a75e-8158448063dc-kube-api-access-hj5qq\") pod \"model-serving-api-7b954ffc95-n97lj\" (UID: \"7ae618a2-5cd5-40e0-a75e-8158448063dc\") " pod="kserve/model-serving-api-7b954ffc95-n97lj" Jul 28 01:34:19.185257 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:34:19.185240 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqj4g\" (UniqueName: \"kubernetes.io/projected/8ae9bd87-7366-4663-bb27-883eca661d42-kube-api-access-wqj4g\") pod \"odh-model-controller-75d9cdbddf-zfd9w\" (UID: \"8ae9bd87-7366-4663-bb27-883eca661d42\") " pod="kserve/odh-model-controller-75d9cdbddf-zfd9w" Jul 28 01:34:19.187597 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:34:19.187578 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ae618a2-5cd5-40e0-a75e-8158448063dc-tls-certs\") pod \"model-serving-api-7b954ffc95-n97lj\" (UID: \"7ae618a2-5cd5-40e0-a75e-8158448063dc\") " pod="kserve/model-serving-api-7b954ffc95-n97lj" Jul 28 01:34:19.187681 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:34:19.187653 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8ae9bd87-7366-4663-bb27-883eca661d42-cert\") pod \"odh-model-controller-75d9cdbddf-zfd9w\" (UID: \"8ae9bd87-7366-4663-bb27-883eca661d42\") " pod="kserve/odh-model-controller-75d9cdbddf-zfd9w" Jul 28 01:34:19.193846 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:34:19.193823 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hj5qq\" (UniqueName: \"kubernetes.io/projected/7ae618a2-5cd5-40e0-a75e-8158448063dc-kube-api-access-hj5qq\") pod \"model-serving-api-7b954ffc95-n97lj\" (UID: \"7ae618a2-5cd5-40e0-a75e-8158448063dc\") " pod="kserve/model-serving-api-7b954ffc95-n97lj" Jul 28 01:34:19.193936 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:34:19.193832 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqj4g\" (UniqueName: \"kubernetes.io/projected/8ae9bd87-7366-4663-bb27-883eca661d42-kube-api-access-wqj4g\") pod \"odh-model-controller-75d9cdbddf-zfd9w\" (UID: \"8ae9bd87-7366-4663-bb27-883eca661d42\") " pod="kserve/odh-model-controller-75d9cdbddf-zfd9w" Jul 28 01:34:19.292642 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:34:19.292608 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-7b954ffc95-n97lj" Jul 28 01:34:19.309523 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:34:19.309505 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-75d9cdbddf-zfd9w" Jul 28 01:34:19.432647 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:34:19.432624 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-7b954ffc95-n97lj"] Jul 28 01:34:19.435256 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:34:19.435225 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ae618a2_5cd5_40e0_a75e_8158448063dc.slice/crio-6a748605fa3d1a11a39c93adf7caa73c547e4587af78927a02a0859603bcb598 WatchSource:0}: Error finding container 6a748605fa3d1a11a39c93adf7caa73c547e4587af78927a02a0859603bcb598: Status 404 returned error can't find the container with id 6a748605fa3d1a11a39c93adf7caa73c547e4587af78927a02a0859603bcb598 Jul 28 01:34:19.451384 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:34:19.451361 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-75d9cdbddf-zfd9w"] Jul 28 01:34:19.452758 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:34:19.452731 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ae9bd87_7366_4663_bb27_883eca661d42.slice/crio-594c5f5225ffcc428a44152097ba27b16308246250fdf15d31801911b074c242 WatchSource:0}: Error finding container 594c5f5225ffcc428a44152097ba27b16308246250fdf15d31801911b074c242: Status 404 returned error can't find the container with id 594c5f5225ffcc428a44152097ba27b16308246250fdf15d31801911b074c242 Jul 28 01:34:20.379312 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:34:20.379277 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-75d9cdbddf-zfd9w" event={"ID":"8ae9bd87-7366-4663-bb27-883eca661d42","Type":"ContainerStarted","Data":"594c5f5225ffcc428a44152097ba27b16308246250fdf15d31801911b074c242"} Jul 28 01:34:20.381427 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:34:20.381395 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-7b954ffc95-n97lj" event={"ID":"7ae618a2-5cd5-40e0-a75e-8158448063dc","Type":"ContainerStarted","Data":"6a748605fa3d1a11a39c93adf7caa73c547e4587af78927a02a0859603bcb598"} Jul 28 01:34:23.398171 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:34:23.398128 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-75d9cdbddf-zfd9w" event={"ID":"8ae9bd87-7366-4663-bb27-883eca661d42","Type":"ContainerStarted","Data":"ad852c6813705268dae71bbda85b558cd9b120dca9ecf2d5194240ce2a761f22"} Jul 28 01:34:23.398608 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:34:23.398227 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-75d9cdbddf-zfd9w" Jul 28 01:34:23.399662 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:34:23.399637 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-7b954ffc95-n97lj" event={"ID":"7ae618a2-5cd5-40e0-a75e-8158448063dc","Type":"ContainerStarted","Data":"c177632425e81a4ceff35bdcf1f1bf6bf0a790148a7b78c43a8015cda84dba4a"} Jul 28 01:34:23.399803 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:34:23.399688 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/model-serving-api-7b954ffc95-n97lj" Jul 28 01:34:23.419792 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:34:23.419742 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/odh-model-controller-75d9cdbddf-zfd9w" podStartSLOduration=2.057015004 podStartE2EDuration="5.419723718s" totalImagesPullingTime="3.362708714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:34:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:34:23.417846148 +0000 UTC m=+501.971970946" watchObservedRunningTime="2026-07-28 01:34:23.419723718 +0000 UTC m=+501.973848517" Jul 28 01:34:23.434800 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:34:23.434740 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/model-serving-api-7b954ffc95-n97lj" podStartSLOduration=2.0670017019999998 podStartE2EDuration="5.434727273s" totalImagesPullingTime="3.367725571s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:34:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:34:23.433522693 +0000 UTC m=+501.987647487" watchObservedRunningTime="2026-07-28 01:34:23.434727273 +0000 UTC m=+501.988852070" Jul 28 01:34:34.407564 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:34:34.407523 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-75d9cdbddf-zfd9w" Jul 28 01:34:34.409351 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:34:34.409332 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/model-serving-api-7b954ffc95-n97lj" Jul 28 01:36:01.840059 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:36:01.840032 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-k2wlm_3503116a-1398-42ea-ac86-743f5f51e989/console-operator/1.log" Jul 28 01:36:01.842852 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:36:01.842830 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-k2wlm_3503116a-1398-42ea-ac86-743f5f51e989/console-operator/1.log" Jul 28 01:39:31.872168 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:39:31.872076 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-b5420-6978dfb8f7-sb7sp"] Jul 28 01:39:31.875754 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:39:31.875732 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-b5420-6978dfb8f7-sb7sp" Jul 28 01:39:31.878447 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:39:31.878425 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-raw-b5420-serving-cert\"" Jul 28 01:39:31.878447 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:39:31.878442 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-cl94s\"" Jul 28 01:39:31.878620 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:39:31.878426 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-raw-b5420-kube-rbac-proxy-sar-config\"" Jul 28 01:39:31.879034 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:39:31.879021 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 28 01:39:31.885396 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:39:31.885370 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-b5420-6978dfb8f7-sb7sp"] Jul 28 01:39:31.991530 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:39:31.991501 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/baea18be-28fe-4026-9c81-dc3046df4a57-proxy-tls\") pod \"model-chainer-raw-b5420-6978dfb8f7-sb7sp\" (UID: \"baea18be-28fe-4026-9c81-dc3046df4a57\") " pod="kserve-ci-e2e-test/model-chainer-raw-b5420-6978dfb8f7-sb7sp" Jul 28 01:39:31.991530 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:39:31.991534 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/baea18be-28fe-4026-9c81-dc3046df4a57-openshift-service-ca-bundle\") pod \"model-chainer-raw-b5420-6978dfb8f7-sb7sp\" (UID: \"baea18be-28fe-4026-9c81-dc3046df4a57\") " pod="kserve-ci-e2e-test/model-chainer-raw-b5420-6978dfb8f7-sb7sp" Jul 28 01:39:32.092456 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:39:32.092417 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/baea18be-28fe-4026-9c81-dc3046df4a57-proxy-tls\") pod \"model-chainer-raw-b5420-6978dfb8f7-sb7sp\" (UID: \"baea18be-28fe-4026-9c81-dc3046df4a57\") " pod="kserve-ci-e2e-test/model-chainer-raw-b5420-6978dfb8f7-sb7sp" Jul 28 01:39:32.092682 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:39:32.092463 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/baea18be-28fe-4026-9c81-dc3046df4a57-openshift-service-ca-bundle\") pod \"model-chainer-raw-b5420-6978dfb8f7-sb7sp\" (UID: \"baea18be-28fe-4026-9c81-dc3046df4a57\") " pod="kserve-ci-e2e-test/model-chainer-raw-b5420-6978dfb8f7-sb7sp" Jul 28 01:39:32.092682 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:39:32.092586 2578 secret.go:189] Couldn't get secret kserve-ci-e2e-test/model-chainer-raw-b5420-serving-cert: secret "model-chainer-raw-b5420-serving-cert" not found Jul 28 01:39:32.092682 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:39:32.092652 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/baea18be-28fe-4026-9c81-dc3046df4a57-proxy-tls podName:baea18be-28fe-4026-9c81-dc3046df4a57 nodeName:}" failed. No retries permitted until 2026-07-28 01:39:32.592636455 +0000 UTC m=+811.146761231 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/baea18be-28fe-4026-9c81-dc3046df4a57-proxy-tls") pod "model-chainer-raw-b5420-6978dfb8f7-sb7sp" (UID: "baea18be-28fe-4026-9c81-dc3046df4a57") : secret "model-chainer-raw-b5420-serving-cert" not found Jul 28 01:39:32.093081 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:39:32.093061 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/baea18be-28fe-4026-9c81-dc3046df4a57-openshift-service-ca-bundle\") pod \"model-chainer-raw-b5420-6978dfb8f7-sb7sp\" (UID: \"baea18be-28fe-4026-9c81-dc3046df4a57\") " pod="kserve-ci-e2e-test/model-chainer-raw-b5420-6978dfb8f7-sb7sp" Jul 28 01:39:32.598337 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:39:32.598297 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/baea18be-28fe-4026-9c81-dc3046df4a57-proxy-tls\") pod \"model-chainer-raw-b5420-6978dfb8f7-sb7sp\" (UID: \"baea18be-28fe-4026-9c81-dc3046df4a57\") " pod="kserve-ci-e2e-test/model-chainer-raw-b5420-6978dfb8f7-sb7sp" Jul 28 01:39:32.600675 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:39:32.600646 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/baea18be-28fe-4026-9c81-dc3046df4a57-proxy-tls\") pod \"model-chainer-raw-b5420-6978dfb8f7-sb7sp\" (UID: \"baea18be-28fe-4026-9c81-dc3046df4a57\") " pod="kserve-ci-e2e-test/model-chainer-raw-b5420-6978dfb8f7-sb7sp" Jul 28 01:39:32.786372 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:39:32.786337 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-b5420-6978dfb8f7-sb7sp" Jul 28 01:39:32.917320 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:39:32.917291 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-b5420-6978dfb8f7-sb7sp"] Jul 28 01:39:32.919620 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:39:32.919594 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaea18be_28fe_4026_9c81_dc3046df4a57.slice/crio-ef2ceb4880b98994b1103599b1e97c04097e9f6802cab5e69ce5a991cbea1603 WatchSource:0}: Error finding container ef2ceb4880b98994b1103599b1e97c04097e9f6802cab5e69ce5a991cbea1603: Status 404 returned error can't find the container with id ef2ceb4880b98994b1103599b1e97c04097e9f6802cab5e69ce5a991cbea1603 Jul 28 01:39:32.921498 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:39:32.921480 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 01:39:33.841387 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:39:33.841344 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-b5420-6978dfb8f7-sb7sp" event={"ID":"baea18be-28fe-4026-9c81-dc3046df4a57","Type":"ContainerStarted","Data":"ef2ceb4880b98994b1103599b1e97c04097e9f6802cab5e69ce5a991cbea1603"} Jul 28 01:39:34.847560 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:39:34.847515 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-b5420-6978dfb8f7-sb7sp" event={"ID":"baea18be-28fe-4026-9c81-dc3046df4a57","Type":"ContainerStarted","Data":"0b7fd931f877a0f5bb5fce43fcbf40721a7dffaf533aa684330c90afa903a4ec"} Jul 28 01:39:34.848013 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:39:34.847610 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-raw-b5420-6978dfb8f7-sb7sp" Jul 28 01:39:34.868462 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:39:34.868414 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/model-chainer-raw-b5420-6978dfb8f7-sb7sp" podStartSLOduration=2.143043001 podStartE2EDuration="3.868399104s" totalImagesPullingTime="1.725356103s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:39:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:39:34.867022697 +0000 UTC m=+813.421147497" watchObservedRunningTime="2026-07-28 01:39:34.868399104 +0000 UTC m=+813.422523902" Jul 28 01:39:40.858505 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:39:40.858468 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/model-chainer-raw-b5420-6978dfb8f7-sb7sp" Jul 28 01:39:41.909767 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:39:41.909733 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-b5420-6978dfb8f7-sb7sp"] Jul 28 01:39:41.910247 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:39:41.909952 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/model-chainer-raw-b5420-6978dfb8f7-sb7sp" podUID="baea18be-28fe-4026-9c81-dc3046df4a57" containerName="model-chainer-raw-b5420" containerID="cri-o://0b7fd931f877a0f5bb5fce43fcbf40721a7dffaf533aa684330c90afa903a4ec" gracePeriod=30 Jul 28 01:39:45.856810 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:39:45.856769 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-b5420-6978dfb8f7-sb7sp" podUID="baea18be-28fe-4026-9c81-dc3046df4a57" containerName="model-chainer-raw-b5420" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 01:39:50.857029 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:39:50.856993 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-b5420-6978dfb8f7-sb7sp" podUID="baea18be-28fe-4026-9c81-dc3046df4a57" containerName="model-chainer-raw-b5420" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 01:39:55.856859 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:39:55.856817 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-b5420-6978dfb8f7-sb7sp" podUID="baea18be-28fe-4026-9c81-dc3046df4a57" containerName="model-chainer-raw-b5420" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 01:39:55.857332 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:39:55.856951 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-raw-b5420-6978dfb8f7-sb7sp" Jul 28 01:40:00.856057 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:40:00.856019 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-b5420-6978dfb8f7-sb7sp" podUID="baea18be-28fe-4026-9c81-dc3046df4a57" containerName="model-chainer-raw-b5420" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 01:40:05.856626 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:40:05.856579 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-b5420-6978dfb8f7-sb7sp" podUID="baea18be-28fe-4026-9c81-dc3046df4a57" containerName="model-chainer-raw-b5420" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 01:40:10.856610 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:40:10.856571 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-b5420-6978dfb8f7-sb7sp" podUID="baea18be-28fe-4026-9c81-dc3046df4a57" containerName="model-chainer-raw-b5420" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 01:40:12.021111 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:40:12.021075 2578 generic.go:356] "Generic (PLEG): container finished" podID="baea18be-28fe-4026-9c81-dc3046df4a57" containerID="0b7fd931f877a0f5bb5fce43fcbf40721a7dffaf533aa684330c90afa903a4ec" exitCode=0 Jul 28 01:40:12.021458 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:40:12.021145 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-b5420-6978dfb8f7-sb7sp" event={"ID":"baea18be-28fe-4026-9c81-dc3046df4a57","Type":"ContainerDied","Data":"0b7fd931f877a0f5bb5fce43fcbf40721a7dffaf533aa684330c90afa903a4ec"} Jul 28 01:40:12.064862 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:40:12.064837 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-b5420-6978dfb8f7-sb7sp" Jul 28 01:40:12.123334 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:40:12.123304 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/baea18be-28fe-4026-9c81-dc3046df4a57-openshift-service-ca-bundle\") pod \"baea18be-28fe-4026-9c81-dc3046df4a57\" (UID: \"baea18be-28fe-4026-9c81-dc3046df4a57\") " Jul 28 01:40:12.123489 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:40:12.123450 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/baea18be-28fe-4026-9c81-dc3046df4a57-proxy-tls\") pod \"baea18be-28fe-4026-9c81-dc3046df4a57\" (UID: \"baea18be-28fe-4026-9c81-dc3046df4a57\") " Jul 28 01:40:12.123747 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:40:12.123711 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/baea18be-28fe-4026-9c81-dc3046df4a57-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "baea18be-28fe-4026-9c81-dc3046df4a57" (UID: "baea18be-28fe-4026-9c81-dc3046df4a57"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 01:40:12.123893 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:40:12.123817 2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/baea18be-28fe-4026-9c81-dc3046df4a57-openshift-service-ca-bundle\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:40:12.125617 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:40:12.125595 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/baea18be-28fe-4026-9c81-dc3046df4a57-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "baea18be-28fe-4026-9c81-dc3046df4a57" (UID: "baea18be-28fe-4026-9c81-dc3046df4a57"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 01:40:12.224911 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:40:12.224830 2578 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/baea18be-28fe-4026-9c81-dc3046df4a57-proxy-tls\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:40:13.026797 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:40:13.026769 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-b5420-6978dfb8f7-sb7sp" Jul 28 01:40:13.027228 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:40:13.026769 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-b5420-6978dfb8f7-sb7sp" event={"ID":"baea18be-28fe-4026-9c81-dc3046df4a57","Type":"ContainerDied","Data":"ef2ceb4880b98994b1103599b1e97c04097e9f6802cab5e69ce5a991cbea1603"} Jul 28 01:40:13.027228 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:40:13.026884 2578 scope.go:117] "RemoveContainer" containerID="0b7fd931f877a0f5bb5fce43fcbf40721a7dffaf533aa684330c90afa903a4ec" Jul 28 01:40:13.048573 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:40:13.048527 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-b5420-6978dfb8f7-sb7sp"] Jul 28 01:40:13.051732 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:40:13.051709 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-b5420-6978dfb8f7-sb7sp"] Jul 28 01:40:14.009777 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:40:14.009745 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="baea18be-28fe-4026-9c81-dc3046df4a57" path="/var/lib/kubelet/pods/baea18be-28fe-4026-9c81-dc3046df4a57/volumes" Jul 28 01:41:01.872608 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:41:01.872581 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-k2wlm_3503116a-1398-42ea-ac86-743f5f51e989/console-operator/1.log" Jul 28 01:41:01.873063 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:41:01.872615 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-k2wlm_3503116a-1398-42ea-ac86-743f5f51e989/console-operator/1.log" Jul 28 01:41:22.184760 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:41:22.184725 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-cd9f7-547498d659-2qrnc"] Jul 28 01:41:22.185121 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:41:22.185105 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="baea18be-28fe-4026-9c81-dc3046df4a57" containerName="model-chainer-raw-b5420" Jul 28 01:41:22.185121 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:41:22.185117 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="baea18be-28fe-4026-9c81-dc3046df4a57" containerName="model-chainer-raw-b5420" Jul 28 01:41:22.185227 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:41:22.185183 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="baea18be-28fe-4026-9c81-dc3046df4a57" containerName="model-chainer-raw-b5420" Jul 28 01:41:22.189750 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:41:22.189733 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-cd9f7-547498d659-2qrnc" Jul 28 01:41:22.192153 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:41:22.192131 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-raw-hpa-cd9f7-serving-cert\"" Jul 28 01:41:22.192278 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:41:22.192154 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-cl94s\"" Jul 28 01:41:22.192483 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:41:22.192467 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-raw-hpa-cd9f7-kube-rbac-proxy-sar-config\"" Jul 28 01:41:22.193123 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:41:22.193104 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 28 01:41:22.196724 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:41:22.196702 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-cd9f7-547498d659-2qrnc"] Jul 28 01:41:22.219789 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:41:22.219764 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9ed5802c-51d4-4a5b-82aa-eb8399a9746e-proxy-tls\") pod \"model-chainer-raw-hpa-cd9f7-547498d659-2qrnc\" (UID: \"9ed5802c-51d4-4a5b-82aa-eb8399a9746e\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-cd9f7-547498d659-2qrnc" Jul 28 01:41:22.219916 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:41:22.219840 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ed5802c-51d4-4a5b-82aa-eb8399a9746e-openshift-service-ca-bundle\") pod \"model-chainer-raw-hpa-cd9f7-547498d659-2qrnc\" (UID: \"9ed5802c-51d4-4a5b-82aa-eb8399a9746e\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-cd9f7-547498d659-2qrnc" Jul 28 01:41:22.321207 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:41:22.321171 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ed5802c-51d4-4a5b-82aa-eb8399a9746e-openshift-service-ca-bundle\") pod \"model-chainer-raw-hpa-cd9f7-547498d659-2qrnc\" (UID: \"9ed5802c-51d4-4a5b-82aa-eb8399a9746e\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-cd9f7-547498d659-2qrnc" Jul 28 01:41:22.321399 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:41:22.321233 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9ed5802c-51d4-4a5b-82aa-eb8399a9746e-proxy-tls\") pod \"model-chainer-raw-hpa-cd9f7-547498d659-2qrnc\" (UID: \"9ed5802c-51d4-4a5b-82aa-eb8399a9746e\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-cd9f7-547498d659-2qrnc" Jul 28 01:41:22.321399 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:41:22.321357 2578 secret.go:189] Couldn't get secret kserve-ci-e2e-test/model-chainer-raw-hpa-cd9f7-serving-cert: secret "model-chainer-raw-hpa-cd9f7-serving-cert" not found Jul 28 01:41:22.321516 ip-10-0-143-186 kubenswrapper[2578]: E0728 01:41:22.321437 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ed5802c-51d4-4a5b-82aa-eb8399a9746e-proxy-tls podName:9ed5802c-51d4-4a5b-82aa-eb8399a9746e nodeName:}" failed. No retries permitted until 2026-07-28 01:41:22.821413714 +0000 UTC m=+921.375538496 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/9ed5802c-51d4-4a5b-82aa-eb8399a9746e-proxy-tls") pod "model-chainer-raw-hpa-cd9f7-547498d659-2qrnc" (UID: "9ed5802c-51d4-4a5b-82aa-eb8399a9746e") : secret "model-chainer-raw-hpa-cd9f7-serving-cert" not found Jul 28 01:41:22.321817 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:41:22.321798 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ed5802c-51d4-4a5b-82aa-eb8399a9746e-openshift-service-ca-bundle\") pod \"model-chainer-raw-hpa-cd9f7-547498d659-2qrnc\" (UID: \"9ed5802c-51d4-4a5b-82aa-eb8399a9746e\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-cd9f7-547498d659-2qrnc" Jul 28 01:41:22.826776 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:41:22.826743 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9ed5802c-51d4-4a5b-82aa-eb8399a9746e-proxy-tls\") pod \"model-chainer-raw-hpa-cd9f7-547498d659-2qrnc\" (UID: \"9ed5802c-51d4-4a5b-82aa-eb8399a9746e\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-cd9f7-547498d659-2qrnc" Jul 28 01:41:22.829151 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:41:22.829126 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9ed5802c-51d4-4a5b-82aa-eb8399a9746e-proxy-tls\") pod \"model-chainer-raw-hpa-cd9f7-547498d659-2qrnc\" (UID: \"9ed5802c-51d4-4a5b-82aa-eb8399a9746e\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-cd9f7-547498d659-2qrnc" Jul 28 01:41:23.100747 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:41:23.100669 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-cd9f7-547498d659-2qrnc" Jul 28 01:41:23.228829 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:41:23.228803 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-cd9f7-547498d659-2qrnc"] Jul 28 01:41:23.231038 ip-10-0-143-186 kubenswrapper[2578]: W0728 01:41:23.231014 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ed5802c_51d4_4a5b_82aa_eb8399a9746e.slice/crio-d2bcd45e53f931f20a3fd1c5ac233c41cc7daf419f1ad2f80be3a856fb59ac1b WatchSource:0}: Error finding container d2bcd45e53f931f20a3fd1c5ac233c41cc7daf419f1ad2f80be3a856fb59ac1b: Status 404 returned error can't find the container with id d2bcd45e53f931f20a3fd1c5ac233c41cc7daf419f1ad2f80be3a856fb59ac1b Jul 28 01:41:23.352672 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:41:23.352596 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-cd9f7-547498d659-2qrnc" event={"ID":"9ed5802c-51d4-4a5b-82aa-eb8399a9746e","Type":"ContainerStarted","Data":"334c234f878d36d7dc9470ec32d8a3017b33466fa13d23adbcdc3fa88e1afcc9"} Jul 28 01:41:23.352672 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:41:23.352647 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-cd9f7-547498d659-2qrnc" event={"ID":"9ed5802c-51d4-4a5b-82aa-eb8399a9746e","Type":"ContainerStarted","Data":"d2bcd45e53f931f20a3fd1c5ac233c41cc7daf419f1ad2f80be3a856fb59ac1b"} Jul 28 01:41:23.352847 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:41:23.352679 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-cd9f7-547498d659-2qrnc" Jul 28 01:41:23.369051 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:41:23.369006 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-cd9f7-547498d659-2qrnc" podStartSLOduration=1.368992505 podStartE2EDuration="1.368992505s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:41:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:41:23.368208181 +0000 UTC m=+921.922332980" watchObservedRunningTime="2026-07-28 01:41:23.368992505 +0000 UTC m=+921.923117332" Jul 28 01:41:29.364245 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:41:29.364208 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-cd9f7-547498d659-2qrnc" Jul 28 01:41:32.236829 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:41:32.236796 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-cd9f7-547498d659-2qrnc"] Jul 28 01:41:32.237254 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:41:32.236990 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-cd9f7-547498d659-2qrnc" podUID="9ed5802c-51d4-4a5b-82aa-eb8399a9746e" containerName="model-chainer-raw-hpa-cd9f7" containerID="cri-o://334c234f878d36d7dc9470ec32d8a3017b33466fa13d23adbcdc3fa88e1afcc9" gracePeriod=30 Jul 28 01:41:34.362583 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:41:34.362529 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-cd9f7-547498d659-2qrnc" podUID="9ed5802c-51d4-4a5b-82aa-eb8399a9746e" containerName="model-chainer-raw-hpa-cd9f7" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 01:41:39.362388 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:41:39.362351 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-cd9f7-547498d659-2qrnc" podUID="9ed5802c-51d4-4a5b-82aa-eb8399a9746e" containerName="model-chainer-raw-hpa-cd9f7" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 01:41:44.361959 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:41:44.361923 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-cd9f7-547498d659-2qrnc" podUID="9ed5802c-51d4-4a5b-82aa-eb8399a9746e" containerName="model-chainer-raw-hpa-cd9f7" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 01:41:44.364334 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:41:44.362027 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-cd9f7-547498d659-2qrnc" Jul 28 01:41:49.362304 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:41:49.362268 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-cd9f7-547498d659-2qrnc" podUID="9ed5802c-51d4-4a5b-82aa-eb8399a9746e" containerName="model-chainer-raw-hpa-cd9f7" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 01:41:54.362601 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:41:54.362534 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-cd9f7-547498d659-2qrnc" podUID="9ed5802c-51d4-4a5b-82aa-eb8399a9746e" containerName="model-chainer-raw-hpa-cd9f7" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 01:41:59.362461 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:41:59.362424 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-cd9f7-547498d659-2qrnc" podUID="9ed5802c-51d4-4a5b-82aa-eb8399a9746e" containerName="model-chainer-raw-hpa-cd9f7" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 01:42:02.538032 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:42:02.537998 2578 generic.go:356] "Generic (PLEG): container finished" podID="9ed5802c-51d4-4a5b-82aa-eb8399a9746e" containerID="334c234f878d36d7dc9470ec32d8a3017b33466fa13d23adbcdc3fa88e1afcc9" exitCode=0 Jul 28 01:42:02.538411 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:42:02.538072 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-cd9f7-547498d659-2qrnc" event={"ID":"9ed5802c-51d4-4a5b-82aa-eb8399a9746e","Type":"ContainerDied","Data":"334c234f878d36d7dc9470ec32d8a3017b33466fa13d23adbcdc3fa88e1afcc9"} Jul 28 01:42:02.888774 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:42:02.888747 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-cd9f7-547498d659-2qrnc" Jul 28 01:42:03.059131 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:42:03.059040 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ed5802c-51d4-4a5b-82aa-eb8399a9746e-openshift-service-ca-bundle\") pod \"9ed5802c-51d4-4a5b-82aa-eb8399a9746e\" (UID: \"9ed5802c-51d4-4a5b-82aa-eb8399a9746e\") " Jul 28 01:42:03.059131 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:42:03.059119 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9ed5802c-51d4-4a5b-82aa-eb8399a9746e-proxy-tls\") pod \"9ed5802c-51d4-4a5b-82aa-eb8399a9746e\" (UID: \"9ed5802c-51d4-4a5b-82aa-eb8399a9746e\") " Jul 28 01:42:03.059460 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:42:03.059426 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ed5802c-51d4-4a5b-82aa-eb8399a9746e-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "9ed5802c-51d4-4a5b-82aa-eb8399a9746e" (UID: "9ed5802c-51d4-4a5b-82aa-eb8399a9746e"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 01:42:03.061124 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:42:03.061103 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ed5802c-51d4-4a5b-82aa-eb8399a9746e-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "9ed5802c-51d4-4a5b-82aa-eb8399a9746e" (UID: "9ed5802c-51d4-4a5b-82aa-eb8399a9746e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 01:42:03.160132 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:42:03.160098 2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ed5802c-51d4-4a5b-82aa-eb8399a9746e-openshift-service-ca-bundle\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:42:03.160132 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:42:03.160122 2578 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9ed5802c-51d4-4a5b-82aa-eb8399a9746e-proxy-tls\") on node \"ip-10-0-143-186.ec2.internal\" DevicePath \"\"" Jul 28 01:42:03.543861 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:42:03.543833 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-cd9f7-547498d659-2qrnc" Jul 28 01:42:03.544271 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:42:03.543862 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-cd9f7-547498d659-2qrnc" event={"ID":"9ed5802c-51d4-4a5b-82aa-eb8399a9746e","Type":"ContainerDied","Data":"d2bcd45e53f931f20a3fd1c5ac233c41cc7daf419f1ad2f80be3a856fb59ac1b"} Jul 28 01:42:03.544271 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:42:03.543920 2578 scope.go:117] "RemoveContainer" containerID="334c234f878d36d7dc9470ec32d8a3017b33466fa13d23adbcdc3fa88e1afcc9" Jul 28 01:42:03.563884 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:42:03.563857 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-cd9f7-547498d659-2qrnc"] Jul 28 01:42:03.567666 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:42:03.567646 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-cd9f7-547498d659-2qrnc"] Jul 28 01:42:04.010667 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:42:04.010636 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ed5802c-51d4-4a5b-82aa-eb8399a9746e" path="/var/lib/kubelet/pods/9ed5802c-51d4-4a5b-82aa-eb8399a9746e/volumes" Jul 28 01:46:01.901746 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:46:01.901674 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-k2wlm_3503116a-1398-42ea-ac86-743f5f51e989/console-operator/1.log" Jul 28 01:46:01.902298 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:46:01.902217 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-k2wlm_3503116a-1398-42ea-ac86-743f5f51e989/console-operator/1.log" Jul 28 01:51:01.931067 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:51:01.931037 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-k2wlm_3503116a-1398-42ea-ac86-743f5f51e989/console-operator/1.log" Jul 28 01:51:01.932337 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:51:01.932311 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-k2wlm_3503116a-1398-42ea-ac86-743f5f51e989/console-operator/1.log" Jul 28 01:56:01.960039 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:56:01.960012 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-k2wlm_3503116a-1398-42ea-ac86-743f5f51e989/console-operator/1.log" Jul 28 01:56:01.963594 ip-10-0-143-186 kubenswrapper[2578]: I0728 01:56:01.963576 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-k2wlm_3503116a-1398-42ea-ac86-743f5f51e989/console-operator/1.log" Jul 28 02:01:01.993577 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:01.993530 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-k2wlm_3503116a-1398-42ea-ac86-743f5f51e989/console-operator/1.log" Jul 28 02:01:01.997635 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:01.997603 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-k2wlm_3503116a-1398-42ea-ac86-743f5f51e989/console-operator/1.log" Jul 28 02:01:22.259917 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:22.259872 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-7gfx9_1bf2b1e4-0123-4a0b-9a30-b2ea3794732c/global-pull-secret-syncer/0.log" Jul 28 02:01:22.429215 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:22.429190 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-hnpn2_751f8878-a0ed-4756-93c1-97a66b2afd10/konnectivity-agent/0.log" Jul 28 02:01:22.525691 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:22.525607 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-186.ec2.internal_9fe4fb4d5152991d383fb6f3c8471009/haproxy/0.log" Jul 28 02:01:26.477934 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:26.477904 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d9edc30d-4554-40ed-96a0-6bbeb0bf82b4/alertmanager/0.log" Jul 28 02:01:26.517688 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:26.517661 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d9edc30d-4554-40ed-96a0-6bbeb0bf82b4/config-reloader/0.log" Jul 28 02:01:26.559661 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:26.559625 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d9edc30d-4554-40ed-96a0-6bbeb0bf82b4/kube-rbac-proxy-web/0.log" Jul 28 02:01:26.598213 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:26.598186 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d9edc30d-4554-40ed-96a0-6bbeb0bf82b4/kube-rbac-proxy/0.log" Jul 28 02:01:26.626337 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:26.626316 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d9edc30d-4554-40ed-96a0-6bbeb0bf82b4/kube-rbac-proxy-metric/0.log" Jul 28 02:01:26.653083 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:26.653060 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d9edc30d-4554-40ed-96a0-6bbeb0bf82b4/prom-label-proxy/0.log" Jul 28 02:01:26.679868 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:26.679846 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d9edc30d-4554-40ed-96a0-6bbeb0bf82b4/init-config-reloader/0.log" Jul 28 02:01:26.727250 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:26.727219 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-f297n_56e7bae5-9b23-4f6b-8084-20287730dd30/cluster-monitoring-operator/0.log" Jul 28 02:01:26.879691 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:26.879664 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-cr46m_339cd32a-6e07-4d34-a4ee-e72122365e49/monitoring-plugin/0.log" Jul 28 02:01:27.106569 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:27.106519 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t7gkq_38c8a5bd-f01b-407f-9651-8d8665b13672/node-exporter/0.log" Jul 28 02:01:27.135427 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:27.135335 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t7gkq_38c8a5bd-f01b-407f-9651-8d8665b13672/kube-rbac-proxy/0.log" Jul 28 02:01:27.212156 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:27.212122 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t7gkq_38c8a5bd-f01b-407f-9651-8d8665b13672/init-textfile/0.log" Jul 28 02:01:27.268492 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:27.268459 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-vrvns_ab39056b-6a5b-41cd-9ccb-cccd5662180a/kube-rbac-proxy-main/0.log" Jul 28 02:01:27.297099 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:27.297062 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-vrvns_ab39056b-6a5b-41cd-9ccb-cccd5662180a/kube-rbac-proxy-self/0.log" Jul 28 02:01:27.321929 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:27.321906 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-vrvns_ab39056b-6a5b-41cd-9ccb-cccd5662180a/openshift-state-metrics/0.log" Jul 28 02:01:27.842322 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:27.842280 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84c8d95df8-22666_6335a2b9-cce3-440c-a57b-cfc264e024de/thanos-query/0.log" Jul 28 02:01:27.874763 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:27.874722 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84c8d95df8-22666_6335a2b9-cce3-440c-a57b-cfc264e024de/kube-rbac-proxy-web/0.log" Jul 28 02:01:27.907998 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:27.907966 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84c8d95df8-22666_6335a2b9-cce3-440c-a57b-cfc264e024de/kube-rbac-proxy/0.log" Jul 28 02:01:27.947502 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:27.947468 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84c8d95df8-22666_6335a2b9-cce3-440c-a57b-cfc264e024de/prom-label-proxy/0.log" Jul 28 02:01:27.990505 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:27.990481 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84c8d95df8-22666_6335a2b9-cce3-440c-a57b-cfc264e024de/kube-rbac-proxy-rules/0.log" Jul 28 02:01:28.025530 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:28.025501 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84c8d95df8-22666_6335a2b9-cce3-440c-a57b-cfc264e024de/kube-rbac-proxy-metrics/0.log" Jul 28 02:01:29.254826 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:29.254798 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-4pl5d_1f2321ec-7f9d-4e33-9bed-d48fd717ca07/networking-console-plugin/0.log" Jul 28 02:01:29.830501 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:29.830406 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-k2wlm_3503116a-1398-42ea-ac86-743f5f51e989/console-operator/1.log" Jul 28 02:01:29.835095 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:29.835076 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-k2wlm_3503116a-1398-42ea-ac86-743f5f51e989/console-operator/2.log" Jul 28 02:01:30.229990 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:30.229916 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-798f5678bc-wm5rv_1f661790-6ef9-4a48-9ea1-40e2e42a76d2/console/0.log" Jul 28 02:01:30.686337 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:30.686299 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4jnbh/perf-node-gather-daemonset-s8598"] Jul 28 02:01:30.686740 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:30.686685 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ed5802c-51d4-4a5b-82aa-eb8399a9746e" containerName="model-chainer-raw-hpa-cd9f7" Jul 28 02:01:30.686740 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:30.686696 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ed5802c-51d4-4a5b-82aa-eb8399a9746e" containerName="model-chainer-raw-hpa-cd9f7" Jul 28 02:01:30.686810 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:30.686773 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ed5802c-51d4-4a5b-82aa-eb8399a9746e" containerName="model-chainer-raw-hpa-cd9f7" Jul 28 02:01:30.690078 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:30.690058 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4jnbh/perf-node-gather-daemonset-s8598" Jul 28 02:01:30.692592 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:30.692575 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4jnbh\"/\"default-dockercfg-5ljcm\"" Jul 28 02:01:30.692963 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:30.692947 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4jnbh\"/\"openshift-service-ca.crt\"" Jul 28 02:01:30.693584 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:30.693571 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4jnbh\"/\"kube-root-ca.crt\"" Jul 28 02:01:30.700031 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:30.700010 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4jnbh/perf-node-gather-daemonset-s8598"] Jul 28 02:01:30.791720 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:30.791681 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-8fbhx_dbab9a89-f5fa-4a89-ba0a-d0ab2b4b8957/volume-data-source-validator/0.log" Jul 28 02:01:30.843942 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:30.843914 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3810e2e1-caae-4050-a0a5-8d5dc1473ebd-podres\") pod \"perf-node-gather-daemonset-s8598\" (UID: \"3810e2e1-caae-4050-a0a5-8d5dc1473ebd\") " pod="openshift-must-gather-4jnbh/perf-node-gather-daemonset-s8598" Jul 28 02:01:30.843942 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:30.843948 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3810e2e1-caae-4050-a0a5-8d5dc1473ebd-proc\") pod \"perf-node-gather-daemonset-s8598\" (UID: \"3810e2e1-caae-4050-a0a5-8d5dc1473ebd\") " pod="openshift-must-gather-4jnbh/perf-node-gather-daemonset-s8598" Jul 28 02:01:30.844143 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:30.843965 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3810e2e1-caae-4050-a0a5-8d5dc1473ebd-lib-modules\") pod \"perf-node-gather-daemonset-s8598\" (UID: \"3810e2e1-caae-4050-a0a5-8d5dc1473ebd\") " pod="openshift-must-gather-4jnbh/perf-node-gather-daemonset-s8598" Jul 28 02:01:30.844143 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:30.844081 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3810e2e1-caae-4050-a0a5-8d5dc1473ebd-sys\") pod \"perf-node-gather-daemonset-s8598\" (UID: \"3810e2e1-caae-4050-a0a5-8d5dc1473ebd\") " pod="openshift-must-gather-4jnbh/perf-node-gather-daemonset-s8598" Jul 28 02:01:30.844143 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:30.844111 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6h8p\" (UniqueName: \"kubernetes.io/projected/3810e2e1-caae-4050-a0a5-8d5dc1473ebd-kube-api-access-f6h8p\") pod \"perf-node-gather-daemonset-s8598\" (UID: \"3810e2e1-caae-4050-a0a5-8d5dc1473ebd\") " pod="openshift-must-gather-4jnbh/perf-node-gather-daemonset-s8598" Jul 28 02:01:30.945607 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:30.945493 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3810e2e1-caae-4050-a0a5-8d5dc1473ebd-podres\") pod \"perf-node-gather-daemonset-s8598\" (UID: \"3810e2e1-caae-4050-a0a5-8d5dc1473ebd\") " pod="openshift-must-gather-4jnbh/perf-node-gather-daemonset-s8598" Jul 28 02:01:30.945607 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:30.945571 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3810e2e1-caae-4050-a0a5-8d5dc1473ebd-proc\") pod \"perf-node-gather-daemonset-s8598\" (UID: \"3810e2e1-caae-4050-a0a5-8d5dc1473ebd\") " pod="openshift-must-gather-4jnbh/perf-node-gather-daemonset-s8598" Jul 28 02:01:30.945607 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:30.945597 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3810e2e1-caae-4050-a0a5-8d5dc1473ebd-lib-modules\") pod \"perf-node-gather-daemonset-s8598\" (UID: \"3810e2e1-caae-4050-a0a5-8d5dc1473ebd\") " pod="openshift-must-gather-4jnbh/perf-node-gather-daemonset-s8598" Jul 28 02:01:30.945874 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:30.945680 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3810e2e1-caae-4050-a0a5-8d5dc1473ebd-podres\") pod \"perf-node-gather-daemonset-s8598\" (UID: \"3810e2e1-caae-4050-a0a5-8d5dc1473ebd\") " pod="openshift-must-gather-4jnbh/perf-node-gather-daemonset-s8598" Jul 28 02:01:30.945874 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:30.945681 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3810e2e1-caae-4050-a0a5-8d5dc1473ebd-sys\") pod \"perf-node-gather-daemonset-s8598\" (UID: \"3810e2e1-caae-4050-a0a5-8d5dc1473ebd\") " pod="openshift-must-gather-4jnbh/perf-node-gather-daemonset-s8598" Jul 28 02:01:30.945874 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:30.945719 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6h8p\" (UniqueName: \"kubernetes.io/projected/3810e2e1-caae-4050-a0a5-8d5dc1473ebd-kube-api-access-f6h8p\") pod \"perf-node-gather-daemonset-s8598\" (UID: \"3810e2e1-caae-4050-a0a5-8d5dc1473ebd\") " pod="openshift-must-gather-4jnbh/perf-node-gather-daemonset-s8598" Jul 28 02:01:30.945874 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:30.945687 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3810e2e1-caae-4050-a0a5-8d5dc1473ebd-proc\") pod \"perf-node-gather-daemonset-s8598\" (UID: \"3810e2e1-caae-4050-a0a5-8d5dc1473ebd\") " pod="openshift-must-gather-4jnbh/perf-node-gather-daemonset-s8598" Jul 28 02:01:30.945874 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:30.945767 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3810e2e1-caae-4050-a0a5-8d5dc1473ebd-lib-modules\") pod \"perf-node-gather-daemonset-s8598\" (UID: \"3810e2e1-caae-4050-a0a5-8d5dc1473ebd\") " pod="openshift-must-gather-4jnbh/perf-node-gather-daemonset-s8598" Jul 28 02:01:30.945874 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:30.945725 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3810e2e1-caae-4050-a0a5-8d5dc1473ebd-sys\") pod \"perf-node-gather-daemonset-s8598\" (UID: \"3810e2e1-caae-4050-a0a5-8d5dc1473ebd\") " pod="openshift-must-gather-4jnbh/perf-node-gather-daemonset-s8598" Jul 28 02:01:30.953599 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:30.953561 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6h8p\" (UniqueName: \"kubernetes.io/projected/3810e2e1-caae-4050-a0a5-8d5dc1473ebd-kube-api-access-f6h8p\") pod \"perf-node-gather-daemonset-s8598\" (UID: \"3810e2e1-caae-4050-a0a5-8d5dc1473ebd\") " pod="openshift-must-gather-4jnbh/perf-node-gather-daemonset-s8598" Jul 28 02:01:31.000342 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:31.000309 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4jnbh/perf-node-gather-daemonset-s8598" Jul 28 02:01:31.127227 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:31.127086 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4jnbh/perf-node-gather-daemonset-s8598"] Jul 28 02:01:31.129695 ip-10-0-143-186 kubenswrapper[2578]: W0728 02:01:31.129653 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3810e2e1_caae_4050_a0a5_8d5dc1473ebd.slice/crio-51da27654452e5d79856a05598a63ca7bbd20fa6fe78e87f232471d5b1053204 WatchSource:0}: Error finding container 51da27654452e5d79856a05598a63ca7bbd20fa6fe78e87f232471d5b1053204: Status 404 returned error can't find the container with id 51da27654452e5d79856a05598a63ca7bbd20fa6fe78e87f232471d5b1053204 Jul 28 02:01:31.800135 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:31.800104 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-n5dq2_5873b663-f99f-453e-8aaf-d57bf3fd0105/dns/0.log" Jul 28 02:01:31.822690 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:31.822665 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-n5dq2_5873b663-f99f-453e-8aaf-d57bf3fd0105/kube-rbac-proxy/0.log" Jul 28 02:01:31.855333 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:31.855306 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2vjkb_516e3ead-ffab-4201-9715-2684dcfc7aa6/dns-node-resolver/0.log" Jul 28 02:01:31.943016 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:31.942978 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4jnbh/perf-node-gather-daemonset-s8598" event={"ID":"3810e2e1-caae-4050-a0a5-8d5dc1473ebd","Type":"ContainerStarted","Data":"5daf2afc6b50caf3f653ff683a89863f2b64b5e533c0606a246a83accc14a163"} Jul 28 02:01:31.943219 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:31.943061 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4jnbh/perf-node-gather-daemonset-s8598" event={"ID":"3810e2e1-caae-4050-a0a5-8d5dc1473ebd","Type":"ContainerStarted","Data":"51da27654452e5d79856a05598a63ca7bbd20fa6fe78e87f232471d5b1053204"} Jul 28 02:01:31.943219 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:31.943104 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4jnbh/perf-node-gather-daemonset-s8598" Jul 28 02:01:31.961160 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:31.961112 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4jnbh/perf-node-gather-daemonset-s8598" podStartSLOduration=1.9610955730000001 podStartE2EDuration="1.961095573s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 02:01:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 02:01:31.960079289 +0000 UTC m=+2130.514204088" watchObservedRunningTime="2026-07-28 02:01:31.961095573 +0000 UTC m=+2130.515220370" Jul 28 02:01:32.418194 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:32.418161 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bgvmj_e1808965-dc33-46a8-a355-b3ebbe849c65/node-ca/0.log" Jul 28 02:01:33.196099 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:33.196070 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-74577ddb85-mm25q_923f7f4a-e4e7-42a0-a7ef-85f35c3c2a71/router/0.log" Jul 28 02:01:33.564584 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:33.564528 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-sc92r_3b34c5ff-3aed-4bba-990e-878c2de528ed/serve-healthcheck-canary/0.log" Jul 28 02:01:33.894409 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:33.894330 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-q8ldl_18e29850-170d-4a22-b890-95237b45246d/insights-operator/0.log" Jul 28 02:01:33.895266 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:33.895251 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-q8ldl_18e29850-170d-4a22-b890-95237b45246d/insights-operator/1.log" Jul 28 02:01:33.980942 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:33.980914 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-68grr_0888b376-095e-4e9c-9671-92986acaf8d8/kube-rbac-proxy/0.log" Jul 28 02:01:34.000636 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:34.000575 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-68grr_0888b376-095e-4e9c-9671-92986acaf8d8/exporter/0.log" Jul 28 02:01:34.022344 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:34.022317 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-68grr_0888b376-095e-4e9c-9671-92986acaf8d8/extractor/0.log" Jul 28 02:01:36.357761 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:36.357732 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_model-serving-api-7b954ffc95-n97lj_7ae618a2-5cd5-40e0-a75e-8158448063dc/server/0.log" Jul 28 02:01:36.464934 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:36.464904 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_odh-model-controller-75d9cdbddf-zfd9w_8ae9bd87-7366-4663-bb27-883eca661d42/manager/0.log" Jul 28 02:01:36.545536 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:36.545508 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-64568bcd49-5jv79_4da4e640-b010-44d0-b78d-e6eb94c03309/seaweedfs/0.log" Jul 28 02:01:37.958680 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:37.958645 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4jnbh/perf-node-gather-daemonset-s8598" Jul 28 02:01:40.980778 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:40.980733 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-m7p9h_b4e84366-546a-4004-b614-6b3c35c61dfe/kube-storage-version-migrator-operator/1.log" Jul 28 02:01:40.981785 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:40.981749 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-m7p9h_b4e84366-546a-4004-b614-6b3c35c61dfe/kube-storage-version-migrator-operator/0.log" Jul 28 02:01:42.308460 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:42.308432 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wgdbs_3394f81c-6c57-451b-8d8e-2b6504f34ed3/kube-multus-additional-cni-plugins/0.log" Jul 28 02:01:42.330407 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:42.330382 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wgdbs_3394f81c-6c57-451b-8d8e-2b6504f34ed3/egress-router-binary-copy/0.log" Jul 28 02:01:42.353842 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:42.353813 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wgdbs_3394f81c-6c57-451b-8d8e-2b6504f34ed3/cni-plugins/0.log" Jul 28 02:01:42.377117 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:42.377084 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wgdbs_3394f81c-6c57-451b-8d8e-2b6504f34ed3/bond-cni-plugin/0.log" Jul 28 02:01:42.398227 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:42.398159 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wgdbs_3394f81c-6c57-451b-8d8e-2b6504f34ed3/routeoverride-cni/0.log" Jul 28 02:01:42.420380 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:42.420361 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wgdbs_3394f81c-6c57-451b-8d8e-2b6504f34ed3/whereabouts-cni-bincopy/0.log" Jul 28 02:01:42.441727 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:42.441701 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wgdbs_3394f81c-6c57-451b-8d8e-2b6504f34ed3/whereabouts-cni/0.log" Jul 28 02:01:42.502260 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:42.502230 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-x598j_7af7909b-e421-437d-b0a8-895ad38876c7/kube-multus/0.log" Jul 28 02:01:42.575190 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:42.575160 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-brftp_005aafcd-65c4-45cb-aa21-484a0c94167e/network-metrics-daemon/0.log" Jul 28 02:01:42.596152 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:42.596125 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-brftp_005aafcd-65c4-45cb-aa21-484a0c94167e/kube-rbac-proxy/0.log" Jul 28 02:01:43.388870 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:43.388842 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6m66_3a77a2d8-54f1-4eae-b356-213cd00c7462/ovn-controller/0.log" Jul 28 02:01:43.407864 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:43.407839 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6m66_3a77a2d8-54f1-4eae-b356-213cd00c7462/ovn-acl-logging/0.log" Jul 28 02:01:43.430500 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:43.430474 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6m66_3a77a2d8-54f1-4eae-b356-213cd00c7462/kube-rbac-proxy-node/0.log" Jul 28 02:01:43.453805 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:43.453782 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6m66_3a77a2d8-54f1-4eae-b356-213cd00c7462/kube-rbac-proxy-ovn-metrics/0.log" Jul 28 02:01:43.476057 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:43.476028 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6m66_3a77a2d8-54f1-4eae-b356-213cd00c7462/northd/0.log" Jul 28 02:01:43.502867 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:43.502844 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6m66_3a77a2d8-54f1-4eae-b356-213cd00c7462/nbdb/0.log" Jul 28 02:01:43.523905 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:43.523879 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6m66_3a77a2d8-54f1-4eae-b356-213cd00c7462/sbdb/0.log" Jul 28 02:01:43.568470 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:43.568436 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6m66_3a77a2d8-54f1-4eae-b356-213cd00c7462/ovnkube-controller/0.log" Jul 28 02:01:44.992534 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:44.992507 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-7d88cb55d8-hrdd9_b91abb25-2132-46b6-b0b5-7f00d007fac3/check-endpoints/0.log" Jul 28 02:01:45.014708 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:45.014680 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-2znt8_5060905d-b8ac-4bbd-94cb-b1a8a8952a9a/network-check-target-container/0.log" Jul 28 02:01:45.938345 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:45.938319 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-qs68r_3f0ab34c-4530-44b3-9bea-e54cdbb50377/iptables-alerter/0.log" Jul 28 02:01:46.671397 ip-10-0-143-186 kubenswrapper[2578]: I0728 02:01:46.671350 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-kmk8x_03b14c3f-225a-4d46-9f03-31f3c8da2916/tuned/0.log"