Jul 09 11:11:33.302691 ip-10-0-143-142 systemd[1]: Starting Kubernetes Kubelet... Jul 09 11:11:35.077015 ip-10-0-143-142 kubenswrapper[2550]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 11:11:35.077015 ip-10-0-143-142 kubenswrapper[2550]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 09 11:11:35.077015 ip-10-0-143-142 kubenswrapper[2550]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 11:11:35.077015 ip-10-0-143-142 kubenswrapper[2550]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 09 11:11:35.077015 ip-10-0-143-142 kubenswrapper[2550]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 11:11:35.078589 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.078473 2550 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 09 11:11:35.083213 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083197 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 11:11:35.083213 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083213 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 11:11:35.083291 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083217 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 11:11:35.083291 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083220 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 11:11:35.083291 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083223 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 11:11:35.083291 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083226 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 11:11:35.083291 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083229 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 11:11:35.083291 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083232 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 11:11:35.083291 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083235 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 11:11:35.083291 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083238 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 11:11:35.083291 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083240 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 11:11:35.083291 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083243 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 11:11:35.083291 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083245 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 11:11:35.083291 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083249 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 11:11:35.083291 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083251 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 11:11:35.083291 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083259 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 11:11:35.083291 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083262 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 11:11:35.083291 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083265 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 11:11:35.083291 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083267 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 11:11:35.083291 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083270 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 11:11:35.083291 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083272 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 11:11:35.083755 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083277 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 11:11:35.083755 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083281 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 11:11:35.083755 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083283 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 11:11:35.083755 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083286 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 11:11:35.083755 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083290 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 11:11:35.083755 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083293 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 11:11:35.083755 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083296 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 11:11:35.083755 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083299 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 11:11:35.083755 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083303 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 11:11:35.083755 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083308 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 11:11:35.083755 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083311 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 11:11:35.083755 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083314 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 11:11:35.083755 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083317 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 11:11:35.083755 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083319 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 11:11:35.083755 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083322 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 11:11:35.083755 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083325 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 11:11:35.083755 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083342 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 11:11:35.083755 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083345 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 11:11:35.083755 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083348 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 11:11:35.084210 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083357 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 11:11:35.084210 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083360 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 11:11:35.084210 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083363 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 11:11:35.084210 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083367 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 11:11:35.084210 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083370 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 11:11:35.084210 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083372 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 11:11:35.084210 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083375 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 11:11:35.084210 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083377 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 11:11:35.084210 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083380 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 11:11:35.084210 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083382 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 11:11:35.084210 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083385 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 11:11:35.084210 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083387 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 11:11:35.084210 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083390 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 11:11:35.084210 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083392 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 11:11:35.084210 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083396 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 11:11:35.084210 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083398 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 11:11:35.084210 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083401 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 11:11:35.084210 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083404 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 11:11:35.084210 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083407 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 11:11:35.084210 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083410 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 11:11:35.084730 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083412 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 11:11:35.084730 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083415 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 11:11:35.084730 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083417 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 11:11:35.084730 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083420 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 11:11:35.084730 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083423 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 11:11:35.084730 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083425 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 11:11:35.084730 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083428 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 11:11:35.084730 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083430 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 11:11:35.084730 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083433 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 11:11:35.084730 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083435 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 11:11:35.084730 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083437 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 11:11:35.084730 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083440 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 11:11:35.084730 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083442 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 11:11:35.084730 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083445 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 11:11:35.084730 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083448 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 11:11:35.084730 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083450 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 11:11:35.084730 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083452 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 11:11:35.084730 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083456 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 11:11:35.084730 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083459 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 11:11:35.085165 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083461 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 11:11:35.085165 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083464 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 11:11:35.085165 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083466 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 11:11:35.085165 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083469 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 11:11:35.085165 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083471 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 11:11:35.085165 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083474 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 11:11:35.085165 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083476 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 11:11:35.085165 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083481 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 11:11:35.085165 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083484 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 11:11:35.085165 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083486 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 11:11:35.085165 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083489 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 11:11:35.085165 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083491 2550 feature_gate.go:328] unrecognized feature gate: Example Jul 09 11:11:35.085165 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083494 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 11:11:35.085165 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083497 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 11:11:35.085165 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083499 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 11:11:35.085165 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083502 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 11:11:35.085165 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083504 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 11:11:35.085165 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083507 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 11:11:35.085165 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083509 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 11:11:35.085165 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.083512 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 11:11:35.085675 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084217 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 11:11:35.085675 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084223 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 11:11:35.085675 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084226 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 11:11:35.085675 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084229 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 11:11:35.085675 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084232 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 11:11:35.085675 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084235 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 11:11:35.085675 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084238 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 11:11:35.085675 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084241 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 11:11:35.085675 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084244 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 11:11:35.085675 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084246 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 11:11:35.085675 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084249 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 11:11:35.085675 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084251 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 11:11:35.085675 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084254 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 11:11:35.085675 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084257 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 11:11:35.085675 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084259 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 11:11:35.085675 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084262 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 11:11:35.085675 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084265 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 11:11:35.085675 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084268 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 11:11:35.085675 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084270 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 11:11:35.085675 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084273 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 11:11:35.086170 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084276 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 11:11:35.086170 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084278 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 11:11:35.086170 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084281 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 11:11:35.086170 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084286 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 11:11:35.086170 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084290 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 11:11:35.086170 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084293 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 11:11:35.086170 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084295 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 11:11:35.086170 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084300 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 11:11:35.086170 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084303 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 11:11:35.086170 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084306 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 11:11:35.086170 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084309 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 11:11:35.086170 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084313 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 11:11:35.086170 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084316 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 11:11:35.086170 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084318 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 11:11:35.086170 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084321 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 11:11:35.086170 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084323 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 11:11:35.086170 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084341 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 11:11:35.086170 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084343 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 11:11:35.086170 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084346 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 11:11:35.086635 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084348 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 11:11:35.086635 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084352 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 11:11:35.086635 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084354 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 11:11:35.086635 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084357 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 11:11:35.086635 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084360 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 11:11:35.086635 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084362 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 11:11:35.086635 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084365 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 11:11:35.086635 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084367 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 11:11:35.086635 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084370 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 11:11:35.086635 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084373 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 11:11:35.086635 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084375 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 11:11:35.086635 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084378 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 11:11:35.086635 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084381 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 11:11:35.086635 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084384 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 11:11:35.086635 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084386 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 11:11:35.086635 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084388 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 11:11:35.086635 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084391 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 11:11:35.086635 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084393 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 11:11:35.086635 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084397 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 11:11:35.087084 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084400 2550 feature_gate.go:328] unrecognized feature gate: Example Jul 09 11:11:35.087084 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084404 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 11:11:35.087084 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084407 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 11:11:35.087084 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084410 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 11:11:35.087084 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084412 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 11:11:35.087084 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084415 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 11:11:35.087084 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084417 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 11:11:35.087084 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084420 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 11:11:35.087084 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084423 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 11:11:35.087084 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084425 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 11:11:35.087084 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084428 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 11:11:35.087084 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084431 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 11:11:35.087084 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084433 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 11:11:35.087084 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084436 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 11:11:35.087084 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084438 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 11:11:35.087084 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084441 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 11:11:35.087084 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084444 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 11:11:35.087084 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084446 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 11:11:35.087084 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084449 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 11:11:35.087084 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084452 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 11:11:35.087596 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084454 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 11:11:35.087596 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084456 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 11:11:35.087596 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084459 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 11:11:35.087596 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084462 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 11:11:35.087596 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084464 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 11:11:35.087596 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084467 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 11:11:35.087596 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084469 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 11:11:35.087596 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084472 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 11:11:35.087596 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084474 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 11:11:35.087596 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084477 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 11:11:35.087596 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084479 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 11:11:35.087596 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084482 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 11:11:35.087596 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084485 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 11:11:35.087596 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084489 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 11:11:35.087596 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084492 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 11:11:35.087596 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084495 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 11:11:35.087596 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084497 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 11:11:35.087596 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084500 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 11:11:35.087596 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084502 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 11:11:35.087596 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084504 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 11:11:35.088070 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084507 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 11:11:35.088070 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084877 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 11:11:35.088070 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084883 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 11:11:35.088070 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084886 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 11:11:35.088070 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084889 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 11:11:35.088070 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084892 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 11:11:35.088070 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084895 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 11:11:35.088070 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084898 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 11:11:35.088070 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084900 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 11:11:35.088070 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084903 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 11:11:35.088070 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084906 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 11:11:35.088070 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084908 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 11:11:35.088070 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084911 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 11:11:35.088070 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084913 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 11:11:35.088070 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084916 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 11:11:35.088070 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084919 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 11:11:35.088070 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084923 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 11:11:35.088070 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084925 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 11:11:35.088070 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084928 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 11:11:35.088070 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084930 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 11:11:35.088566 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084933 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 11:11:35.088566 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084935 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 11:11:35.088566 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084938 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 11:11:35.088566 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084940 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 11:11:35.088566 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084943 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 11:11:35.088566 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084947 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 11:11:35.088566 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084949 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 11:11:35.088566 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084952 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 11:11:35.088566 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084954 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 11:11:35.088566 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084957 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 11:11:35.088566 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084959 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 11:11:35.088566 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084962 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 11:11:35.088566 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084965 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 11:11:35.088566 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084967 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 11:11:35.088566 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084971 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 11:11:35.088566 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084974 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 11:11:35.088566 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084977 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 11:11:35.088566 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084979 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 11:11:35.088566 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084982 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 11:11:35.089054 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084985 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 11:11:35.089054 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084988 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 11:11:35.089054 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084990 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 11:11:35.089054 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084993 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 11:11:35.089054 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084995 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 11:11:35.089054 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.084998 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 11:11:35.089054 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085000 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 11:11:35.089054 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085002 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 11:11:35.089054 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085005 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 11:11:35.089054 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085007 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 11:11:35.089054 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085010 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 11:11:35.089054 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085012 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 11:11:35.089054 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085014 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 11:11:35.089054 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085017 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 11:11:35.089054 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085019 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 11:11:35.089054 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085022 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 11:11:35.089054 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085025 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 11:11:35.089054 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085029 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 11:11:35.089054 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085033 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 11:11:35.089530 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085036 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 11:11:35.089530 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085039 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 11:11:35.089530 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085042 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 11:11:35.089530 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085044 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 11:11:35.089530 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085047 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 11:11:35.089530 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085049 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 11:11:35.089530 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085052 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 11:11:35.089530 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085054 2550 feature_gate.go:328] unrecognized feature gate: Example Jul 09 11:11:35.089530 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085059 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 11:11:35.089530 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085061 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 11:11:35.089530 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085064 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 11:11:35.089530 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085067 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 11:11:35.089530 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085070 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 11:11:35.089530 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085072 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 11:11:35.089530 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085075 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 11:11:35.089530 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085077 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 11:11:35.089530 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085080 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 11:11:35.089530 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085082 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 11:11:35.089530 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085085 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 11:11:35.089530 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085087 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 11:11:35.090008 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085090 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 11:11:35.090008 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085092 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 11:11:35.090008 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085095 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 11:11:35.090008 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085097 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 11:11:35.090008 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085101 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 11:11:35.090008 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085104 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 11:11:35.090008 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085110 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 11:11:35.090008 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085112 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 11:11:35.090008 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085115 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 11:11:35.090008 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085117 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 11:11:35.090008 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085120 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 11:11:35.090008 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085123 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 11:11:35.090008 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085126 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 11:11:35.090008 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085129 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 11:11:35.090008 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085131 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 11:11:35.090008 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085133 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 11:11:35.090008 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085136 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 11:11:35.090008 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085138 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 11:11:35.090008 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085141 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 11:11:35.090008 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085143 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 11:11:35.090503 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085147 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 11:11:35.090503 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085150 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 11:11:35.090503 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085215 2550 flags.go:64] FLAG: --address="0.0.0.0" Jul 09 11:11:35.090503 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085222 2550 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 09 11:11:35.090503 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085228 2550 flags.go:64] FLAG: --anonymous-auth="true" Jul 09 11:11:35.090503 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085232 2550 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 09 11:11:35.090503 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085236 2550 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 09 11:11:35.090503 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085239 2550 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 09 11:11:35.090503 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085244 2550 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 09 11:11:35.090503 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085248 2550 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 09 11:11:35.090503 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085252 2550 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 09 11:11:35.090503 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085255 2550 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 09 11:11:35.090503 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085258 2550 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 09 11:11:35.090503 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085261 2550 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 09 11:11:35.090503 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085264 2550 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 09 11:11:35.090503 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085267 2550 flags.go:64] FLAG: --cgroup-root="" Jul 09 11:11:35.090503 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085270 2550 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 09 11:11:35.090503 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085273 2550 flags.go:64] FLAG: --client-ca-file="" Jul 09 11:11:35.090503 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085275 2550 flags.go:64] FLAG: --cloud-provider="external" Jul 09 11:11:35.090503 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085278 2550 flags.go:64] FLAG: --cluster-dns="[]" Jul 09 11:11:35.090503 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085282 2550 flags.go:64] FLAG: --cluster-domain="" Jul 09 11:11:35.090503 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085285 2550 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 09 11:11:35.090503 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085288 2550 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 09 11:11:35.090503 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085292 2550 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 09 11:11:35.091067 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085295 2550 flags.go:64] FLAG: --container-log-max-files="5" Jul 09 11:11:35.091067 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085299 2550 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 09 11:11:35.091067 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085302 2550 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 09 11:11:35.091067 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085305 2550 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 09 11:11:35.091067 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085308 2550 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 09 11:11:35.091067 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085311 2550 flags.go:64] FLAG: --contention-profiling="false" Jul 09 11:11:35.091067 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085314 2550 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 09 11:11:35.091067 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085317 2550 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 09 11:11:35.091067 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085321 2550 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 09 11:11:35.091067 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085323 2550 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 09 11:11:35.091067 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085341 2550 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 09 11:11:35.091067 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085344 2550 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 09 11:11:35.091067 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085347 2550 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 09 11:11:35.091067 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085350 2550 flags.go:64] FLAG: --enable-load-reader="false" Jul 09 11:11:35.091067 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085353 2550 flags.go:64] FLAG: --enable-server="true" Jul 09 11:11:35.091067 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085356 2550 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 09 11:11:35.091067 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085361 2550 flags.go:64] FLAG: --event-burst="100" Jul 09 11:11:35.091067 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085364 2550 flags.go:64] FLAG: --event-qps="50" Jul 09 11:11:35.091067 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085367 2550 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 09 11:11:35.091067 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085370 2550 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 09 11:11:35.091067 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085373 2550 flags.go:64] FLAG: --eviction-hard="" Jul 09 11:11:35.091067 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085377 2550 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 09 11:11:35.091067 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085379 2550 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 09 11:11:35.091067 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085382 2550 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 09 11:11:35.091067 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085385 2550 flags.go:64] FLAG: --eviction-soft="" Jul 09 11:11:35.091670 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085388 2550 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 09 11:11:35.091670 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085391 2550 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 09 11:11:35.091670 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085394 2550 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 09 11:11:35.091670 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085397 2550 flags.go:64] FLAG: --experimental-mounter-path="" Jul 09 11:11:35.091670 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085399 2550 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 09 11:11:35.091670 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085402 2550 flags.go:64] FLAG: --fail-swap-on="true" Jul 09 11:11:35.091670 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085405 2550 flags.go:64] FLAG: --feature-gates="" Jul 09 11:11:35.091670 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085409 2550 flags.go:64] FLAG: --file-check-frequency="20s" Jul 09 11:11:35.091670 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085412 2550 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 09 11:11:35.091670 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085415 2550 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 09 11:11:35.091670 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085418 2550 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 09 11:11:35.091670 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085421 2550 flags.go:64] FLAG: --healthz-port="10248" Jul 09 11:11:35.091670 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085424 2550 flags.go:64] FLAG: --help="false" Jul 09 11:11:35.091670 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085427 2550 flags.go:64] FLAG: --hostname-override="ip-10-0-143-142.ec2.internal" Jul 09 11:11:35.091670 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085430 2550 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 09 11:11:35.091670 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085433 2550 flags.go:64] FLAG: --http-check-frequency="20s" Jul 09 11:11:35.091670 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085436 2550 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 09 11:11:35.091670 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085439 2550 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 09 11:11:35.091670 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085443 2550 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 09 11:11:35.091670 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085446 2550 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 09 11:11:35.091670 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085448 2550 flags.go:64] FLAG: --image-service-endpoint="" Jul 09 11:11:35.091670 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085452 2550 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 09 11:11:35.091670 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085455 2550 flags.go:64] FLAG: --kube-api-burst="100" Jul 09 11:11:35.092272 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085457 2550 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 09 11:11:35.092272 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085460 2550 flags.go:64] FLAG: --kube-api-qps="50" Jul 09 11:11:35.092272 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085464 2550 flags.go:64] FLAG: --kube-reserved="" Jul 09 11:11:35.092272 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085467 2550 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 09 11:11:35.092272 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085469 2550 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 09 11:11:35.092272 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085472 2550 flags.go:64] FLAG: --kubelet-cgroups="" Jul 09 11:11:35.092272 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085475 2550 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 09 11:11:35.092272 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085478 2550 flags.go:64] FLAG: --lock-file="" Jul 09 11:11:35.092272 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085481 2550 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 09 11:11:35.092272 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085484 2550 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 09 11:11:35.092272 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085486 2550 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 09 11:11:35.092272 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085491 2550 flags.go:64] FLAG: --log-json-split-stream="false" Jul 09 11:11:35.092272 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085494 2550 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 09 11:11:35.092272 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085497 2550 flags.go:64] FLAG: --log-text-split-stream="false" Jul 09 11:11:35.092272 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085500 2550 flags.go:64] FLAG: --logging-format="text" Jul 09 11:11:35.092272 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085503 2550 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 09 11:11:35.092272 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085507 2550 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 09 11:11:35.092272 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085509 2550 flags.go:64] FLAG: --manifest-url="" Jul 09 11:11:35.092272 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085512 2550 flags.go:64] FLAG: --manifest-url-header="" Jul 09 11:11:35.092272 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085517 2550 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 09 11:11:35.092272 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085520 2550 flags.go:64] FLAG: --max-open-files="1000000" Jul 09 11:11:35.092272 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085524 2550 flags.go:64] FLAG: --max-pods="110" Jul 09 11:11:35.092272 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085526 2550 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 09 11:11:35.092272 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085530 2550 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 09 11:11:35.092272 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085553 2550 flags.go:64] FLAG: --memory-manager-policy="None" Jul 09 11:11:35.093108 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085566 2550 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 09 11:11:35.093108 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085571 2550 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 09 11:11:35.093108 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085575 2550 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 09 11:11:35.093108 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085578 2550 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 09 11:11:35.093108 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085588 2550 flags.go:64] FLAG: --node-status-max-images="50" Jul 09 11:11:35.093108 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085592 2550 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 09 11:11:35.093108 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085596 2550 flags.go:64] FLAG: --oom-score-adj="-999" Jul 09 11:11:35.093108 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085599 2550 flags.go:64] FLAG: --pod-cidr="" Jul 09 11:11:35.093108 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085602 2550 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 09 11:11:35.093108 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085608 2550 flags.go:64] FLAG: --pod-manifest-path="" Jul 09 11:11:35.093108 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085611 2550 flags.go:64] FLAG: --pod-max-pids="-1" Jul 09 11:11:35.093108 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085615 2550 flags.go:64] FLAG: --pods-per-core="0" Jul 09 11:11:35.093108 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085618 2550 flags.go:64] FLAG: --port="10250" Jul 09 11:11:35.093108 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085621 2550 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 09 11:11:35.093108 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085625 2550 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-07dd432da9b30d843" Jul 09 11:11:35.093108 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085628 2550 flags.go:64] FLAG: --qos-reserved="" Jul 09 11:11:35.093108 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085632 2550 flags.go:64] FLAG: --read-only-port="10255" Jul 09 11:11:35.093108 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085635 2550 flags.go:64] FLAG: --register-node="true" Jul 09 11:11:35.093108 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085638 2550 flags.go:64] FLAG: --register-with-taints="" Jul 09 11:11:35.093108 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085642 2550 flags.go:64] FLAG: --registry-burst="10" Jul 09 11:11:35.093108 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085645 2550 flags.go:64] FLAG: --registry-qps="5" Jul 09 11:11:35.093108 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085647 2550 flags.go:64] FLAG: --reserved-cpus="" Jul 09 11:11:35.093108 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085651 2550 flags.go:64] FLAG: --reserved-memory="" Jul 09 11:11:35.093108 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085662 2550 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 09 11:11:35.093108 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085666 2550 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 09 11:11:35.094096 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085669 2550 flags.go:64] FLAG: --rotate-certificates="false" Jul 09 11:11:35.094096 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085672 2550 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 09 11:11:35.094096 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085675 2550 flags.go:64] FLAG: --runonce="false" Jul 09 11:11:35.094096 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085678 2550 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 09 11:11:35.094096 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085681 2550 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 09 11:11:35.094096 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085684 2550 flags.go:64] FLAG: --seccomp-default="false" Jul 09 11:11:35.094096 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085688 2550 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 09 11:11:35.094096 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085691 2550 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 09 11:11:35.094096 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085695 2550 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 09 11:11:35.094096 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085699 2550 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 09 11:11:35.094096 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085702 2550 flags.go:64] FLAG: --storage-driver-password="root" Jul 09 11:11:35.094096 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085705 2550 flags.go:64] FLAG: --storage-driver-secure="false" Jul 09 11:11:35.094096 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085708 2550 flags.go:64] FLAG: --storage-driver-table="stats" Jul 09 11:11:35.094096 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085711 2550 flags.go:64] FLAG: --storage-driver-user="root" Jul 09 11:11:35.094096 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085714 2550 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 09 11:11:35.094096 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085717 2550 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 09 11:11:35.094096 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085720 2550 flags.go:64] FLAG: --system-cgroups="" Jul 09 11:11:35.094096 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085723 2550 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 09 11:11:35.094096 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085729 2550 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 09 11:11:35.094096 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085731 2550 flags.go:64] FLAG: --tls-cert-file="" Jul 09 11:11:35.094096 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085734 2550 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 09 11:11:35.094096 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085740 2550 flags.go:64] FLAG: --tls-min-version="" Jul 09 11:11:35.094096 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085743 2550 flags.go:64] FLAG: --tls-private-key-file="" Jul 09 11:11:35.094096 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085746 2550 flags.go:64] FLAG: --topology-manager-policy="none" Jul 09 11:11:35.094096 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085749 2550 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 09 11:11:35.094768 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085752 2550 flags.go:64] FLAG: --topology-manager-scope="container" Jul 09 11:11:35.094768 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085755 2550 flags.go:64] FLAG: --v="2" Jul 09 11:11:35.094768 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085759 2550 flags.go:64] FLAG: --version="false" Jul 09 11:11:35.094768 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085763 2550 flags.go:64] FLAG: --vmodule="" Jul 09 11:11:35.094768 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085767 2550 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 09 11:11:35.094768 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.085770 2550 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 09 11:11:35.094768 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085925 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 11:11:35.094768 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085930 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 11:11:35.094768 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085934 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 11:11:35.094768 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085937 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 11:11:35.094768 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085940 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 11:11:35.094768 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085943 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 11:11:35.094768 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085945 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 11:11:35.094768 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085948 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 11:11:35.094768 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085951 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 11:11:35.094768 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085955 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 11:11:35.094768 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085957 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 11:11:35.094768 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085960 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 11:11:35.094768 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085963 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 11:11:35.094768 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085966 2550 feature_gate.go:328] unrecognized feature gate: Example Jul 09 11:11:35.094768 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085969 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 11:11:35.095289 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085973 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 11:11:35.095289 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085977 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 11:11:35.095289 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085980 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 11:11:35.095289 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085983 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 11:11:35.095289 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085986 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 11:11:35.095289 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085989 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 11:11:35.095289 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085991 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 11:11:35.095289 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085994 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 11:11:35.095289 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085996 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 11:11:35.095289 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.085999 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 11:11:35.095289 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086001 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 11:11:35.095289 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086004 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 11:11:35.095289 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086006 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 11:11:35.095289 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086009 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 11:11:35.095289 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086011 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 11:11:35.095289 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086014 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 11:11:35.095289 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086017 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 11:11:35.095289 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086019 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 11:11:35.095289 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086022 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 11:11:35.095788 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086025 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 11:11:35.095788 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086028 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 11:11:35.095788 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086030 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 11:11:35.095788 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086033 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 11:11:35.095788 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086035 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 11:11:35.095788 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086038 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 11:11:35.095788 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086040 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 11:11:35.095788 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086043 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 11:11:35.095788 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086046 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 11:11:35.095788 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086048 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 11:11:35.095788 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086051 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 11:11:35.095788 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086055 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 11:11:35.095788 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086058 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 11:11:35.095788 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086061 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 11:11:35.095788 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086064 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 11:11:35.095788 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086066 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 11:11:35.095788 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086069 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 11:11:35.095788 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086071 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 11:11:35.095788 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086074 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 11:11:35.095788 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086076 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 11:11:35.096289 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086079 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 11:11:35.096289 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086081 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 11:11:35.096289 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086083 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 11:11:35.096289 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086086 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 11:11:35.096289 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086088 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 11:11:35.096289 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086090 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 11:11:35.096289 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086093 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 11:11:35.096289 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086096 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 11:11:35.096289 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086100 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 11:11:35.096289 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086103 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 11:11:35.096289 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086106 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 11:11:35.096289 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086108 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 11:11:35.096289 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086111 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 11:11:35.096289 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086113 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 11:11:35.096289 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086116 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 11:11:35.096289 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086118 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 11:11:35.096289 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086121 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 11:11:35.096289 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086125 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 11:11:35.096289 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086128 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 11:11:35.096289 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086133 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 11:11:35.096867 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086136 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 11:11:35.096867 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086138 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 11:11:35.096867 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086141 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 11:11:35.096867 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086144 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 11:11:35.096867 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086146 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 11:11:35.096867 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086149 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 11:11:35.096867 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086152 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 11:11:35.096867 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086154 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 11:11:35.096867 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086157 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 11:11:35.096867 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086159 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 11:11:35.096867 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086162 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 11:11:35.096867 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086165 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 11:11:35.096867 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086167 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 11:11:35.096867 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086170 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 11:11:35.096867 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086172 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 11:11:35.096867 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086175 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 11:11:35.096867 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086178 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 11:11:35.096867 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086181 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 11:11:35.096867 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086184 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 11:11:35.096867 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086186 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 11:11:35.097354 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086191 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 11:11:35.097354 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086193 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 11:11:35.097354 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086196 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 11:11:35.097354 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086198 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 11:11:35.097354 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.086201 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 11:11:35.097354 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.086812 2550 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 11:11:35.097354 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.094984 2550 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 09 11:11:35.097354 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.095000 2550 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 09 11:11:35.097354 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.095651 2550 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 09 11:11:35.097354 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.095660 2550 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 09 11:11:35.097354 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095719 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 11:11:35.097354 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095724 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 11:11:35.097354 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095728 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 11:11:35.097354 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095733 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 11:11:35.097354 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095736 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 11:11:35.097728 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095739 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 11:11:35.097728 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095742 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 11:11:35.097728 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095745 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 11:11:35.097728 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095748 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 11:11:35.097728 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095750 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 11:11:35.097728 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095754 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 11:11:35.097728 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095758 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 11:11:35.097728 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095761 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 11:11:35.097728 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095764 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 11:11:35.097728 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095766 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 11:11:35.097728 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095770 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 11:11:35.097728 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095773 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 11:11:35.097728 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095775 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 11:11:35.097728 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095778 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 11:11:35.097728 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095781 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 11:11:35.097728 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095783 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 11:11:35.097728 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095787 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 11:11:35.097728 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095790 2550 feature_gate.go:328] unrecognized feature gate: Example Jul 09 11:11:35.097728 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095798 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 11:11:35.097728 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095801 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 11:11:35.098207 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095805 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 11:11:35.098207 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095809 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 11:11:35.098207 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095812 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 11:11:35.098207 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095814 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 11:11:35.098207 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095817 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 11:11:35.098207 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095820 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 11:11:35.098207 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095822 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 11:11:35.098207 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095825 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 11:11:35.098207 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095827 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 11:11:35.098207 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095830 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 11:11:35.098207 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095833 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 11:11:35.098207 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095836 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 11:11:35.098207 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095839 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 11:11:35.098207 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095841 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 11:11:35.098207 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095844 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 11:11:35.098207 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095846 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 11:11:35.098207 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095849 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 11:11:35.098207 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095851 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 11:11:35.098207 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095854 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 11:11:35.098207 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095856 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 11:11:35.098733 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095859 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 11:11:35.098733 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095861 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 11:11:35.098733 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095864 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 11:11:35.098733 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095868 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 11:11:35.098733 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095872 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 11:11:35.098733 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095875 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 11:11:35.098733 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095878 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 11:11:35.098733 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095880 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 11:11:35.098733 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095883 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 11:11:35.098733 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095886 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 11:11:35.098733 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095888 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 11:11:35.098733 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095891 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 11:11:35.098733 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095894 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 11:11:35.098733 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095896 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 11:11:35.098733 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095899 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 11:11:35.098733 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095902 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 11:11:35.098733 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095904 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 11:11:35.098733 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095907 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 11:11:35.099157 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095910 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 11:11:35.099157 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095912 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 11:11:35.099157 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095915 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 11:11:35.099157 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095918 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 11:11:35.099157 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095921 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 11:11:35.099157 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095923 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 11:11:35.099157 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095926 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 11:11:35.099157 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095929 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 11:11:35.099157 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095931 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 11:11:35.099157 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095935 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 11:11:35.099157 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095937 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 11:11:35.099157 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095940 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 11:11:35.099157 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095942 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 11:11:35.099157 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095945 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 11:11:35.099157 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095948 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 11:11:35.099157 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095950 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 11:11:35.099157 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095953 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 11:11:35.099157 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095956 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 11:11:35.099157 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095958 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 11:11:35.099157 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095961 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 11:11:35.099654 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095963 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 11:11:35.099654 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095966 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 11:11:35.099654 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095968 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 11:11:35.099654 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095970 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 11:11:35.099654 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095973 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 11:11:35.099654 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095976 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 11:11:35.099654 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095978 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 11:11:35.099654 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095981 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 11:11:35.099654 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095983 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 11:11:35.099654 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095986 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 11:11:35.099654 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095988 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 11:11:35.099654 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095991 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 11:11:35.099654 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095994 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 11:11:35.099654 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095997 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 11:11:35.099654 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.095999 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 11:11:35.099654 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096002 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 11:11:35.100023 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.096007 2550 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 11:11:35.100023 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096121 2550 feature_gate.go:328] unrecognized feature gate: Example Jul 09 11:11:35.100023 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096126 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 11:11:35.100023 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096129 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 11:11:35.100023 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096132 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 11:11:35.100023 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096135 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 11:11:35.100023 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096138 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 11:11:35.100023 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096141 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 11:11:35.100023 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096144 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 11:11:35.100023 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096147 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 11:11:35.100023 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096149 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 11:11:35.100023 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096152 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 11:11:35.100023 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096154 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 11:11:35.100023 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096157 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 11:11:35.100023 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096160 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 11:11:35.100023 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096163 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 11:11:35.100512 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096165 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 11:11:35.100512 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096168 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 11:11:35.100512 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096171 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 11:11:35.100512 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096173 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 11:11:35.100512 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096176 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 11:11:35.100512 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096178 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 11:11:35.100512 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096181 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 11:11:35.100512 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096184 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 11:11:35.100512 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096186 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 11:11:35.100512 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096189 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 11:11:35.100512 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096191 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 11:11:35.100512 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096194 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 11:11:35.100512 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096197 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 11:11:35.100512 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096199 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 11:11:35.100512 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096202 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 11:11:35.100512 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096204 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 11:11:35.100512 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096208 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 11:11:35.100512 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096212 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 11:11:35.100512 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096215 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 11:11:35.100966 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096218 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 11:11:35.100966 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096221 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 11:11:35.100966 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096223 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 11:11:35.100966 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096226 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 11:11:35.100966 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096228 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 11:11:35.100966 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096230 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 11:11:35.100966 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096233 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 11:11:35.100966 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096236 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 11:11:35.100966 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096238 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 11:11:35.100966 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096241 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 11:11:35.100966 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096243 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 11:11:35.100966 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096247 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 11:11:35.100966 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096249 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 11:11:35.100966 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096252 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 11:11:35.100966 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096256 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 11:11:35.100966 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096259 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 11:11:35.100966 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096261 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 11:11:35.100966 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096264 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 11:11:35.100966 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096268 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 11:11:35.101432 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096271 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 11:11:35.101432 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096274 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 11:11:35.101432 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096277 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 11:11:35.101432 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096280 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 11:11:35.101432 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096283 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 11:11:35.101432 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096285 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 11:11:35.101432 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096288 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 11:11:35.101432 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096291 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 11:11:35.101432 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096293 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 11:11:35.101432 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096296 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 11:11:35.101432 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096299 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 11:11:35.101432 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096302 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 11:11:35.101432 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096304 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 11:11:35.101432 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096307 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 11:11:35.101432 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096310 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 11:11:35.101432 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096312 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 11:11:35.101432 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096315 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 11:11:35.101432 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096318 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 11:11:35.101432 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096320 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 11:11:35.101432 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096323 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 11:11:35.101959 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096340 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 11:11:35.101959 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096345 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 11:11:35.101959 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096348 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 11:11:35.101959 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096350 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 11:11:35.101959 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096354 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 11:11:35.101959 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096357 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 11:11:35.101959 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096360 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 11:11:35.101959 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096364 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 11:11:35.101959 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096367 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 11:11:35.101959 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096369 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 11:11:35.101959 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096372 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 11:11:35.101959 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096374 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 11:11:35.101959 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096377 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 11:11:35.101959 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096380 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 11:11:35.101959 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096382 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 11:11:35.101959 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096385 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 11:11:35.101959 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096390 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 11:11:35.101959 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096393 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 11:11:35.101959 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096396 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 11:11:35.101959 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096398 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 11:11:35.102466 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096401 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 11:11:35.102466 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096404 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 11:11:35.102466 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096407 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 11:11:35.102466 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096411 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 11:11:35.102466 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096413 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 11:11:35.102466 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:35.096416 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 11:11:35.102466 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.096421 2550 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 11:11:35.102466 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.096529 2550 server.go:962] "Client rotation is on, will bootstrap in background" Jul 09 11:11:35.102466 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.098875 2550 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 09 11:11:35.102466 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.101150 2550 server.go:1019] "Starting client certificate rotation" Jul 09 11:11:35.102466 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.101256 2550 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 11:11:35.102466 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.101286 2550 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 11:11:35.103573 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.103560 2550 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 11:11:35.104703 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.104686 2550 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 11:11:35.113566 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.113548 2550 log.go:25] "Validated CRI v1 runtime API" Jul 09 11:11:35.115420 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.115400 2550 log.go:25] "Validated CRI v1 image API" Jul 09 11:11:35.116288 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.116266 2550 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 09 11:11:35.118986 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.118967 2550 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 b32b6fe8-d53a-4ea9-b8c4-1e3ded6f7baa:/dev/nvme0n1p3 f4b8488f-94db-4694-be26-ae1012278713:/dev/nvme0n1p4] Jul 09 11:11:35.119058 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.118985 2550 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 09 11:11:35.123072 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.122961 2550 manager.go:216] Machine: {Timestamp:2026-07-09 11:11:35.122494428 +0000 UTC m=+0.324813569 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3106655 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec260d240facb341b76aab525dc0666f SystemUUID:ec260d24-0fac-b341-b76a-ab525dc0666f BootID:bb0c557e-bc23-42ff-befe-922fbed7646e Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:dd:e1:23:79:ff Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:dd:e1:23:79:ff Speed:0 Mtu:9001} {Name:ovs-system MacAddress:c2:ad:ac:07:57:1b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 09 11:11:35.123072 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.123065 2550 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 09 11:11:35.123192 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.123144 2550 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 09 11:11:35.126068 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.126043 2550 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 09 11:11:35.126202 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.126071 2550 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-142.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 09 11:11:35.127070 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.127060 2550 topology_manager.go:138] "Creating topology manager with none policy" Jul 09 11:11:35.127105 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.127073 2550 container_manager_linux.go:309] "Creating device plugin manager" Jul 09 11:11:35.127105 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.127081 2550 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 11:11:35.127658 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.127644 2550 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 11:11:35.128818 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.128807 2550 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 09 11:11:35.131486 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.131472 2550 state_mem.go:36] "Initialized new in-memory state store" Jul 09 11:11:35.132167 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.132140 2550 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 09 11:11:35.132252 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.132198 2550 kubelet.go:486] "Attempting to sync node with API server" Jul 09 11:11:35.132252 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.132222 2550 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 09 11:11:35.132252 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.132244 2550 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 09 11:11:35.132396 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.132260 2550 kubelet.go:398] "Adding apiserver pod source" Jul 09 11:11:35.132396 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.132278 2550 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 09 11:11:35.133364 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.132833 2550 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 11:11:35.133364 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.132867 2550 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 11:11:35.134585 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.134566 2550 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 11:11:35.136647 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.136627 2550 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 09 11:11:35.138422 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.138408 2550 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 09 11:11:35.138473 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.138438 2550 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 09 11:11:35.139663 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.139649 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 09 11:11:35.139742 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.139668 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 09 11:11:35.139742 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.139676 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 09 11:11:35.139742 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.139681 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 09 11:11:35.139742 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.139690 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 09 11:11:35.139742 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.139696 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 09 11:11:35.139742 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.139704 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 09 11:11:35.139742 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.139709 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 09 11:11:35.139742 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.139716 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 09 11:11:35.139742 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.139721 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 09 11:11:35.139742 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.139729 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 09 11:11:35.139742 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.139734 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 09 11:11:35.140380 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.140370 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 09 11:11:35.140380 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.140379 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 09 11:11:35.143459 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.143444 2550 server.go:1268] "Started kubelet" Jul 09 11:11:35.143553 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.143524 2550 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 09 11:11:35.143612 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.143568 2550 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 09 11:11:35.144141 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.144104 2550 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 09 11:11:35.144232 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:35.144133 2550 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-142.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 09 11:11:35.144232 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:35.144172 2550 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 09 11:11:35.144316 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.144121 2550 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-142.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 09 11:11:35.144298 ip-10-0-143-142 systemd[1]: Started Kubernetes Kubelet. Jul 09 11:11:35.145437 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.145422 2550 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 09 11:11:35.146076 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.146062 2550 server.go:310] "Adding debug handlers to kubelet server" Jul 09 11:11:35.149900 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:35.149090 2550 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-142.ec2.internal.18c09aa72702a9b0 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-142.ec2.internal,UID:ip-10-0-143-142.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-142.ec2.internal,},FirstTimestamp:2026-07-09 11:11:35.143418288 +0000 UTC m=+0.345737427,LastTimestamp:2026-07-09 11:11:35.143418288 +0000 UTC m=+0.345737427,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-142.ec2.internal,}" Jul 09 11:11:35.150764 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.150740 2550 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 09 11:11:35.150857 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.150756 2550 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 09 11:11:35.151119 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:35.151007 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-142.ec2.internal\" not found" Jul 09 11:11:35.151539 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.151524 2550 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 09 11:11:35.151638 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.151627 2550 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 09 11:11:35.151834 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.151817 2550 reconstruct.go:97] "Volume reconstruction finished" Jul 09 11:11:35.151834 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.151834 2550 reconciler.go:29] "Reconciler: start to sync state" Jul 09 11:11:35.152061 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.152046 2550 factory.go:55] Registering systemd factory Jul 09 11:11:35.152129 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.152064 2550 factory.go:223] Registration of the systemd container factory successfully Jul 09 11:11:35.152206 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:35.152126 2550 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 09 11:11:35.152285 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.152269 2550 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 09 11:11:35.152352 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.152285 2550 factory.go:153] Registering CRI-O factory Jul 09 11:11:35.152352 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.152300 2550 factory.go:223] Registration of the crio container factory successfully Jul 09 11:11:35.152408 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.152374 2550 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 09 11:11:35.152445 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.152421 2550 factory.go:103] Registering Raw factory Jul 09 11:11:35.152445 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.152438 2550 manager.go:1196] Started watching for new ooms in manager Jul 09 11:11:35.153586 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.153571 2550 manager.go:349] Starting recovery of all containers Jul 09 11:11:35.158551 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.158530 2550 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kq7dp" Jul 09 11:11:35.161180 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:35.161144 2550 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-142.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 09 11:11:35.161288 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:35.161184 2550 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 09 11:11:35.165149 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.165131 2550 manager.go:354] Recovery completed Jul 09 11:11:35.166297 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.166275 2550 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kq7dp" Jul 09 11:11:35.169675 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.169662 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 11:11:35.172507 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.172478 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeHasSufficientMemory" Jul 09 11:11:35.172551 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.172514 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 11:11:35.172551 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.172524 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeHasSufficientPID" Jul 09 11:11:35.173071 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.173056 2550 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 09 11:11:35.173071 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.173067 2550 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 09 11:11:35.173172 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.173083 2550 state_mem.go:36] "Initialized new in-memory state store" Jul 09 11:11:35.173172 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.173058 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 11:11:35.173828 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.173784 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeHasSufficientMemory" Jul 09 11:11:35.173828 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.173812 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 11:11:35.173828 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.173826 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeHasSufficientPID" Jul 09 11:11:35.174318 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.174305 2550 policy_none.go:49] "None policy: Start" Jul 09 11:11:35.174318 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.174320 2550 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 09 11:11:35.174420 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.174383 2550 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 09 11:11:35.175286 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.175276 2550 policy_none.go:47] "Start" Jul 09 11:11:35.210994 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.210860 2550 manager.go:337] "Starting Device Plugin manager" Jul 09 11:11:35.211102 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:35.211009 2550 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 09 11:11:35.211102 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.211022 2550 server.go:87] "Starting device plugin registration server" Jul 09 11:11:35.211273 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.211260 2550 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 09 11:11:35.211346 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.211276 2550 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 09 11:11:35.211403 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.211392 2550 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 09 11:11:35.211512 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.211485 2550 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 09 11:11:35.211512 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.211509 2550 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 09 11:11:35.212523 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:35.212506 2550 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 09 11:11:35.212612 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:35.212546 2550 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-142.ec2.internal\" not found" Jul 09 11:11:35.291909 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.291868 2550 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 09 11:11:35.293454 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.293277 2550 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 09 11:11:35.293454 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.293317 2550 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 09 11:11:35.293454 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.293350 2550 kubelet.go:2448] "Starting kubelet main sync loop" Jul 09 11:11:35.293454 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:35.293387 2550 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 09 11:11:35.296159 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.296134 2550 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 11:11:35.312272 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.312253 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 11:11:35.313124 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.313108 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeHasSufficientMemory" Jul 09 11:11:35.313227 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.313136 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 11:11:35.313227 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.313145 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeHasSufficientPID" Jul 09 11:11:35.313227 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.313168 2550 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-142.ec2.internal" Jul 09 11:11:35.321338 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.321315 2550 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-142.ec2.internal" Jul 09 11:11:35.321388 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:35.321349 2550 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-142.ec2.internal\": node \"ip-10-0-143-142.ec2.internal\" not found" Jul 09 11:11:35.343149 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:35.343125 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-142.ec2.internal\" not found" Jul 09 11:11:35.393894 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.393832 2550 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-142.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-142.ec2.internal"] Jul 09 11:11:35.393993 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.393937 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 11:11:35.395153 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.395136 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeHasSufficientMemory" Jul 09 11:11:35.395233 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.395167 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 11:11:35.395233 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.395176 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeHasSufficientPID" Jul 09 11:11:35.396465 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.396453 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 11:11:35.396572 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.396558 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-142.ec2.internal" Jul 09 11:11:35.396627 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.396587 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 11:11:35.397255 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.397240 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeHasSufficientMemory" Jul 09 11:11:35.397358 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.397240 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeHasSufficientMemory" Jul 09 11:11:35.397358 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.397291 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 11:11:35.397358 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.397302 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeHasSufficientPID" Jul 09 11:11:35.397358 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.397270 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 11:11:35.397358 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.397351 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeHasSufficientPID" Jul 09 11:11:35.398673 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.398658 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-142.ec2.internal" Jul 09 11:11:35.398740 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.398683 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 11:11:35.399365 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.399348 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeHasSufficientMemory" Jul 09 11:11:35.399429 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.399378 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 11:11:35.399429 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.399390 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeHasSufficientPID" Jul 09 11:11:35.429048 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:35.428988 2550 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-142.ec2.internal\" not found" node="ip-10-0-143-142.ec2.internal" Jul 09 11:11:35.433587 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:35.433568 2550 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-142.ec2.internal\" not found" node="ip-10-0-143-142.ec2.internal" Jul 09 11:11:35.444119 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:35.444101 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-142.ec2.internal\" not found" Jul 09 11:11:35.453445 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.453424 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7dd8c39790ec51df8183e1a3a4cad09b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-142.ec2.internal\" (UID: \"7dd8c39790ec51df8183e1a3a4cad09b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-142.ec2.internal" Jul 09 11:11:35.453525 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.453450 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7dd8c39790ec51df8183e1a3a4cad09b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-142.ec2.internal\" (UID: \"7dd8c39790ec51df8183e1a3a4cad09b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-142.ec2.internal" Jul 09 11:11:35.453525 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.453477 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c0a0fd4f8c4f29eccb1f0904d07ef791-config\") pod \"kube-apiserver-proxy-ip-10-0-143-142.ec2.internal\" (UID: \"c0a0fd4f8c4f29eccb1f0904d07ef791\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-142.ec2.internal" Jul 09 11:11:35.544954 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:35.544927 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-142.ec2.internal\" not found" Jul 09 11:11:35.553987 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.553962 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7dd8c39790ec51df8183e1a3a4cad09b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-142.ec2.internal\" (UID: \"7dd8c39790ec51df8183e1a3a4cad09b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-142.ec2.internal" Jul 09 11:11:35.554053 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.553995 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7dd8c39790ec51df8183e1a3a4cad09b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-142.ec2.internal\" (UID: \"7dd8c39790ec51df8183e1a3a4cad09b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-142.ec2.internal" Jul 09 11:11:35.554053 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.554013 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c0a0fd4f8c4f29eccb1f0904d07ef791-config\") pod \"kube-apiserver-proxy-ip-10-0-143-142.ec2.internal\" (UID: \"c0a0fd4f8c4f29eccb1f0904d07ef791\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-142.ec2.internal" Jul 09 11:11:35.554132 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.554056 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7dd8c39790ec51df8183e1a3a4cad09b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-142.ec2.internal\" (UID: \"7dd8c39790ec51df8183e1a3a4cad09b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-142.ec2.internal" Jul 09 11:11:35.554189 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.554136 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c0a0fd4f8c4f29eccb1f0904d07ef791-config\") pod \"kube-apiserver-proxy-ip-10-0-143-142.ec2.internal\" (UID: \"c0a0fd4f8c4f29eccb1f0904d07ef791\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-142.ec2.internal" Jul 09 11:11:35.554232 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.554183 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7dd8c39790ec51df8183e1a3a4cad09b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-142.ec2.internal\" (UID: \"7dd8c39790ec51df8183e1a3a4cad09b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-142.ec2.internal" Jul 09 11:11:35.645443 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:35.645400 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-142.ec2.internal\" not found" Jul 09 11:11:35.731964 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.731870 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-142.ec2.internal" Jul 09 11:11:35.736578 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:35.736558 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-142.ec2.internal" Jul 09 11:11:35.746102 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:35.746080 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-142.ec2.internal\" not found" Jul 09 11:11:35.846683 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:35.846646 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-142.ec2.internal\" not found" Jul 09 11:11:35.947221 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:35.947187 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-142.ec2.internal\" not found" Jul 09 11:11:36.047727 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:36.047699 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-142.ec2.internal\" not found" Jul 09 11:11:36.099190 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:36.099162 2550 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 09 11:11:36.099645 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:36.099308 2550 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 11:11:36.147803 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:36.147772 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-142.ec2.internal\" not found" Jul 09 11:11:36.168674 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:36.168637 2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 11:06:35 +0000 UTC" deadline="2027-12-19 12:54:50.919128732 +0000 UTC" Jul 09 11:11:36.168674 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:36.168662 2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12673h43m14.75046937s" Jul 09 11:11:36.224024 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:36.223996 2550 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 11:11:36.248128 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:36.248102 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-142.ec2.internal\" not found" Jul 09 11:11:36.329758 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:36.329726 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0a0fd4f8c4f29eccb1f0904d07ef791.slice/crio-ea7c7ab18132568e883cefcb4046292ee4e544a0cd4ddc1928ed74a5dc9a1b12 WatchSource:0}: Error finding container ea7c7ab18132568e883cefcb4046292ee4e544a0cd4ddc1928ed74a5dc9a1b12: Status 404 returned error can't find the container with id ea7c7ab18132568e883cefcb4046292ee4e544a0cd4ddc1928ed74a5dc9a1b12 Jul 09 11:11:36.330559 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:36.330533 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dd8c39790ec51df8183e1a3a4cad09b.slice/crio-e4763abb3fba60dbc139058c1679b867df13ff50708e7d102693775420b1462b WatchSource:0}: Error finding container e4763abb3fba60dbc139058c1679b867df13ff50708e7d102693775420b1462b: Status 404 returned error can't find the container with id e4763abb3fba60dbc139058c1679b867df13ff50708e7d102693775420b1462b Jul 09 11:11:36.335538 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:36.335519 2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 11:11:36.348966 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:36.348941 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-142.ec2.internal\" not found" Jul 09 11:11:36.391984 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:36.391957 2550 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 11:11:36.450042 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:36.450011 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-142.ec2.internal\" not found" Jul 09 11:11:36.461094 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:36.461071 2550 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 11:11:36.551320 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:36.551290 2550 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-142.ec2.internal" Jul 09 11:11:36.562579 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:36.562561 2550 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 11:11:36.563504 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:36.563491 2550 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-142.ec2.internal" Jul 09 11:11:36.575362 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:36.575299 2550 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 11:11:37.134859 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.134826 2550 apiserver.go:52] "Watching apiserver" Jul 09 11:11:37.141544 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.141518 2550 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 09 11:11:37.142604 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.142581 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-gfmtp","openshift-network-diagnostics/network-check-target-4hjlg","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn4qf","openshift-image-registry/node-ca-ppz4r","openshift-multus/multus-additional-cni-plugins-n2nj6","openshift-multus/network-metrics-daemon-6bvn2","openshift-network-operator/iptables-alerter-cbgs5","openshift-ovn-kubernetes/ovnkube-node-fr9h4","kube-system/global-pull-secret-syncer-sg6nt","kube-system/konnectivity-agent-lpprd","kube-system/kube-apiserver-proxy-ip-10-0-143-142.ec2.internal","openshift-cluster-node-tuning-operator/tuned-wnqm7","openshift-dns/node-resolver-wsfmn","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-142.ec2.internal"] Jul 09 11:11:37.143978 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.143956 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cbgs5" Jul 09 11:11:37.145156 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.145131 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wsfmn" Jul 09 11:11:37.146431 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.146157 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 09 11:11:37.146431 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.146201 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn4qf" Jul 09 11:11:37.146431 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.146248 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 09 11:11:37.146431 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.146256 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-r86ns\"" Jul 09 11:11:37.146431 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.146362 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 09 11:11:37.147420 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.147283 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 09 11:11:37.147420 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.147290 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 09 11:11:37.148124 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.147614 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-zd7lq\"" Jul 09 11:11:37.148124 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.147711 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ppz4r" Jul 09 11:11:37.148650 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.148523 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 09 11:11:37.148650 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.148582 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 09 11:11:37.148786 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.148529 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 09 11:11:37.148989 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.148975 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-m8g8j\"" Jul 09 11:11:37.149195 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.149178 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-n2nj6" Jul 09 11:11:37.150052 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.150006 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 09 11:11:37.150052 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.150026 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-nrtnk\"" Jul 09 11:11:37.150366 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.150019 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 09 11:11:37.150492 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.150477 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bvn2" Jul 09 11:11:37.150576 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:37.150556 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bvn2" podUID="91432ff0-fa58-460b-a6d6-3a14a9f5f295" Jul 09 11:11:37.150683 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.150666 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 09 11:11:37.150984 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.150970 2550 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 09 11:11:37.151551 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.151532 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-tvwpd\"" Jul 09 11:11:37.151646 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.151565 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 09 11:11:37.151870 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.151850 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sg6nt" Jul 09 11:11:37.151965 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.151884 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 09 11:11:37.151965 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.151906 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 09 11:11:37.151965 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:37.151914 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sg6nt" podUID="aefb1aae-31d5-4c63-8245-3b4544e21e49" Jul 09 11:11:37.152121 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.151998 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 09 11:11:37.152219 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.152201 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 09 11:11:37.153642 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.153618 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.156006 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.155983 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 09 11:11:37.156728 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.156460 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-ncqts\"" Jul 09 11:11:37.156996 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.156976 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4hjlg" Jul 09 11:11:37.157095 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:37.157074 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4hjlg" podUID="00faf136-9553-489b-a648-408d89197e53" Jul 09 11:11:37.158860 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.158838 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lpprd" Jul 09 11:11:37.160303 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.160283 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.161594 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.161572 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.161722 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.161703 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 09 11:11:37.161790 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.161760 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 09 11:11:37.161858 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.161842 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-qqhzv\"" Jul 09 11:11:37.162651 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.162510 2550 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 11:11:37.162770 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.162743 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30d995da-9325-4192-9695-ad8ae483a809-system-cni-dir\") pod \"multus-additional-cni-plugins-n2nj6\" (UID: \"30d995da-9325-4192-9695-ad8ae483a809\") " pod="openshift-multus/multus-additional-cni-plugins-n2nj6" Jul 09 11:11:37.162902 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.162780 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/30d995da-9325-4192-9695-ad8ae483a809-cni-binary-copy\") pod \"multus-additional-cni-plugins-n2nj6\" (UID: \"30d995da-9325-4192-9695-ad8ae483a809\") " pod="openshift-multus/multus-additional-cni-plugins-n2nj6" Jul 09 11:11:37.162902 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.162791 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 09 11:11:37.162902 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.162807 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/13ee2110-7683-4113-a051-ff4ce0ac8f55-hostroot\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.162902 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.162835 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/13ee2110-7683-4113-a051-ff4ce0ac8f55-multus-daemon-config\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.162902 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.162885 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/13ee2110-7683-4113-a051-ff4ce0ac8f55-etc-kubernetes\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.163221 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.162936 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4465f\" (UniqueName: \"kubernetes.io/projected/eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6-kube-api-access-4465f\") pod \"aws-ebs-csi-driver-node-tn4qf\" (UID: \"eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn4qf" Jul 09 11:11:37.163221 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.162962 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8207614d-56bb-42cd-a5e6-d7fd5da8dac1-host\") pod \"node-ca-ppz4r\" (UID: \"8207614d-56bb-42cd-a5e6-d7fd5da8dac1\") " pod="openshift-image-registry/node-ca-ppz4r" Jul 09 11:11:37.163221 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.162990 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 09 11:11:37.163221 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.163043 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 09 11:11:37.163221 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.163153 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 09 11:11:37.163221 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.163162 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/30d995da-9325-4192-9695-ad8ae483a809-os-release\") pod \"multus-additional-cni-plugins-n2nj6\" (UID: \"30d995da-9325-4192-9695-ad8ae483a809\") " pod="openshift-multus/multus-additional-cni-plugins-n2nj6" Jul 09 11:11:37.163221 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.163221 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/30d995da-9325-4192-9695-ad8ae483a809-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n2nj6\" (UID: \"30d995da-9325-4192-9695-ad8ae483a809\") " pod="openshift-multus/multus-additional-cni-plugins-n2nj6" Jul 09 11:11:37.163564 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.163230 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-r8zpx\"" Jul 09 11:11:37.163564 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.163274 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 09 11:11:37.163564 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.163247 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/13ee2110-7683-4113-a051-ff4ce0ac8f55-host-var-lib-cni-bin\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.163564 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.163319 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxkbf\" (UniqueName: \"kubernetes.io/projected/8117b6cb-dfab-462d-ae86-50a342e88a9a-kube-api-access-jxkbf\") pod \"node-resolver-wsfmn\" (UID: \"8117b6cb-dfab-462d-ae86-50a342e88a9a\") " pod="openshift-dns/node-resolver-wsfmn" Jul 09 11:11:37.163564 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.163392 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tn4qf\" (UID: \"eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn4qf" Jul 09 11:11:37.163564 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.163406 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 09 11:11:37.163564 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.163453 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d8c8014c-ce48-40d4-94a3-b0a5cd71b2a9-iptables-alerter-script\") pod \"iptables-alerter-cbgs5\" (UID: \"d8c8014c-ce48-40d4-94a3-b0a5cd71b2a9\") " pod="openshift-network-operator/iptables-alerter-cbgs5" Jul 09 11:11:37.163564 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.163478 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fxwt\" (UniqueName: \"kubernetes.io/projected/d8c8014c-ce48-40d4-94a3-b0a5cd71b2a9-kube-api-access-6fxwt\") pod \"iptables-alerter-cbgs5\" (UID: \"d8c8014c-ce48-40d4-94a3-b0a5cd71b2a9\") " pod="openshift-network-operator/iptables-alerter-cbgs5" Jul 09 11:11:37.163564 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.163563 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/30d995da-9325-4192-9695-ad8ae483a809-cnibin\") pod \"multus-additional-cni-plugins-n2nj6\" (UID: \"30d995da-9325-4192-9695-ad8ae483a809\") " pod="openshift-multus/multus-additional-cni-plugins-n2nj6" Jul 09 11:11:37.164128 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.163717 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/13ee2110-7683-4113-a051-ff4ce0ac8f55-multus-cni-dir\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.164128 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.163757 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91432ff0-fa58-460b-a6d6-3a14a9f5f295-metrics-certs\") pod \"network-metrics-daemon-6bvn2\" (UID: \"91432ff0-fa58-460b-a6d6-3a14a9f5f295\") " pod="openshift-multus/network-metrics-daemon-6bvn2" Jul 09 11:11:37.164128 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.163782 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxgsx\" (UniqueName: \"kubernetes.io/projected/91432ff0-fa58-460b-a6d6-3a14a9f5f295-kube-api-access-kxgsx\") pod \"network-metrics-daemon-6bvn2\" (UID: \"91432ff0-fa58-460b-a6d6-3a14a9f5f295\") " pod="openshift-multus/network-metrics-daemon-6bvn2" Jul 09 11:11:37.164128 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.163807 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8207614d-56bb-42cd-a5e6-d7fd5da8dac1-serviceca\") pod \"node-ca-ppz4r\" (UID: \"8207614d-56bb-42cd-a5e6-d7fd5da8dac1\") " pod="openshift-image-registry/node-ca-ppz4r" Jul 09 11:11:37.164128 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.163829 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aefb1aae-31d5-4c63-8245-3b4544e21e49-original-pull-secret\") pod \"global-pull-secret-syncer-sg6nt\" (UID: \"aefb1aae-31d5-4c63-8245-3b4544e21e49\") " pod="kube-system/global-pull-secret-syncer-sg6nt" Jul 09 11:11:37.164128 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.163876 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/13ee2110-7683-4113-a051-ff4ce0ac8f55-cnibin\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.164128 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.163907 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/13ee2110-7683-4113-a051-ff4ce0ac8f55-os-release\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.164128 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.163934 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/13ee2110-7683-4113-a051-ff4ce0ac8f55-multus-socket-dir-parent\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.164128 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.163959 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/13ee2110-7683-4113-a051-ff4ce0ac8f55-host-var-lib-cni-multus\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.164128 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.163978 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 09 11:11:37.164128 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.163982 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/13ee2110-7683-4113-a051-ff4ce0ac8f55-host-run-multus-certs\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.164128 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.164006 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfv7m\" (UniqueName: \"kubernetes.io/projected/13ee2110-7683-4113-a051-ff4ce0ac8f55-kube-api-access-sfv7m\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.164128 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.164030 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/13ee2110-7683-4113-a051-ff4ce0ac8f55-host-run-netns\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.164128 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.164067 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/30d995da-9325-4192-9695-ad8ae483a809-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n2nj6\" (UID: \"30d995da-9325-4192-9695-ad8ae483a809\") " pod="openshift-multus/multus-additional-cni-plugins-n2nj6" Jul 09 11:11:37.164128 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.164114 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/13ee2110-7683-4113-a051-ff4ce0ac8f55-system-cni-dir\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.164768 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.164156 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/13ee2110-7683-4113-a051-ff4ce0ac8f55-cni-binary-copy\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.164768 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.164239 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8117b6cb-dfab-462d-ae86-50a342e88a9a-tmp-dir\") pod \"node-resolver-wsfmn\" (UID: \"8117b6cb-dfab-462d-ae86-50a342e88a9a\") " pod="openshift-dns/node-resolver-wsfmn" Jul 09 11:11:37.164768 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.164267 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6-registration-dir\") pod \"aws-ebs-csi-driver-node-tn4qf\" (UID: \"eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn4qf" Jul 09 11:11:37.164768 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.164291 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6-device-dir\") pod \"aws-ebs-csi-driver-node-tn4qf\" (UID: \"eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn4qf" Jul 09 11:11:37.164768 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.164363 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnd4b\" (UniqueName: \"kubernetes.io/projected/8207614d-56bb-42cd-a5e6-d7fd5da8dac1-kube-api-access-qnd4b\") pod \"node-ca-ppz4r\" (UID: \"8207614d-56bb-42cd-a5e6-d7fd5da8dac1\") " pod="openshift-image-registry/node-ca-ppz4r" Jul 09 11:11:37.164768 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.164399 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8117b6cb-dfab-462d-ae86-50a342e88a9a-hosts-file\") pod \"node-resolver-wsfmn\" (UID: \"8117b6cb-dfab-462d-ae86-50a342e88a9a\") " pod="openshift-dns/node-resolver-wsfmn" Jul 09 11:11:37.164768 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.164424 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6-socket-dir\") pod \"aws-ebs-csi-driver-node-tn4qf\" (UID: \"eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn4qf" Jul 09 11:11:37.164768 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.164447 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6-etc-selinux\") pod \"aws-ebs-csi-driver-node-tn4qf\" (UID: \"eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn4qf" Jul 09 11:11:37.164768 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.164484 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcfdl\" (UniqueName: \"kubernetes.io/projected/30d995da-9325-4192-9695-ad8ae483a809-kube-api-access-tcfdl\") pod \"multus-additional-cni-plugins-n2nj6\" (UID: \"30d995da-9325-4192-9695-ad8ae483a809\") " pod="openshift-multus/multus-additional-cni-plugins-n2nj6" Jul 09 11:11:37.164768 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.164512 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/13ee2110-7683-4113-a051-ff4ce0ac8f55-host-var-lib-kubelet\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.164768 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.164534 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/13ee2110-7683-4113-a051-ff4ce0ac8f55-host-run-k8s-cni-cncf-io\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.164768 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.164567 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d8c8014c-ce48-40d4-94a3-b0a5cd71b2a9-host-slash\") pod \"iptables-alerter-cbgs5\" (UID: \"d8c8014c-ce48-40d4-94a3-b0a5cd71b2a9\") " pod="openshift-network-operator/iptables-alerter-cbgs5" Jul 09 11:11:37.164768 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.164599 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/30d995da-9325-4192-9695-ad8ae483a809-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n2nj6\" (UID: \"30d995da-9325-4192-9695-ad8ae483a809\") " pod="openshift-multus/multus-additional-cni-plugins-n2nj6" Jul 09 11:11:37.164768 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.164627 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/aefb1aae-31d5-4c63-8245-3b4544e21e49-kubelet-config\") pod \"global-pull-secret-syncer-sg6nt\" (UID: \"aefb1aae-31d5-4c63-8245-3b4544e21e49\") " pod="kube-system/global-pull-secret-syncer-sg6nt" Jul 09 11:11:37.164768 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.164647 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/aefb1aae-31d5-4c63-8245-3b4544e21e49-dbus\") pod \"global-pull-secret-syncer-sg6nt\" (UID: \"aefb1aae-31d5-4c63-8245-3b4544e21e49\") " pod="kube-system/global-pull-secret-syncer-sg6nt" Jul 09 11:11:37.164768 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.164678 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/13ee2110-7683-4113-a051-ff4ce0ac8f55-multus-conf-dir\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.165713 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.164705 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6-sys-fs\") pod \"aws-ebs-csi-driver-node-tn4qf\" (UID: \"eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn4qf" Jul 09 11:11:37.165713 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.164725 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-cvfzt\"" Jul 09 11:11:37.165713 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.164927 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 09 11:11:37.165713 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.165103 2550 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 11:11:37.185254 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.185230 2550 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hzrcj" Jul 09 11:11:37.193964 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.193878 2550 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hzrcj" Jul 09 11:11:37.253066 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.253044 2550 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 09 11:11:37.265417 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.265135 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d8c8014c-ce48-40d4-94a3-b0a5cd71b2a9-iptables-alerter-script\") pod \"iptables-alerter-cbgs5\" (UID: \"d8c8014c-ce48-40d4-94a3-b0a5cd71b2a9\") " pod="openshift-network-operator/iptables-alerter-cbgs5" Jul 09 11:11:37.265417 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.265172 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/30d995da-9325-4192-9695-ad8ae483a809-cnibin\") pod \"multus-additional-cni-plugins-n2nj6\" (UID: \"30d995da-9325-4192-9695-ad8ae483a809\") " pod="openshift-multus/multus-additional-cni-plugins-n2nj6" Jul 09 11:11:37.265417 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.265201 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/13ee2110-7683-4113-a051-ff4ce0ac8f55-multus-cni-dir\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.265417 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.265224 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91432ff0-fa58-460b-a6d6-3a14a9f5f295-metrics-certs\") pod \"network-metrics-daemon-6bvn2\" (UID: \"91432ff0-fa58-460b-a6d6-3a14a9f5f295\") " pod="openshift-multus/network-metrics-daemon-6bvn2" Jul 09 11:11:37.265417 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.265252 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6ab7210c-0aa5-49ab-8a57-295dad58dce8-etc-sysctl-conf\") pod \"tuned-wnqm7\" (UID: \"6ab7210c-0aa5-49ab-8a57-295dad58dce8\") " pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.265417 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.265273 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a6b1c406-e931-4645-9fca-97ee5da046e1-ovn-node-metrics-cert\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.265417 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.265296 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/47077616-7ca2-4b7d-ba31-92d393ab7439-agent-certs\") pod \"konnectivity-agent-lpprd\" (UID: \"47077616-7ca2-4b7d-ba31-92d393ab7439\") " pod="kube-system/konnectivity-agent-lpprd" Jul 09 11:11:37.265417 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.265320 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6ab7210c-0aa5-49ab-8a57-295dad58dce8-etc-sysctl-d\") pod \"tuned-wnqm7\" (UID: \"6ab7210c-0aa5-49ab-8a57-295dad58dce8\") " pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.265417 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.265360 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aefb1aae-31d5-4c63-8245-3b4544e21e49-original-pull-secret\") pod \"global-pull-secret-syncer-sg6nt\" (UID: \"aefb1aae-31d5-4c63-8245-3b4544e21e49\") " pod="kube-system/global-pull-secret-syncer-sg6nt" Jul 09 11:11:37.265417 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.265380 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/13ee2110-7683-4113-a051-ff4ce0ac8f55-cnibin\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.265417 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.265401 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/13ee2110-7683-4113-a051-ff4ce0ac8f55-os-release\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.265417 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.265424 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/13ee2110-7683-4113-a051-ff4ce0ac8f55-host-run-multus-certs\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.266095 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.265466 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfv7m\" (UniqueName: \"kubernetes.io/projected/13ee2110-7683-4113-a051-ff4ce0ac8f55-kube-api-access-sfv7m\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.266095 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.265489 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-run-ovn\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.266095 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.265515 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6-etc-selinux\") pod \"aws-ebs-csi-driver-node-tn4qf\" (UID: \"eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn4qf" Jul 09 11:11:37.266095 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.265540 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ab7210c-0aa5-49ab-8a57-295dad58dce8-etc-kubernetes\") pod \"tuned-wnqm7\" (UID: \"6ab7210c-0aa5-49ab-8a57-295dad58dce8\") " pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.266095 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.265563 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/13ee2110-7683-4113-a051-ff4ce0ac8f55-host-run-netns\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.266095 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.265589 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8117b6cb-dfab-462d-ae86-50a342e88a9a-tmp-dir\") pod \"node-resolver-wsfmn\" (UID: \"8117b6cb-dfab-462d-ae86-50a342e88a9a\") " pod="openshift-dns/node-resolver-wsfmn" Jul 09 11:11:37.266095 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.265611 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-etc-openvswitch\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.266095 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.265633 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-host-run-ovn-kubernetes\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.266095 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.265657 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a6b1c406-e931-4645-9fca-97ee5da046e1-env-overrides\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.266095 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.265684 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnd4b\" (UniqueName: \"kubernetes.io/projected/8207614d-56bb-42cd-a5e6-d7fd5da8dac1-kube-api-access-qnd4b\") pod \"node-ca-ppz4r\" (UID: \"8207614d-56bb-42cd-a5e6-d7fd5da8dac1\") " pod="openshift-image-registry/node-ca-ppz4r" Jul 09 11:11:37.266095 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.265709 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8117b6cb-dfab-462d-ae86-50a342e88a9a-hosts-file\") pod \"node-resolver-wsfmn\" (UID: \"8117b6cb-dfab-462d-ae86-50a342e88a9a\") " pod="openshift-dns/node-resolver-wsfmn" Jul 09 11:11:37.266095 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.265733 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6-socket-dir\") pod \"aws-ebs-csi-driver-node-tn4qf\" (UID: \"eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn4qf" Jul 09 11:11:37.266095 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.265757 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-host-kubelet\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.266095 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.265781 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-host-run-netns\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.266095 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.265803 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-systemd-units\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.266095 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.265807 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d8c8014c-ce48-40d4-94a3-b0a5cd71b2a9-iptables-alerter-script\") pod \"iptables-alerter-cbgs5\" (UID: \"d8c8014c-ce48-40d4-94a3-b0a5cd71b2a9\") " pod="openshift-network-operator/iptables-alerter-cbgs5" Jul 09 11:11:37.266095 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.265891 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/30d995da-9325-4192-9695-ad8ae483a809-cnibin\") pod \"multus-additional-cni-plugins-n2nj6\" (UID: \"30d995da-9325-4192-9695-ad8ae483a809\") " pod="openshift-multus/multus-additional-cni-plugins-n2nj6" Jul 09 11:11:37.266968 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.265953 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6-etc-selinux\") pod \"aws-ebs-csi-driver-node-tn4qf\" (UID: \"eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn4qf" Jul 09 11:11:37.266968 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:37.265993 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 11:11:37.266968 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.266049 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8117b6cb-dfab-462d-ae86-50a342e88a9a-hosts-file\") pod \"node-resolver-wsfmn\" (UID: \"8117b6cb-dfab-462d-ae86-50a342e88a9a\") " pod="openshift-dns/node-resolver-wsfmn" Jul 09 11:11:37.266968 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.266070 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/13ee2110-7683-4113-a051-ff4ce0ac8f55-multus-cni-dir\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.266968 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:37.266114 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aefb1aae-31d5-4c63-8245-3b4544e21e49-original-pull-secret podName:aefb1aae-31d5-4c63-8245-3b4544e21e49 nodeName:}" failed. No retries permitted until 2026-07-09 11:11:37.766061451 +0000 UTC m=+2.968380591 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/aefb1aae-31d5-4c63-8245-3b4544e21e49-original-pull-secret") pod "global-pull-secret-syncer-sg6nt" (UID: "aefb1aae-31d5-4c63-8245-3b4544e21e49") : object "kube-system"/"original-pull-secret" not registered Jul 09 11:11:37.266968 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:37.266148 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 11:11:37.266968 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.266152 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6-socket-dir\") pod \"aws-ebs-csi-driver-node-tn4qf\" (UID: \"eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn4qf" Jul 09 11:11:37.266968 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:37.266193 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91432ff0-fa58-460b-a6d6-3a14a9f5f295-metrics-certs podName:91432ff0-fa58-460b-a6d6-3a14a9f5f295 nodeName:}" failed. No retries permitted until 2026-07-09 11:11:37.766176512 +0000 UTC m=+2.968495651 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/91432ff0-fa58-460b-a6d6-3a14a9f5f295-metrics-certs") pod "network-metrics-daemon-6bvn2" (UID: "91432ff0-fa58-460b-a6d6-3a14a9f5f295") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 11:11:37.266968 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.266194 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/13ee2110-7683-4113-a051-ff4ce0ac8f55-os-release\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.266968 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.265999 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/13ee2110-7683-4113-a051-ff4ce0ac8f55-host-run-netns\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.266968 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.266224 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/13ee2110-7683-4113-a051-ff4ce0ac8f55-host-run-k8s-cni-cncf-io\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.266968 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.266195 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/13ee2110-7683-4113-a051-ff4ce0ac8f55-cnibin\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.266968 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.266251 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/30d995da-9325-4192-9695-ad8ae483a809-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n2nj6\" (UID: \"30d995da-9325-4192-9695-ad8ae483a809\") " pod="openshift-multus/multus-additional-cni-plugins-n2nj6" Jul 09 11:11:37.266968 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.266275 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/13ee2110-7683-4113-a051-ff4ce0ac8f55-host-run-multus-certs\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.266968 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.266280 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6-sys-fs\") pod \"aws-ebs-csi-driver-node-tn4qf\" (UID: \"eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn4qf" Jul 09 11:11:37.266968 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.266317 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6ab7210c-0aa5-49ab-8a57-295dad58dce8-etc-systemd\") pod \"tuned-wnqm7\" (UID: \"6ab7210c-0aa5-49ab-8a57-295dad58dce8\") " pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.266968 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.266321 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6-sys-fs\") pod \"aws-ebs-csi-driver-node-tn4qf\" (UID: \"eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn4qf" Jul 09 11:11:37.267836 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.266362 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6ab7210c-0aa5-49ab-8a57-295dad58dce8-lib-modules\") pod \"tuned-wnqm7\" (UID: \"6ab7210c-0aa5-49ab-8a57-295dad58dce8\") " pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.267836 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.266385 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-host-cni-netd\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.267836 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.266391 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/13ee2110-7683-4113-a051-ff4ce0ac8f55-host-run-k8s-cni-cncf-io\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.267836 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.266416 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30d995da-9325-4192-9695-ad8ae483a809-system-cni-dir\") pod \"multus-additional-cni-plugins-n2nj6\" (UID: \"30d995da-9325-4192-9695-ad8ae483a809\") " pod="openshift-multus/multus-additional-cni-plugins-n2nj6" Jul 09 11:11:37.267836 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.266442 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/13ee2110-7683-4113-a051-ff4ce0ac8f55-hostroot\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.267836 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.266459 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30d995da-9325-4192-9695-ad8ae483a809-system-cni-dir\") pod \"multus-additional-cni-plugins-n2nj6\" (UID: \"30d995da-9325-4192-9695-ad8ae483a809\") " pod="openshift-multus/multus-additional-cni-plugins-n2nj6" Jul 09 11:11:37.267836 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.266466 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/13ee2110-7683-4113-a051-ff4ce0ac8f55-etc-kubernetes\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.267836 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.266480 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8117b6cb-dfab-462d-ae86-50a342e88a9a-tmp-dir\") pod \"node-resolver-wsfmn\" (UID: \"8117b6cb-dfab-462d-ae86-50a342e88a9a\") " pod="openshift-dns/node-resolver-wsfmn" Jul 09 11:11:37.267836 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.266500 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-node-log\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.267836 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.266510 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/13ee2110-7683-4113-a051-ff4ce0ac8f55-etc-kubernetes\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.267836 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.266521 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/13ee2110-7683-4113-a051-ff4ce0ac8f55-hostroot\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.267836 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.266528 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/30d995da-9325-4192-9695-ad8ae483a809-os-release\") pod \"multus-additional-cni-plugins-n2nj6\" (UID: \"30d995da-9325-4192-9695-ad8ae483a809\") " pod="openshift-multus/multus-additional-cni-plugins-n2nj6" Jul 09 11:11:37.267836 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.266568 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/13ee2110-7683-4113-a051-ff4ce0ac8f55-host-var-lib-cni-bin\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.267836 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.266590 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/30d995da-9325-4192-9695-ad8ae483a809-os-release\") pod \"multus-additional-cni-plugins-n2nj6\" (UID: \"30d995da-9325-4192-9695-ad8ae483a809\") " pod="openshift-multus/multus-additional-cni-plugins-n2nj6" Jul 09 11:11:37.267836 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.266598 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tn4qf\" (UID: \"eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn4qf" Jul 09 11:11:37.267836 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.266625 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/13ee2110-7683-4113-a051-ff4ce0ac8f55-host-var-lib-cni-bin\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.267836 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.266626 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a6b1c406-e931-4645-9fca-97ee5da046e1-ovnkube-config\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.268734 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.266678 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tn4qf\" (UID: \"eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn4qf" Jul 09 11:11:37.268734 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.266699 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fxwt\" (UniqueName: \"kubernetes.io/projected/d8c8014c-ce48-40d4-94a3-b0a5cd71b2a9-kube-api-access-6fxwt\") pod \"iptables-alerter-cbgs5\" (UID: \"d8c8014c-ce48-40d4-94a3-b0a5cd71b2a9\") " pod="openshift-network-operator/iptables-alerter-cbgs5" Jul 09 11:11:37.268734 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.266725 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxgsx\" (UniqueName: \"kubernetes.io/projected/91432ff0-fa58-460b-a6d6-3a14a9f5f295-kube-api-access-kxgsx\") pod \"network-metrics-daemon-6bvn2\" (UID: \"91432ff0-fa58-460b-a6d6-3a14a9f5f295\") " pod="openshift-multus/network-metrics-daemon-6bvn2" Jul 09 11:11:37.268734 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.266750 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6ab7210c-0aa5-49ab-8a57-295dad58dce8-tmp\") pod \"tuned-wnqm7\" (UID: \"6ab7210c-0aa5-49ab-8a57-295dad58dce8\") " pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.268734 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.266772 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a6b1c406-e931-4645-9fca-97ee5da046e1-ovnkube-script-lib\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.268734 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.266796 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6ab7210c-0aa5-49ab-8a57-295dad58dce8-etc-modprobe-d\") pod \"tuned-wnqm7\" (UID: \"6ab7210c-0aa5-49ab-8a57-295dad58dce8\") " pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.268734 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.266820 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8207614d-56bb-42cd-a5e6-d7fd5da8dac1-serviceca\") pod \"node-ca-ppz4r\" (UID: \"8207614d-56bb-42cd-a5e6-d7fd5da8dac1\") " pod="openshift-image-registry/node-ca-ppz4r" Jul 09 11:11:37.268734 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.266847 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/13ee2110-7683-4113-a051-ff4ce0ac8f55-multus-socket-dir-parent\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.268734 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.266866 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/30d995da-9325-4192-9695-ad8ae483a809-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n2nj6\" (UID: \"30d995da-9325-4192-9695-ad8ae483a809\") " pod="openshift-multus/multus-additional-cni-plugins-n2nj6" Jul 09 11:11:37.268734 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.266871 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/13ee2110-7683-4113-a051-ff4ce0ac8f55-host-var-lib-cni-multus\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.268734 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.266908 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6ab7210c-0aa5-49ab-8a57-295dad58dce8-sys\") pod \"tuned-wnqm7\" (UID: \"6ab7210c-0aa5-49ab-8a57-295dad58dce8\") " pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.268734 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.266932 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6ab7210c-0aa5-49ab-8a57-295dad58dce8-host\") pod \"tuned-wnqm7\" (UID: \"6ab7210c-0aa5-49ab-8a57-295dad58dce8\") " pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.268734 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.266932 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/13ee2110-7683-4113-a051-ff4ce0ac8f55-multus-socket-dir-parent\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.268734 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.266970 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmbpv\" (UniqueName: \"kubernetes.io/projected/6ab7210c-0aa5-49ab-8a57-295dad58dce8-kube-api-access-nmbpv\") pod \"tuned-wnqm7\" (UID: \"6ab7210c-0aa5-49ab-8a57-295dad58dce8\") " pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.268734 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.267019 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/13ee2110-7683-4113-a051-ff4ce0ac8f55-host-var-lib-cni-multus\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.268734 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.267063 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-run-systemd\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.268734 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.267086 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6ab7210c-0aa5-49ab-8a57-295dad58dce8-etc-sysconfig\") pod \"tuned-wnqm7\" (UID: \"6ab7210c-0aa5-49ab-8a57-295dad58dce8\") " pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.269545 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.267114 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/30d995da-9325-4192-9695-ad8ae483a809-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n2nj6\" (UID: \"30d995da-9325-4192-9695-ad8ae483a809\") " pod="openshift-multus/multus-additional-cni-plugins-n2nj6" Jul 09 11:11:37.269545 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.267160 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/13ee2110-7683-4113-a051-ff4ce0ac8f55-system-cni-dir\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.269545 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.267227 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8207614d-56bb-42cd-a5e6-d7fd5da8dac1-serviceca\") pod \"node-ca-ppz4r\" (UID: \"8207614d-56bb-42cd-a5e6-d7fd5da8dac1\") " pod="openshift-image-registry/node-ca-ppz4r" Jul 09 11:11:37.269545 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.267216 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/13ee2110-7683-4113-a051-ff4ce0ac8f55-cni-binary-copy\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.269545 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.267274 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6-registration-dir\") pod \"aws-ebs-csi-driver-node-tn4qf\" (UID: \"eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn4qf" Jul 09 11:11:37.269545 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.267307 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6-device-dir\") pod \"aws-ebs-csi-driver-node-tn4qf\" (UID: \"eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn4qf" Jul 09 11:11:37.269545 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.267347 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-var-lib-openvswitch\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.269545 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.267373 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvwsc\" (UniqueName: \"kubernetes.io/projected/a6b1c406-e931-4645-9fca-97ee5da046e1-kube-api-access-cvwsc\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.269545 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.267399 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnrk8\" (UniqueName: \"kubernetes.io/projected/00faf136-9553-489b-a648-408d89197e53-kube-api-access-bnrk8\") pod \"network-check-target-4hjlg\" (UID: \"00faf136-9553-489b-a648-408d89197e53\") " pod="openshift-network-diagnostics/network-check-target-4hjlg" Jul 09 11:11:37.269545 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.267424 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6ab7210c-0aa5-49ab-8a57-295dad58dce8-var-lib-kubelet\") pod \"tuned-wnqm7\" (UID: \"6ab7210c-0aa5-49ab-8a57-295dad58dce8\") " pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.269545 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.267400 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6-registration-dir\") pod \"aws-ebs-csi-driver-node-tn4qf\" (UID: \"eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn4qf" Jul 09 11:11:37.269545 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.267458 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.269545 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.267488 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6-device-dir\") pod \"aws-ebs-csi-driver-node-tn4qf\" (UID: \"eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn4qf" Jul 09 11:11:37.269545 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.267510 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/13ee2110-7683-4113-a051-ff4ce0ac8f55-system-cni-dir\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.269545 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.267526 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcfdl\" (UniqueName: \"kubernetes.io/projected/30d995da-9325-4192-9695-ad8ae483a809-kube-api-access-tcfdl\") pod \"multus-additional-cni-plugins-n2nj6\" (UID: \"30d995da-9325-4192-9695-ad8ae483a809\") " pod="openshift-multus/multus-additional-cni-plugins-n2nj6" Jul 09 11:11:37.269545 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.267579 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/13ee2110-7683-4113-a051-ff4ce0ac8f55-host-var-lib-kubelet\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.270356 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.267584 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/30d995da-9325-4192-9695-ad8ae483a809-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n2nj6\" (UID: \"30d995da-9325-4192-9695-ad8ae483a809\") " pod="openshift-multus/multus-additional-cni-plugins-n2nj6" Jul 09 11:11:37.270356 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.267694 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6ab7210c-0aa5-49ab-8a57-295dad58dce8-etc-tuned\") pod \"tuned-wnqm7\" (UID: \"6ab7210c-0aa5-49ab-8a57-295dad58dce8\") " pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.270356 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.267719 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-host-cni-bin\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.270356 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.267763 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d8c8014c-ce48-40d4-94a3-b0a5cd71b2a9-host-slash\") pod \"iptables-alerter-cbgs5\" (UID: \"d8c8014c-ce48-40d4-94a3-b0a5cd71b2a9\") " pod="openshift-network-operator/iptables-alerter-cbgs5" Jul 09 11:11:37.270356 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.267789 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/aefb1aae-31d5-4c63-8245-3b4544e21e49-kubelet-config\") pod \"global-pull-secret-syncer-sg6nt\" (UID: \"aefb1aae-31d5-4c63-8245-3b4544e21e49\") " pod="kube-system/global-pull-secret-syncer-sg6nt" Jul 09 11:11:37.270356 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.267795 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/13ee2110-7683-4113-a051-ff4ce0ac8f55-host-var-lib-kubelet\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.270356 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.267812 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/aefb1aae-31d5-4c63-8245-3b4544e21e49-dbus\") pod \"global-pull-secret-syncer-sg6nt\" (UID: \"aefb1aae-31d5-4c63-8245-3b4544e21e49\") " pod="kube-system/global-pull-secret-syncer-sg6nt" Jul 09 11:11:37.270356 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.267838 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/13ee2110-7683-4113-a051-ff4ce0ac8f55-cni-binary-copy\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.270356 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.267844 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/13ee2110-7683-4113-a051-ff4ce0ac8f55-multus-conf-dir\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.270356 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.267873 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/aefb1aae-31d5-4c63-8245-3b4544e21e49-kubelet-config\") pod \"global-pull-secret-syncer-sg6nt\" (UID: \"aefb1aae-31d5-4c63-8245-3b4544e21e49\") " pod="kube-system/global-pull-secret-syncer-sg6nt" Jul 09 11:11:37.270356 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.267878 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/13ee2110-7683-4113-a051-ff4ce0ac8f55-multus-conf-dir\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.270356 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.267888 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/47077616-7ca2-4b7d-ba31-92d393ab7439-konnectivity-ca\") pod \"konnectivity-agent-lpprd\" (UID: \"47077616-7ca2-4b7d-ba31-92d393ab7439\") " pod="kube-system/konnectivity-agent-lpprd" Jul 09 11:11:37.270356 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.267925 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/30d995da-9325-4192-9695-ad8ae483a809-cni-binary-copy\") pod \"multus-additional-cni-plugins-n2nj6\" (UID: \"30d995da-9325-4192-9695-ad8ae483a809\") " pod="openshift-multus/multus-additional-cni-plugins-n2nj6" Jul 09 11:11:37.270356 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.267928 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/aefb1aae-31d5-4c63-8245-3b4544e21e49-dbus\") pod \"global-pull-secret-syncer-sg6nt\" (UID: \"aefb1aae-31d5-4c63-8245-3b4544e21e49\") " pod="kube-system/global-pull-secret-syncer-sg6nt" Jul 09 11:11:37.270356 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.267960 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/13ee2110-7683-4113-a051-ff4ce0ac8f55-multus-daemon-config\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.270356 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.267967 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d8c8014c-ce48-40d4-94a3-b0a5cd71b2a9-host-slash\") pod \"iptables-alerter-cbgs5\" (UID: \"d8c8014c-ce48-40d4-94a3-b0a5cd71b2a9\") " pod="openshift-network-operator/iptables-alerter-cbgs5" Jul 09 11:11:37.270356 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.267994 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4465f\" (UniqueName: \"kubernetes.io/projected/eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6-kube-api-access-4465f\") pod \"aws-ebs-csi-driver-node-tn4qf\" (UID: \"eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn4qf" Jul 09 11:11:37.271121 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.268020 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6ab7210c-0aa5-49ab-8a57-295dad58dce8-run\") pod \"tuned-wnqm7\" (UID: \"6ab7210c-0aa5-49ab-8a57-295dad58dce8\") " pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.271121 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.268043 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-run-openvswitch\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.271121 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.268066 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8207614d-56bb-42cd-a5e6-d7fd5da8dac1-host\") pod \"node-ca-ppz4r\" (UID: \"8207614d-56bb-42cd-a5e6-d7fd5da8dac1\") " pod="openshift-image-registry/node-ca-ppz4r" Jul 09 11:11:37.271121 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.268092 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/30d995da-9325-4192-9695-ad8ae483a809-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n2nj6\" (UID: \"30d995da-9325-4192-9695-ad8ae483a809\") " pod="openshift-multus/multus-additional-cni-plugins-n2nj6" Jul 09 11:11:37.271121 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.268122 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxkbf\" (UniqueName: \"kubernetes.io/projected/8117b6cb-dfab-462d-ae86-50a342e88a9a-kube-api-access-jxkbf\") pod \"node-resolver-wsfmn\" (UID: \"8117b6cb-dfab-462d-ae86-50a342e88a9a\") " pod="openshift-dns/node-resolver-wsfmn" Jul 09 11:11:37.271121 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.268146 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-host-slash\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.271121 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.268171 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-log-socket\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.271121 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.268437 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/13ee2110-7683-4113-a051-ff4ce0ac8f55-multus-daemon-config\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.271121 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.268457 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/30d995da-9325-4192-9695-ad8ae483a809-cni-binary-copy\") pod \"multus-additional-cni-plugins-n2nj6\" (UID: \"30d995da-9325-4192-9695-ad8ae483a809\") " pod="openshift-multus/multus-additional-cni-plugins-n2nj6" Jul 09 11:11:37.271121 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.268478 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8207614d-56bb-42cd-a5e6-d7fd5da8dac1-host\") pod \"node-ca-ppz4r\" (UID: \"8207614d-56bb-42cd-a5e6-d7fd5da8dac1\") " pod="openshift-image-registry/node-ca-ppz4r" Jul 09 11:11:37.271121 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.268590 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/30d995da-9325-4192-9695-ad8ae483a809-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n2nj6\" (UID: \"30d995da-9325-4192-9695-ad8ae483a809\") " pod="openshift-multus/multus-additional-cni-plugins-n2nj6" Jul 09 11:11:37.272918 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.272894 2550 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 09 11:11:37.278531 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.278465 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcfdl\" (UniqueName: \"kubernetes.io/projected/30d995da-9325-4192-9695-ad8ae483a809-kube-api-access-tcfdl\") pod \"multus-additional-cni-plugins-n2nj6\" (UID: \"30d995da-9325-4192-9695-ad8ae483a809\") " pod="openshift-multus/multus-additional-cni-plugins-n2nj6" Jul 09 11:11:37.279882 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.279857 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxkbf\" (UniqueName: \"kubernetes.io/projected/8117b6cb-dfab-462d-ae86-50a342e88a9a-kube-api-access-jxkbf\") pod \"node-resolver-wsfmn\" (UID: \"8117b6cb-dfab-462d-ae86-50a342e88a9a\") " pod="openshift-dns/node-resolver-wsfmn" Jul 09 11:11:37.280562 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.280510 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxgsx\" (UniqueName: \"kubernetes.io/projected/91432ff0-fa58-460b-a6d6-3a14a9f5f295-kube-api-access-kxgsx\") pod \"network-metrics-daemon-6bvn2\" (UID: \"91432ff0-fa58-460b-a6d6-3a14a9f5f295\") " pod="openshift-multus/network-metrics-daemon-6bvn2" Jul 09 11:11:37.281090 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.281052 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfv7m\" (UniqueName: \"kubernetes.io/projected/13ee2110-7683-4113-a051-ff4ce0ac8f55-kube-api-access-sfv7m\") pod \"multus-gfmtp\" (UID: \"13ee2110-7683-4113-a051-ff4ce0ac8f55\") " pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.281616 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.281581 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnd4b\" (UniqueName: \"kubernetes.io/projected/8207614d-56bb-42cd-a5e6-d7fd5da8dac1-kube-api-access-qnd4b\") pod \"node-ca-ppz4r\" (UID: \"8207614d-56bb-42cd-a5e6-d7fd5da8dac1\") " pod="openshift-image-registry/node-ca-ppz4r" Jul 09 11:11:37.282127 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.282087 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fxwt\" (UniqueName: \"kubernetes.io/projected/d8c8014c-ce48-40d4-94a3-b0a5cd71b2a9-kube-api-access-6fxwt\") pod \"iptables-alerter-cbgs5\" (UID: \"d8c8014c-ce48-40d4-94a3-b0a5cd71b2a9\") " pod="openshift-network-operator/iptables-alerter-cbgs5" Jul 09 11:11:37.284960 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.284918 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4465f\" (UniqueName: \"kubernetes.io/projected/eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6-kube-api-access-4465f\") pod \"aws-ebs-csi-driver-node-tn4qf\" (UID: \"eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn4qf" Jul 09 11:11:37.298015 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.297971 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-142.ec2.internal" event={"ID":"7dd8c39790ec51df8183e1a3a4cad09b","Type":"ContainerStarted","Data":"e4763abb3fba60dbc139058c1679b867df13ff50708e7d102693775420b1462b"} Jul 09 11:11:37.299220 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.299192 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-142.ec2.internal" event={"ID":"c0a0fd4f8c4f29eccb1f0904d07ef791","Type":"ContainerStarted","Data":"ea7c7ab18132568e883cefcb4046292ee4e544a0cd4ddc1928ed74a5dc9a1b12"} Jul 09 11:11:37.368475 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.368440 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a6b1c406-e931-4645-9fca-97ee5da046e1-ovnkube-config\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.368475 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.368486 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6ab7210c-0aa5-49ab-8a57-295dad58dce8-tmp\") pod \"tuned-wnqm7\" (UID: \"6ab7210c-0aa5-49ab-8a57-295dad58dce8\") " pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.368747 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.368532 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a6b1c406-e931-4645-9fca-97ee5da046e1-ovnkube-script-lib\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.368747 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.368555 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6ab7210c-0aa5-49ab-8a57-295dad58dce8-etc-modprobe-d\") pod \"tuned-wnqm7\" (UID: \"6ab7210c-0aa5-49ab-8a57-295dad58dce8\") " pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.368747 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.368579 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6ab7210c-0aa5-49ab-8a57-295dad58dce8-sys\") pod \"tuned-wnqm7\" (UID: \"6ab7210c-0aa5-49ab-8a57-295dad58dce8\") " pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.368747 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.368593 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6ab7210c-0aa5-49ab-8a57-295dad58dce8-host\") pod \"tuned-wnqm7\" (UID: \"6ab7210c-0aa5-49ab-8a57-295dad58dce8\") " pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.368747 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.368607 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmbpv\" (UniqueName: \"kubernetes.io/projected/6ab7210c-0aa5-49ab-8a57-295dad58dce8-kube-api-access-nmbpv\") pod \"tuned-wnqm7\" (UID: \"6ab7210c-0aa5-49ab-8a57-295dad58dce8\") " pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.368747 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.368623 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-run-systemd\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.368747 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.368643 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6ab7210c-0aa5-49ab-8a57-295dad58dce8-etc-sysconfig\") pod \"tuned-wnqm7\" (UID: \"6ab7210c-0aa5-49ab-8a57-295dad58dce8\") " pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.368747 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.368669 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-var-lib-openvswitch\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.368747 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.368738 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-var-lib-openvswitch\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.369164 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.368749 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6ab7210c-0aa5-49ab-8a57-295dad58dce8-sys\") pod \"tuned-wnqm7\" (UID: \"6ab7210c-0aa5-49ab-8a57-295dad58dce8\") " pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.369164 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.368814 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-run-systemd\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.369164 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.368877 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6ab7210c-0aa5-49ab-8a57-295dad58dce8-host\") pod \"tuned-wnqm7\" (UID: \"6ab7210c-0aa5-49ab-8a57-295dad58dce8\") " pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.369164 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.368883 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6ab7210c-0aa5-49ab-8a57-295dad58dce8-etc-modprobe-d\") pod \"tuned-wnqm7\" (UID: \"6ab7210c-0aa5-49ab-8a57-295dad58dce8\") " pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.369164 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.368930 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6ab7210c-0aa5-49ab-8a57-295dad58dce8-etc-sysconfig\") pod \"tuned-wnqm7\" (UID: \"6ab7210c-0aa5-49ab-8a57-295dad58dce8\") " pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.369164 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.369040 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvwsc\" (UniqueName: \"kubernetes.io/projected/a6b1c406-e931-4645-9fca-97ee5da046e1-kube-api-access-cvwsc\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.369164 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.369075 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnrk8\" (UniqueName: \"kubernetes.io/projected/00faf136-9553-489b-a648-408d89197e53-kube-api-access-bnrk8\") pod \"network-check-target-4hjlg\" (UID: \"00faf136-9553-489b-a648-408d89197e53\") " pod="openshift-network-diagnostics/network-check-target-4hjlg" Jul 09 11:11:37.369164 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.369100 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6ab7210c-0aa5-49ab-8a57-295dad58dce8-var-lib-kubelet\") pod \"tuned-wnqm7\" (UID: \"6ab7210c-0aa5-49ab-8a57-295dad58dce8\") " pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.369164 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.369127 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.369164 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.369132 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a6b1c406-e931-4645-9fca-97ee5da046e1-ovnkube-config\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.369164 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.369157 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6ab7210c-0aa5-49ab-8a57-295dad58dce8-etc-tuned\") pod \"tuned-wnqm7\" (UID: \"6ab7210c-0aa5-49ab-8a57-295dad58dce8\") " pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.369683 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.369205 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.369683 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.369243 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-host-cni-bin\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.369683 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.369274 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/47077616-7ca2-4b7d-ba31-92d393ab7439-konnectivity-ca\") pod \"konnectivity-agent-lpprd\" (UID: \"47077616-7ca2-4b7d-ba31-92d393ab7439\") " pod="kube-system/konnectivity-agent-lpprd" Jul 09 11:11:37.369683 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.369290 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6ab7210c-0aa5-49ab-8a57-295dad58dce8-var-lib-kubelet\") pod \"tuned-wnqm7\" (UID: \"6ab7210c-0aa5-49ab-8a57-295dad58dce8\") " pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.369683 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.369302 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6ab7210c-0aa5-49ab-8a57-295dad58dce8-run\") pod \"tuned-wnqm7\" (UID: \"6ab7210c-0aa5-49ab-8a57-295dad58dce8\") " pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.369683 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.369358 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-host-cni-bin\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.369683 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.369372 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6ab7210c-0aa5-49ab-8a57-295dad58dce8-run\") pod \"tuned-wnqm7\" (UID: \"6ab7210c-0aa5-49ab-8a57-295dad58dce8\") " pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.369683 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.369389 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-run-openvswitch\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.369683 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.369423 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-host-slash\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.369683 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.369428 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-run-openvswitch\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.369683 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.369430 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a6b1c406-e931-4645-9fca-97ee5da046e1-ovnkube-script-lib\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.369683 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.369448 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-log-socket\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.369683 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.369484 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-host-slash\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.369683 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.369496 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6ab7210c-0aa5-49ab-8a57-295dad58dce8-etc-sysctl-conf\") pod \"tuned-wnqm7\" (UID: \"6ab7210c-0aa5-49ab-8a57-295dad58dce8\") " pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.369683 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.369521 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a6b1c406-e931-4645-9fca-97ee5da046e1-ovn-node-metrics-cert\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.369683 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.369547 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/47077616-7ca2-4b7d-ba31-92d393ab7439-agent-certs\") pod \"konnectivity-agent-lpprd\" (UID: \"47077616-7ca2-4b7d-ba31-92d393ab7439\") " pod="kube-system/konnectivity-agent-lpprd" Jul 09 11:11:37.369683 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.369573 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6ab7210c-0aa5-49ab-8a57-295dad58dce8-etc-sysctl-d\") pod \"tuned-wnqm7\" (UID: \"6ab7210c-0aa5-49ab-8a57-295dad58dce8\") " pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.369683 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.369613 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-run-ovn\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.370542 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.369622 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6ab7210c-0aa5-49ab-8a57-295dad58dce8-etc-sysctl-conf\") pod \"tuned-wnqm7\" (UID: \"6ab7210c-0aa5-49ab-8a57-295dad58dce8\") " pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.370542 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.369637 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ab7210c-0aa5-49ab-8a57-295dad58dce8-etc-kubernetes\") pod \"tuned-wnqm7\" (UID: \"6ab7210c-0aa5-49ab-8a57-295dad58dce8\") " pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.370542 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.369523 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-log-socket\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.370542 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.369666 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-etc-openvswitch\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.370542 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.369690 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-host-run-ovn-kubernetes\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.370542 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.369697 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-run-ovn\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.370542 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.369716 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a6b1c406-e931-4645-9fca-97ee5da046e1-env-overrides\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.370542 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.369746 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-host-kubelet\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.370542 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.369771 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-host-run-netns\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.370542 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.369802 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6ab7210c-0aa5-49ab-8a57-295dad58dce8-etc-sysctl-d\") pod \"tuned-wnqm7\" (UID: \"6ab7210c-0aa5-49ab-8a57-295dad58dce8\") " pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.370542 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.369809 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-systemd-units\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.370542 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.369846 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6ab7210c-0aa5-49ab-8a57-295dad58dce8-etc-systemd\") pod \"tuned-wnqm7\" (UID: \"6ab7210c-0aa5-49ab-8a57-295dad58dce8\") " pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.370542 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.369869 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6ab7210c-0aa5-49ab-8a57-295dad58dce8-lib-modules\") pod \"tuned-wnqm7\" (UID: \"6ab7210c-0aa5-49ab-8a57-295dad58dce8\") " pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.370542 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.369893 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-host-cni-netd\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.370542 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.369913 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/47077616-7ca2-4b7d-ba31-92d393ab7439-konnectivity-ca\") pod \"konnectivity-agent-lpprd\" (UID: \"47077616-7ca2-4b7d-ba31-92d393ab7439\") " pod="kube-system/konnectivity-agent-lpprd" Jul 09 11:11:37.370542 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.369927 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-node-log\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.370542 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.369973 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-systemd-units\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.370542 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.370002 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-node-log\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.371464 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.370042 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-host-kubelet\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.371464 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.370086 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-host-run-netns\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.371464 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.370128 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-etc-openvswitch\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.371464 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.370235 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6ab7210c-0aa5-49ab-8a57-295dad58dce8-lib-modules\") pod \"tuned-wnqm7\" (UID: \"6ab7210c-0aa5-49ab-8a57-295dad58dce8\") " pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.371464 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.370269 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-host-run-ovn-kubernetes\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.371464 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.370352 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6ab7210c-0aa5-49ab-8a57-295dad58dce8-etc-systemd\") pod \"tuned-wnqm7\" (UID: \"6ab7210c-0aa5-49ab-8a57-295dad58dce8\") " pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.371464 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.369871 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ab7210c-0aa5-49ab-8a57-295dad58dce8-etc-kubernetes\") pod \"tuned-wnqm7\" (UID: \"6ab7210c-0aa5-49ab-8a57-295dad58dce8\") " pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.371464 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.370390 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a6b1c406-e931-4645-9fca-97ee5da046e1-env-overrides\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.371464 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.370408 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a6b1c406-e931-4645-9fca-97ee5da046e1-host-cni-netd\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.372448 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.372428 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a6b1c406-e931-4645-9fca-97ee5da046e1-ovn-node-metrics-cert\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.372547 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.372458 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6ab7210c-0aa5-49ab-8a57-295dad58dce8-tmp\") pod \"tuned-wnqm7\" (UID: \"6ab7210c-0aa5-49ab-8a57-295dad58dce8\") " pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.372639 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.372615 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6ab7210c-0aa5-49ab-8a57-295dad58dce8-etc-tuned\") pod \"tuned-wnqm7\" (UID: \"6ab7210c-0aa5-49ab-8a57-295dad58dce8\") " pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.372975 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.372953 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/47077616-7ca2-4b7d-ba31-92d393ab7439-agent-certs\") pod \"konnectivity-agent-lpprd\" (UID: \"47077616-7ca2-4b7d-ba31-92d393ab7439\") " pod="kube-system/konnectivity-agent-lpprd" Jul 09 11:11:37.377413 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:37.377307 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 11:11:37.377413 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:37.377348 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 11:11:37.377413 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:37.377363 2550 projected.go:196] Error preparing data for projected volume kube-api-access-bnrk8 for pod openshift-network-diagnostics/network-check-target-4hjlg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 11:11:37.377629 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:37.377439 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/00faf136-9553-489b-a648-408d89197e53-kube-api-access-bnrk8 podName:00faf136-9553-489b-a648-408d89197e53 nodeName:}" failed. No retries permitted until 2026-07-09 11:11:37.877420332 +0000 UTC m=+3.079739475 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-bnrk8" (UniqueName: "kubernetes.io/projected/00faf136-9553-489b-a648-408d89197e53-kube-api-access-bnrk8") pod "network-check-target-4hjlg" (UID: "00faf136-9553-489b-a648-408d89197e53") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 11:11:37.380010 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.379957 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvwsc\" (UniqueName: \"kubernetes.io/projected/a6b1c406-e931-4645-9fca-97ee5da046e1-kube-api-access-cvwsc\") pod \"ovnkube-node-fr9h4\" (UID: \"a6b1c406-e931-4645-9fca-97ee5da046e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.380408 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.380385 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmbpv\" (UniqueName: \"kubernetes.io/projected/6ab7210c-0aa5-49ab-8a57-295dad58dce8-kube-api-access-nmbpv\") pod \"tuned-wnqm7\" (UID: \"6ab7210c-0aa5-49ab-8a57-295dad58dce8\") " pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.460144 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.460063 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cbgs5" Jul 09 11:11:37.468161 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.468132 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-n2nj6" Jul 09 11:11:37.480945 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.480923 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wsfmn" Jul 09 11:11:37.486725 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.486705 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn4qf" Jul 09 11:11:37.492477 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.492451 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ppz4r" Jul 09 11:11:37.499160 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.499131 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gfmtp" Jul 09 11:11:37.505941 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.505921 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lpprd" Jul 09 11:11:37.512895 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.512874 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:11:37.513939 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.513921 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" Jul 09 11:11:37.773183 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.773103 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91432ff0-fa58-460b-a6d6-3a14a9f5f295-metrics-certs\") pod \"network-metrics-daemon-6bvn2\" (UID: \"91432ff0-fa58-460b-a6d6-3a14a9f5f295\") " pod="openshift-multus/network-metrics-daemon-6bvn2" Jul 09 11:11:37.773183 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.773139 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aefb1aae-31d5-4c63-8245-3b4544e21e49-original-pull-secret\") pod \"global-pull-secret-syncer-sg6nt\" (UID: \"aefb1aae-31d5-4c63-8245-3b4544e21e49\") " pod="kube-system/global-pull-secret-syncer-sg6nt" Jul 09 11:11:37.773389 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:37.773264 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 11:11:37.773389 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:37.773312 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aefb1aae-31d5-4c63-8245-3b4544e21e49-original-pull-secret podName:aefb1aae-31d5-4c63-8245-3b4544e21e49 nodeName:}" failed. No retries permitted until 2026-07-09 11:11:38.773299673 +0000 UTC m=+3.975618799 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/aefb1aae-31d5-4c63-8245-3b4544e21e49-original-pull-secret") pod "global-pull-secret-syncer-sg6nt" (UID: "aefb1aae-31d5-4c63-8245-3b4544e21e49") : object "kube-system"/"original-pull-secret" not registered Jul 09 11:11:37.773389 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:37.773264 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 11:11:37.773389 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:37.773374 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91432ff0-fa58-460b-a6d6-3a14a9f5f295-metrics-certs podName:91432ff0-fa58-460b-a6d6-3a14a9f5f295 nodeName:}" failed. No retries permitted until 2026-07-09 11:11:38.773363685 +0000 UTC m=+3.975682813 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/91432ff0-fa58-460b-a6d6-3a14a9f5f295-metrics-certs") pod "network-metrics-daemon-6bvn2" (UID: "91432ff0-fa58-460b-a6d6-3a14a9f5f295") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 11:11:37.974064 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:37.974032 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnrk8\" (UniqueName: \"kubernetes.io/projected/00faf136-9553-489b-a648-408d89197e53-kube-api-access-bnrk8\") pod \"network-check-target-4hjlg\" (UID: \"00faf136-9553-489b-a648-408d89197e53\") " pod="openshift-network-diagnostics/network-check-target-4hjlg" Jul 09 11:11:37.974230 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:37.974184 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 11:11:37.974230 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:37.974210 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 11:11:37.974230 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:37.974222 2550 projected.go:196] Error preparing data for projected volume kube-api-access-bnrk8 for pod openshift-network-diagnostics/network-check-target-4hjlg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 11:11:37.974364 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:37.974278 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/00faf136-9553-489b-a648-408d89197e53-kube-api-access-bnrk8 podName:00faf136-9553-489b-a648-408d89197e53 nodeName:}" failed. No retries permitted until 2026-07-09 11:11:38.974262469 +0000 UTC m=+4.176581596 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-bnrk8" (UniqueName: "kubernetes.io/projected/00faf136-9553-489b-a648-408d89197e53-kube-api-access-bnrk8") pod "network-check-target-4hjlg" (UID: "00faf136-9553-489b-a648-408d89197e53") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 11:11:38.177719 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:38.177695 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb11ab7b_3fa0_40be_81bc_ba4ed78b69d6.slice/crio-df36dd476edb1c814499c28d2a68b89428b98ce85f7a4fc859a6f953eb2aebfb WatchSource:0}: Error finding container df36dd476edb1c814499c28d2a68b89428b98ce85f7a4fc859a6f953eb2aebfb: Status 404 returned error can't find the container with id df36dd476edb1c814499c28d2a68b89428b98ce85f7a4fc859a6f953eb2aebfb Jul 09 11:11:38.188809 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:38.188787 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47077616_7ca2_4b7d_ba31_92d393ab7439.slice/crio-e87e310f271a4330325a9ebce66e91fef5ede59baca0d43a7b9d87434e27298f WatchSource:0}: Error finding container e87e310f271a4330325a9ebce66e91fef5ede59baca0d43a7b9d87434e27298f: Status 404 returned error can't find the container with id e87e310f271a4330325a9ebce66e91fef5ede59baca0d43a7b9d87434e27298f Jul 09 11:11:38.193271 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:38.193242 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8117b6cb_dfab_462d_ae86_50a342e88a9a.slice/crio-0b5d39ea5a8115bd7d20ca9872d096882055099977ff2c16e280afb882248cdc WatchSource:0}: Error finding container 0b5d39ea5a8115bd7d20ca9872d096882055099977ff2c16e280afb882248cdc: Status 404 returned error can't find the container with id 0b5d39ea5a8115bd7d20ca9872d096882055099977ff2c16e280afb882248cdc Jul 09 11:11:38.194619 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:38.194511 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6b1c406_e931_4645_9fca_97ee5da046e1.slice/crio-73fa54997ad2e19e5610d5f35a2944a242b2fd0114de4e3146516755ec13ce81 WatchSource:0}: Error finding container 73fa54997ad2e19e5610d5f35a2944a242b2fd0114de4e3146516755ec13ce81: Status 404 returned error can't find the container with id 73fa54997ad2e19e5610d5f35a2944a242b2fd0114de4e3146516755ec13ce81 Jul 09 11:11:38.195484 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:38.195460 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30d995da_9325_4192_9695_ad8ae483a809.slice/crio-ff3597df763e650f373d725e1cc5e92e25aae6e5483faba0f259118fbb4803f2 WatchSource:0}: Error finding container ff3597df763e650f373d725e1cc5e92e25aae6e5483faba0f259118fbb4803f2: Status 404 returned error can't find the container with id ff3597df763e650f373d725e1cc5e92e25aae6e5483faba0f259118fbb4803f2 Jul 09 11:11:38.196203 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:38.196181 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ab7210c_0aa5_49ab_8a57_295dad58dce8.slice/crio-9446529f46f963a5f4f6d144ad168bb586e5657360a0161bd230d340a5812b85 WatchSource:0}: Error finding container 9446529f46f963a5f4f6d144ad168bb586e5657360a0161bd230d340a5812b85: Status 404 returned error can't find the container with id 9446529f46f963a5f4f6d144ad168bb586e5657360a0161bd230d340a5812b85 Jul 09 11:11:38.196552 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:38.196483 2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 11:06:37 +0000 UTC" deadline="2028-03-29 05:26:31.335373764 +0000 UTC" Jul 09 11:11:38.196552 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:38.196544 2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15090h14m53.138834231s" Jul 09 11:11:38.197592 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:38.197513 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13ee2110_7683_4113_a051_ff4ce0ac8f55.slice/crio-4eab58e50a45a6e5abb5930d1377d487b71d6b0d3ba25b47599088dadc36eef3 WatchSource:0}: Error finding container 4eab58e50a45a6e5abb5930d1377d487b71d6b0d3ba25b47599088dadc36eef3: Status 404 returned error can't find the container with id 4eab58e50a45a6e5abb5930d1377d487b71d6b0d3ba25b47599088dadc36eef3 Jul 09 11:11:38.198808 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:38.198708 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8207614d_56bb_42cd_a5e6_d7fd5da8dac1.slice/crio-b4fb92cf5c1e582400ebdb73caae0361d8cb3be63b8844191ba8b082bc425235 WatchSource:0}: Error finding container b4fb92cf5c1e582400ebdb73caae0361d8cb3be63b8844191ba8b082bc425235: Status 404 returned error can't find the container with id b4fb92cf5c1e582400ebdb73caae0361d8cb3be63b8844191ba8b082bc425235 Jul 09 11:11:38.199497 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:11:38.199453 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8c8014c_ce48_40d4_94a3_b0a5cd71b2a9.slice/crio-9ef00551d34872a2e3727303e578b3903f66d8f90405af27634139b0299d2285 WatchSource:0}: Error finding container 9ef00551d34872a2e3727303e578b3903f66d8f90405af27634139b0299d2285: Status 404 returned error can't find the container with id 9ef00551d34872a2e3727303e578b3903f66d8f90405af27634139b0299d2285 Jul 09 11:11:38.294111 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:38.293927 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sg6nt" Jul 09 11:11:38.294240 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:38.293962 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bvn2" Jul 09 11:11:38.294240 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:38.294219 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sg6nt" podUID="aefb1aae-31d5-4c63-8245-3b4544e21e49" Jul 09 11:11:38.294387 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:38.294359 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bvn2" podUID="91432ff0-fa58-460b-a6d6-3a14a9f5f295" Jul 09 11:11:38.305468 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:38.305446 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn4qf" event={"ID":"eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6","Type":"ContainerStarted","Data":"df36dd476edb1c814499c28d2a68b89428b98ce85f7a4fc859a6f953eb2aebfb"} Jul 09 11:11:38.307698 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:38.307678 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ppz4r" event={"ID":"8207614d-56bb-42cd-a5e6-d7fd5da8dac1","Type":"ContainerStarted","Data":"b4fb92cf5c1e582400ebdb73caae0361d8cb3be63b8844191ba8b082bc425235"} Jul 09 11:11:38.311839 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:38.311820 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gfmtp" event={"ID":"13ee2110-7683-4113-a051-ff4ce0ac8f55","Type":"ContainerStarted","Data":"4eab58e50a45a6e5abb5930d1377d487b71d6b0d3ba25b47599088dadc36eef3"} Jul 09 11:11:38.312837 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:38.312784 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n2nj6" event={"ID":"30d995da-9325-4192-9695-ad8ae483a809","Type":"ContainerStarted","Data":"ff3597df763e650f373d725e1cc5e92e25aae6e5483faba0f259118fbb4803f2"} Jul 09 11:11:38.314193 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:38.314174 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" event={"ID":"a6b1c406-e931-4645-9fca-97ee5da046e1","Type":"ContainerStarted","Data":"73fa54997ad2e19e5610d5f35a2944a242b2fd0114de4e3146516755ec13ce81"} Jul 09 11:11:38.318273 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:38.318244 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-142.ec2.internal" event={"ID":"c0a0fd4f8c4f29eccb1f0904d07ef791","Type":"ContainerStarted","Data":"6fd92441e3f62c9a423b7e620f87f02a31c93c2099ff1f58d91e72ad129bd112"} Jul 09 11:11:38.319774 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:38.319744 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cbgs5" event={"ID":"d8c8014c-ce48-40d4-94a3-b0a5cd71b2a9","Type":"ContainerStarted","Data":"9ef00551d34872a2e3727303e578b3903f66d8f90405af27634139b0299d2285"} Jul 09 11:11:38.351148 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:38.351089 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-142.ec2.internal" podStartSLOduration=2.351069402 podStartE2EDuration="2.351069402s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 11:11:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:11:38.350266338 +0000 UTC m=+3.552585516" watchObservedRunningTime="2026-07-09 11:11:38.351069402 +0000 UTC m=+3.553388551" Jul 09 11:11:38.351307 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:38.351289 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" event={"ID":"6ab7210c-0aa5-49ab-8a57-295dad58dce8","Type":"ContainerStarted","Data":"9446529f46f963a5f4f6d144ad168bb586e5657360a0161bd230d340a5812b85"} Jul 09 11:11:38.352688 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:38.352653 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wsfmn" event={"ID":"8117b6cb-dfab-462d-ae86-50a342e88a9a","Type":"ContainerStarted","Data":"0b5d39ea5a8115bd7d20ca9872d096882055099977ff2c16e280afb882248cdc"} Jul 09 11:11:38.353862 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:38.353844 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lpprd" event={"ID":"47077616-7ca2-4b7d-ba31-92d393ab7439","Type":"ContainerStarted","Data":"e87e310f271a4330325a9ebce66e91fef5ede59baca0d43a7b9d87434e27298f"} Jul 09 11:11:38.781919 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:38.781116 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91432ff0-fa58-460b-a6d6-3a14a9f5f295-metrics-certs\") pod \"network-metrics-daemon-6bvn2\" (UID: \"91432ff0-fa58-460b-a6d6-3a14a9f5f295\") " pod="openshift-multus/network-metrics-daemon-6bvn2" Jul 09 11:11:38.781919 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:38.781166 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aefb1aae-31d5-4c63-8245-3b4544e21e49-original-pull-secret\") pod \"global-pull-secret-syncer-sg6nt\" (UID: \"aefb1aae-31d5-4c63-8245-3b4544e21e49\") " pod="kube-system/global-pull-secret-syncer-sg6nt" Jul 09 11:11:38.781919 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:38.781302 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 11:11:38.781919 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:38.781383 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aefb1aae-31d5-4c63-8245-3b4544e21e49-original-pull-secret podName:aefb1aae-31d5-4c63-8245-3b4544e21e49 nodeName:}" failed. No retries permitted until 2026-07-09 11:11:40.781363265 +0000 UTC m=+5.983682397 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/aefb1aae-31d5-4c63-8245-3b4544e21e49-original-pull-secret") pod "global-pull-secret-syncer-sg6nt" (UID: "aefb1aae-31d5-4c63-8245-3b4544e21e49") : object "kube-system"/"original-pull-secret" not registered Jul 09 11:11:38.781919 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:38.781775 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 11:11:38.781919 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:38.781835 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91432ff0-fa58-460b-a6d6-3a14a9f5f295-metrics-certs podName:91432ff0-fa58-460b-a6d6-3a14a9f5f295 nodeName:}" failed. No retries permitted until 2026-07-09 11:11:40.781817946 +0000 UTC m=+5.984137078 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/91432ff0-fa58-460b-a6d6-3a14a9f5f295-metrics-certs") pod "network-metrics-daemon-6bvn2" (UID: "91432ff0-fa58-460b-a6d6-3a14a9f5f295") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 11:11:38.983162 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:38.983121 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnrk8\" (UniqueName: \"kubernetes.io/projected/00faf136-9553-489b-a648-408d89197e53-kube-api-access-bnrk8\") pod \"network-check-target-4hjlg\" (UID: \"00faf136-9553-489b-a648-408d89197e53\") " pod="openshift-network-diagnostics/network-check-target-4hjlg" Jul 09 11:11:38.983353 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:38.983290 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 11:11:38.983353 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:38.983311 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 11:11:38.983353 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:38.983323 2550 projected.go:196] Error preparing data for projected volume kube-api-access-bnrk8 for pod openshift-network-diagnostics/network-check-target-4hjlg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 11:11:38.983546 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:38.983400 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/00faf136-9553-489b-a648-408d89197e53-kube-api-access-bnrk8 podName:00faf136-9553-489b-a648-408d89197e53 nodeName:}" failed. No retries permitted until 2026-07-09 11:11:40.983378482 +0000 UTC m=+6.185697614 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-bnrk8" (UniqueName: "kubernetes.io/projected/00faf136-9553-489b-a648-408d89197e53-kube-api-access-bnrk8") pod "network-check-target-4hjlg" (UID: "00faf136-9553-489b-a648-408d89197e53") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 11:11:39.197454 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:39.197385 2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 11:06:37 +0000 UTC" deadline="2028-04-01 18:27:38.345135006 +0000 UTC" Jul 09 11:11:39.197454 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:39.197450 2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15175h15m59.147690648s" Jul 09 11:11:39.296085 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:39.295614 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4hjlg" Jul 09 11:11:39.296085 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:39.295741 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4hjlg" podUID="00faf136-9553-489b-a648-408d89197e53" Jul 09 11:11:39.413339 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:39.413285 2550 generic.go:356] "Generic (PLEG): container finished" podID="7dd8c39790ec51df8183e1a3a4cad09b" containerID="3ce4da296074ac69c7e42d8b4de21f80b79bad150b2569142347c179687e3b04" exitCode=0 Jul 09 11:11:39.413823 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:39.413799 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-142.ec2.internal" event={"ID":"7dd8c39790ec51df8183e1a3a4cad09b","Type":"ContainerDied","Data":"3ce4da296074ac69c7e42d8b4de21f80b79bad150b2569142347c179687e3b04"} Jul 09 11:11:40.294408 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:40.294374 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bvn2" Jul 09 11:11:40.294944 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:40.294521 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bvn2" podUID="91432ff0-fa58-460b-a6d6-3a14a9f5f295" Jul 09 11:11:40.294944 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:40.294583 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sg6nt" Jul 09 11:11:40.294944 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:40.294691 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sg6nt" podUID="aefb1aae-31d5-4c63-8245-3b4544e21e49" Jul 09 11:11:40.422404 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:40.422208 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-142.ec2.internal" event={"ID":"7dd8c39790ec51df8183e1a3a4cad09b","Type":"ContainerStarted","Data":"16b2c82886ead4b4d45c6eb25201deff51a4e26c2eac35eb2a19c0c54752f82d"} Jul 09 11:11:40.442345 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:40.442080 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-142.ec2.internal" podStartSLOduration=4.442054374 podStartE2EDuration="4.442054374s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 11:11:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:11:40.441093398 +0000 UTC m=+5.643412548" watchObservedRunningTime="2026-07-09 11:11:40.442054374 +0000 UTC m=+5.644373526" Jul 09 11:11:40.798873 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:40.798822 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91432ff0-fa58-460b-a6d6-3a14a9f5f295-metrics-certs\") pod \"network-metrics-daemon-6bvn2\" (UID: \"91432ff0-fa58-460b-a6d6-3a14a9f5f295\") " pod="openshift-multus/network-metrics-daemon-6bvn2" Jul 09 11:11:40.798873 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:40.798871 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aefb1aae-31d5-4c63-8245-3b4544e21e49-original-pull-secret\") pod \"global-pull-secret-syncer-sg6nt\" (UID: \"aefb1aae-31d5-4c63-8245-3b4544e21e49\") " pod="kube-system/global-pull-secret-syncer-sg6nt" Jul 09 11:11:40.799256 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:40.799000 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 11:11:40.799256 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:40.799060 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aefb1aae-31d5-4c63-8245-3b4544e21e49-original-pull-secret podName:aefb1aae-31d5-4c63-8245-3b4544e21e49 nodeName:}" failed. No retries permitted until 2026-07-09 11:11:44.799042532 +0000 UTC m=+10.001361673 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/aefb1aae-31d5-4c63-8245-3b4544e21e49-original-pull-secret") pod "global-pull-secret-syncer-sg6nt" (UID: "aefb1aae-31d5-4c63-8245-3b4544e21e49") : object "kube-system"/"original-pull-secret" not registered Jul 09 11:11:40.799503 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:40.799480 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 11:11:40.799575 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:40.799535 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91432ff0-fa58-460b-a6d6-3a14a9f5f295-metrics-certs podName:91432ff0-fa58-460b-a6d6-3a14a9f5f295 nodeName:}" failed. No retries permitted until 2026-07-09 11:11:44.799518926 +0000 UTC m=+10.001838072 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/91432ff0-fa58-460b-a6d6-3a14a9f5f295-metrics-certs") pod "network-metrics-daemon-6bvn2" (UID: "91432ff0-fa58-460b-a6d6-3a14a9f5f295") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 11:11:41.000250 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:41.000213 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnrk8\" (UniqueName: \"kubernetes.io/projected/00faf136-9553-489b-a648-408d89197e53-kube-api-access-bnrk8\") pod \"network-check-target-4hjlg\" (UID: \"00faf136-9553-489b-a648-408d89197e53\") " pod="openshift-network-diagnostics/network-check-target-4hjlg" Jul 09 11:11:41.000451 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:41.000431 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 11:11:41.000451 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:41.000451 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 11:11:41.000548 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:41.000463 2550 projected.go:196] Error preparing data for projected volume kube-api-access-bnrk8 for pod openshift-network-diagnostics/network-check-target-4hjlg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 11:11:41.000548 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:41.000524 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/00faf136-9553-489b-a648-408d89197e53-kube-api-access-bnrk8 podName:00faf136-9553-489b-a648-408d89197e53 nodeName:}" failed. No retries permitted until 2026-07-09 11:11:45.000503907 +0000 UTC m=+10.202823047 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-bnrk8" (UniqueName: "kubernetes.io/projected/00faf136-9553-489b-a648-408d89197e53-kube-api-access-bnrk8") pod "network-check-target-4hjlg" (UID: "00faf136-9553-489b-a648-408d89197e53") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 11:11:41.294862 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:41.294830 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4hjlg" Jul 09 11:11:41.295308 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:41.294969 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4hjlg" podUID="00faf136-9553-489b-a648-408d89197e53" Jul 09 11:11:42.293968 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:42.293933 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sg6nt" Jul 09 11:11:42.294160 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:42.294074 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sg6nt" podUID="aefb1aae-31d5-4c63-8245-3b4544e21e49" Jul 09 11:11:42.294414 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:42.294222 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bvn2" Jul 09 11:11:42.294414 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:42.294373 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bvn2" podUID="91432ff0-fa58-460b-a6d6-3a14a9f5f295" Jul 09 11:11:43.297634 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:43.297604 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4hjlg" Jul 09 11:11:43.298092 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:43.297729 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4hjlg" podUID="00faf136-9553-489b-a648-408d89197e53" Jul 09 11:11:44.294341 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:44.294257 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bvn2" Jul 09 11:11:44.294532 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:44.294434 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bvn2" podUID="91432ff0-fa58-460b-a6d6-3a14a9f5f295" Jul 09 11:11:44.294779 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:44.294292 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sg6nt" Jul 09 11:11:44.294779 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:44.294733 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sg6nt" podUID="aefb1aae-31d5-4c63-8245-3b4544e21e49" Jul 09 11:11:44.834612 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:44.834067 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91432ff0-fa58-460b-a6d6-3a14a9f5f295-metrics-certs\") pod \"network-metrics-daemon-6bvn2\" (UID: \"91432ff0-fa58-460b-a6d6-3a14a9f5f295\") " pod="openshift-multus/network-metrics-daemon-6bvn2" Jul 09 11:11:44.834612 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:44.834118 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aefb1aae-31d5-4c63-8245-3b4544e21e49-original-pull-secret\") pod \"global-pull-secret-syncer-sg6nt\" (UID: \"aefb1aae-31d5-4c63-8245-3b4544e21e49\") " pod="kube-system/global-pull-secret-syncer-sg6nt" Jul 09 11:11:44.834612 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:44.834260 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 11:11:44.834612 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:44.834321 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aefb1aae-31d5-4c63-8245-3b4544e21e49-original-pull-secret podName:aefb1aae-31d5-4c63-8245-3b4544e21e49 nodeName:}" failed. No retries permitted until 2026-07-09 11:11:52.83430287 +0000 UTC m=+18.036622002 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/aefb1aae-31d5-4c63-8245-3b4544e21e49-original-pull-secret") pod "global-pull-secret-syncer-sg6nt" (UID: "aefb1aae-31d5-4c63-8245-3b4544e21e49") : object "kube-system"/"original-pull-secret" not registered Jul 09 11:11:44.835226 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:44.834706 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 11:11:44.835226 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:44.834767 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91432ff0-fa58-460b-a6d6-3a14a9f5f295-metrics-certs podName:91432ff0-fa58-460b-a6d6-3a14a9f5f295 nodeName:}" failed. No retries permitted until 2026-07-09 11:11:52.834750302 +0000 UTC m=+18.037069435 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/91432ff0-fa58-460b-a6d6-3a14a9f5f295-metrics-certs") pod "network-metrics-daemon-6bvn2" (UID: "91432ff0-fa58-460b-a6d6-3a14a9f5f295") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 11:11:45.036229 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:45.036195 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnrk8\" (UniqueName: \"kubernetes.io/projected/00faf136-9553-489b-a648-408d89197e53-kube-api-access-bnrk8\") pod \"network-check-target-4hjlg\" (UID: \"00faf136-9553-489b-a648-408d89197e53\") " pod="openshift-network-diagnostics/network-check-target-4hjlg" Jul 09 11:11:45.036410 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:45.036392 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 11:11:45.036469 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:45.036413 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 11:11:45.036469 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:45.036427 2550 projected.go:196] Error preparing data for projected volume kube-api-access-bnrk8 for pod openshift-network-diagnostics/network-check-target-4hjlg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 11:11:45.036547 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:45.036489 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/00faf136-9553-489b-a648-408d89197e53-kube-api-access-bnrk8 podName:00faf136-9553-489b-a648-408d89197e53 nodeName:}" failed. No retries permitted until 2026-07-09 11:11:53.036474664 +0000 UTC m=+18.238793791 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-bnrk8" (UniqueName: "kubernetes.io/projected/00faf136-9553-489b-a648-408d89197e53-kube-api-access-bnrk8") pod "network-check-target-4hjlg" (UID: "00faf136-9553-489b-a648-408d89197e53") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 11:11:45.298482 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:45.298449 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4hjlg" Jul 09 11:11:45.298655 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:45.298582 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4hjlg" podUID="00faf136-9553-489b-a648-408d89197e53" Jul 09 11:11:46.293975 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:46.293905 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sg6nt" Jul 09 11:11:46.294503 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:46.294045 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sg6nt" podUID="aefb1aae-31d5-4c63-8245-3b4544e21e49" Jul 09 11:11:46.294503 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:46.294362 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bvn2" Jul 09 11:11:46.294503 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:46.294461 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bvn2" podUID="91432ff0-fa58-460b-a6d6-3a14a9f5f295" Jul 09 11:11:47.297131 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:47.297097 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4hjlg" Jul 09 11:11:47.297613 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:47.297224 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4hjlg" podUID="00faf136-9553-489b-a648-408d89197e53" Jul 09 11:11:48.294527 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:48.294483 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sg6nt" Jul 09 11:11:48.294527 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:48.294521 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bvn2" Jul 09 11:11:48.294757 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:48.294616 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sg6nt" podUID="aefb1aae-31d5-4c63-8245-3b4544e21e49" Jul 09 11:11:48.294757 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:48.294718 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bvn2" podUID="91432ff0-fa58-460b-a6d6-3a14a9f5f295" Jul 09 11:11:49.294260 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:49.294150 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4hjlg" Jul 09 11:11:49.294770 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:49.294278 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4hjlg" podUID="00faf136-9553-489b-a648-408d89197e53" Jul 09 11:11:50.294100 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:50.294072 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sg6nt" Jul 09 11:11:50.294256 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:50.294109 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bvn2" Jul 09 11:11:50.294256 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:50.294183 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sg6nt" podUID="aefb1aae-31d5-4c63-8245-3b4544e21e49" Jul 09 11:11:50.294620 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:50.294309 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bvn2" podUID="91432ff0-fa58-460b-a6d6-3a14a9f5f295" Jul 09 11:11:51.294006 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:51.293967 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4hjlg" Jul 09 11:11:51.294179 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:51.294092 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4hjlg" podUID="00faf136-9553-489b-a648-408d89197e53" Jul 09 11:11:52.293854 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:52.293816 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bvn2" Jul 09 11:11:52.294402 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:52.293818 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sg6nt" Jul 09 11:11:52.294402 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:52.293961 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bvn2" podUID="91432ff0-fa58-460b-a6d6-3a14a9f5f295" Jul 09 11:11:52.294402 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:52.293992 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sg6nt" podUID="aefb1aae-31d5-4c63-8245-3b4544e21e49" Jul 09 11:11:52.900403 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:52.900363 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91432ff0-fa58-460b-a6d6-3a14a9f5f295-metrics-certs\") pod \"network-metrics-daemon-6bvn2\" (UID: \"91432ff0-fa58-460b-a6d6-3a14a9f5f295\") " pod="openshift-multus/network-metrics-daemon-6bvn2" Jul 09 11:11:52.900403 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:52.900401 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aefb1aae-31d5-4c63-8245-3b4544e21e49-original-pull-secret\") pod \"global-pull-secret-syncer-sg6nt\" (UID: \"aefb1aae-31d5-4c63-8245-3b4544e21e49\") " pod="kube-system/global-pull-secret-syncer-sg6nt" Jul 09 11:11:52.900599 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:52.900498 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 11:11:52.900599 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:52.900506 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 11:11:52.900599 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:52.900549 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aefb1aae-31d5-4c63-8245-3b4544e21e49-original-pull-secret podName:aefb1aae-31d5-4c63-8245-3b4544e21e49 nodeName:}" failed. No retries permitted until 2026-07-09 11:12:08.90053607 +0000 UTC m=+34.102855198 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/aefb1aae-31d5-4c63-8245-3b4544e21e49-original-pull-secret") pod "global-pull-secret-syncer-sg6nt" (UID: "aefb1aae-31d5-4c63-8245-3b4544e21e49") : object "kube-system"/"original-pull-secret" not registered Jul 09 11:11:52.900599 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:52.900562 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91432ff0-fa58-460b-a6d6-3a14a9f5f295-metrics-certs podName:91432ff0-fa58-460b-a6d6-3a14a9f5f295 nodeName:}" failed. No retries permitted until 2026-07-09 11:12:08.900556005 +0000 UTC m=+34.102875132 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/91432ff0-fa58-460b-a6d6-3a14a9f5f295-metrics-certs") pod "network-metrics-daemon-6bvn2" (UID: "91432ff0-fa58-460b-a6d6-3a14a9f5f295") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 11:11:53.102185 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:53.102145 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnrk8\" (UniqueName: \"kubernetes.io/projected/00faf136-9553-489b-a648-408d89197e53-kube-api-access-bnrk8\") pod \"network-check-target-4hjlg\" (UID: \"00faf136-9553-489b-a648-408d89197e53\") " pod="openshift-network-diagnostics/network-check-target-4hjlg" Jul 09 11:11:53.102402 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:53.102304 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 11:11:53.102402 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:53.102339 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 11:11:53.102402 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:53.102353 2550 projected.go:196] Error preparing data for projected volume kube-api-access-bnrk8 for pod openshift-network-diagnostics/network-check-target-4hjlg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 11:11:53.102553 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:53.102413 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/00faf136-9553-489b-a648-408d89197e53-kube-api-access-bnrk8 podName:00faf136-9553-489b-a648-408d89197e53 nodeName:}" failed. No retries permitted until 2026-07-09 11:12:09.102398782 +0000 UTC m=+34.304717908 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-bnrk8" (UniqueName: "kubernetes.io/projected/00faf136-9553-489b-a648-408d89197e53-kube-api-access-bnrk8") pod "network-check-target-4hjlg" (UID: "00faf136-9553-489b-a648-408d89197e53") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 11:11:53.294692 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:53.294660 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4hjlg" Jul 09 11:11:53.295105 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:53.294794 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4hjlg" podUID="00faf136-9553-489b-a648-408d89197e53" Jul 09 11:11:54.294201 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:54.294160 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bvn2" Jul 09 11:11:54.294201 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:54.294203 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sg6nt" Jul 09 11:11:54.294492 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:54.294318 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bvn2" podUID="91432ff0-fa58-460b-a6d6-3a14a9f5f295" Jul 09 11:11:54.294492 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:54.294443 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sg6nt" podUID="aefb1aae-31d5-4c63-8245-3b4544e21e49" Jul 09 11:11:55.294676 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:55.294639 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4hjlg" Jul 09 11:11:55.295156 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:55.294740 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4hjlg" podUID="00faf136-9553-489b-a648-408d89197e53" Jul 09 11:11:56.294471 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:56.294436 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sg6nt" Jul 09 11:11:56.294654 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:56.294436 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bvn2" Jul 09 11:11:56.294654 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:56.294579 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sg6nt" podUID="aefb1aae-31d5-4c63-8245-3b4544e21e49" Jul 09 11:11:56.294654 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:56.294643 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bvn2" podUID="91432ff0-fa58-460b-a6d6-3a14a9f5f295" Jul 09 11:11:57.293873 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:57.293693 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4hjlg" Jul 09 11:11:57.294008 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:57.293953 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4hjlg" podUID="00faf136-9553-489b-a648-408d89197e53" Jul 09 11:11:57.461207 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:57.461179 2550 generic.go:356] "Generic (PLEG): container finished" podID="30d995da-9325-4192-9695-ad8ae483a809" containerID="486ac332a5cf125f9c5064219a0522d7d1b105684a9cadf59b1458399fb98e90" exitCode=0 Jul 09 11:11:57.462080 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:57.461260 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n2nj6" event={"ID":"30d995da-9325-4192-9695-ad8ae483a809","Type":"ContainerDied","Data":"486ac332a5cf125f9c5064219a0522d7d1b105684a9cadf59b1458399fb98e90"} Jul 09 11:11:57.465716 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:57.465666 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" event={"ID":"a6b1c406-e931-4645-9fca-97ee5da046e1","Type":"ContainerStarted","Data":"94fb6ed15cd2b29f2613867476d120f9554acf118719efbe9595aff9d01d792e"} Jul 09 11:11:57.465835 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:57.465744 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" event={"ID":"a6b1c406-e931-4645-9fca-97ee5da046e1","Type":"ContainerStarted","Data":"c7e8e6e9d34e5c7ff86f5d62870c4a9fc894b955a5cdde225f907b7dbd01ca01"} Jul 09 11:11:57.465835 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:57.465813 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" event={"ID":"a6b1c406-e931-4645-9fca-97ee5da046e1","Type":"ContainerStarted","Data":"09fe0ebae1003db2f1beff5bafb21176fb317803238c40a2fedee9971361d7b5"} Jul 09 11:11:57.465957 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:57.465863 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" event={"ID":"a6b1c406-e931-4645-9fca-97ee5da046e1","Type":"ContainerStarted","Data":"ac54694ba2f871ee8ab5552d7dd18631730f4b471447a164f1c4721fe3a4fea4"} Jul 09 11:11:57.465957 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:57.465911 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" event={"ID":"a6b1c406-e931-4645-9fca-97ee5da046e1","Type":"ContainerStarted","Data":"56aab22c7d162ed6173160fb9628cee9aa37c2e45b00aff3b0afaea456a9bea9"} Jul 09 11:11:57.467487 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:57.467465 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" event={"ID":"6ab7210c-0aa5-49ab-8a57-295dad58dce8","Type":"ContainerStarted","Data":"4acdbf813ff7cb91047f30b2c8d776de0e64548720218076fd33c936cb6f9b61"} Jul 09 11:11:57.469411 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:57.469144 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wsfmn" event={"ID":"8117b6cb-dfab-462d-ae86-50a342e88a9a","Type":"ContainerStarted","Data":"3280ac250015d481da6caf108f4de0d9ef112e6098413d8548c3aacfcff5ca07"} Jul 09 11:11:57.470733 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:57.470713 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lpprd" event={"ID":"47077616-7ca2-4b7d-ba31-92d393ab7439","Type":"ContainerStarted","Data":"e80458f940e47801dc648286d9c9f89edc7c12f52ba28dd4d21d8f40339acc2e"} Jul 09 11:11:57.475899 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:57.475862 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn4qf" event={"ID":"eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6","Type":"ContainerStarted","Data":"93a619e5081f0b7db945b02963c7b99377a44e6147a632846bb3630c15997e69"} Jul 09 11:11:57.477515 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:57.477489 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ppz4r" event={"ID":"8207614d-56bb-42cd-a5e6-d7fd5da8dac1","Type":"ContainerStarted","Data":"a3405915049e0ecaeb6e8f241f28a64497d77b845fa7969f82c65341a5f779ec"} Jul 09 11:11:57.478935 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:57.478915 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gfmtp" event={"ID":"13ee2110-7683-4113-a051-ff4ce0ac8f55","Type":"ContainerStarted","Data":"d3054d56f4b7135279a01fef93e430ac7f65adf08f629ba24c621e99d802b629"} Jul 09 11:11:57.509153 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:57.509108 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-gfmtp" podStartSLOduration=3.889301266 podStartE2EDuration="22.509091763s" totalImagesPullingTime="18.619790497s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 11:11:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:11:57.508838003 +0000 UTC m=+22.711157176" watchObservedRunningTime="2026-07-09 11:11:57.509091763 +0000 UTC m=+22.711410910" Jul 09 11:11:57.540870 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:57.540829 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-lpprd" podStartSLOduration=3.97394214 podStartE2EDuration="22.540813871s" totalImagesPullingTime="18.566871731s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 11:11:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:11:57.525897592 +0000 UTC m=+22.728216742" watchObservedRunningTime="2026-07-09 11:11:57.540813871 +0000 UTC m=+22.743133021" Jul 09 11:11:57.541014 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:57.540964 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-wsfmn" podStartSLOduration=3.967660252 podStartE2EDuration="22.540960247s" totalImagesPullingTime="18.573299995s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 11:11:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:11:57.540578531 +0000 UTC m=+22.742897690" watchObservedRunningTime="2026-07-09 11:11:57.540960247 +0000 UTC m=+22.743279396" Jul 09 11:11:57.559014 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:57.558965 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-wnqm7" podStartSLOduration=3.997765455 podStartE2EDuration="22.558948975s" totalImagesPullingTime="18.56118352s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 11:11:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:11:57.558880627 +0000 UTC m=+22.761199787" watchObservedRunningTime="2026-07-09 11:11:57.558948975 +0000 UTC m=+22.761268124" Jul 09 11:11:57.875692 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:57.875609 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-lpprd" Jul 09 11:11:57.876727 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:57.876708 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-lpprd" Jul 09 11:11:57.927444 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:57.927400 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-ppz4r" podStartSLOduration=4.367258219 podStartE2EDuration="22.927383926s" totalImagesPullingTime="18.560125707s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 11:11:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:11:57.573079018 +0000 UTC m=+22.775398186" watchObservedRunningTime="2026-07-09 11:11:57.927383926 +0000 UTC m=+23.129703075" Jul 09 11:11:58.293655 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:58.293633 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bvn2" Jul 09 11:11:58.293800 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:58.293664 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sg6nt" Jul 09 11:11:58.293866 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:58.293791 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bvn2" podUID="91432ff0-fa58-460b-a6d6-3a14a9f5f295" Jul 09 11:11:58.294048 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:58.294019 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sg6nt" podUID="aefb1aae-31d5-4c63-8245-3b4544e21e49" Jul 09 11:11:58.299045 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:58.299020 2550 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 09 11:11:58.483425 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:58.483352 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn4qf" event={"ID":"eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6","Type":"ContainerStarted","Data":"6de74062518a4b63a6e0a6c77f7760e7469a803aa1b1b600af581bf6d4061963"} Jul 09 11:11:58.486251 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:58.486226 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" event={"ID":"a6b1c406-e931-4645-9fca-97ee5da046e1","Type":"ContainerStarted","Data":"6a712addfad9e4b0a90ba38879cfa41f80d3269c952461ad1b2119c83554124d"} Jul 09 11:11:58.487744 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:58.487695 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cbgs5" event={"ID":"d8c8014c-ce48-40d4-94a3-b0a5cd71b2a9","Type":"ContainerStarted","Data":"1461e6125129e1b013fc862a93484972b1d900e0e15fda5fdfb02200e743d229"} Jul 09 11:11:58.487961 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:58.487936 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-lpprd" Jul 09 11:11:58.488560 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:58.488541 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-lpprd" Jul 09 11:11:58.505497 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:58.505451 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-cbgs5" podStartSLOduration=4.943785424 podStartE2EDuration="23.505433552s" totalImagesPullingTime="18.561648128s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 11:11:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:11:58.504699921 +0000 UTC m=+23.707019071" watchObservedRunningTime="2026-07-09 11:11:58.505433552 +0000 UTC m=+23.707752702" Jul 09 11:11:59.222662 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:59.222559 2550 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T11:11:58.299033448Z","UUID":"29bbb6f9-7cb5-4300-8adc-160f0351b30d","Handler":null,"Name":"","Endpoint":""} Jul 09 11:11:59.224062 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:59.224042 2550 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 09 11:11:59.224169 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:59.224081 2550 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 09 11:11:59.294064 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:59.294032 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4hjlg" Jul 09 11:11:59.294228 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:11:59.294139 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4hjlg" podUID="00faf136-9553-489b-a648-408d89197e53" Jul 09 11:11:59.491678 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:59.491591 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn4qf" event={"ID":"eb11ab7b-3fa0-40be-81bc-ba4ed78b69d6","Type":"ContainerStarted","Data":"4f8790c2954f51154a59c63084efc557bcda87048a0f9c9bf158af878a1140d4"} Jul 09 11:11:59.511152 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:11:59.511087 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tn4qf" podStartSLOduration=3.741617527 podStartE2EDuration="24.511071077s" totalImagesPullingTime="20.76945355s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 11:11:35 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:11:59.510903558 +0000 UTC m=+24.713222755" watchObservedRunningTime="2026-07-09 11:11:59.511071077 +0000 UTC m=+24.713390227" Jul 09 11:12:00.294309 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:00.294270 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sg6nt" Jul 09 11:12:00.294509 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:00.294270 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bvn2" Jul 09 11:12:00.294509 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:12:00.294423 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sg6nt" podUID="aefb1aae-31d5-4c63-8245-3b4544e21e49" Jul 09 11:12:00.294509 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:12:00.294450 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bvn2" podUID="91432ff0-fa58-460b-a6d6-3a14a9f5f295" Jul 09 11:12:00.496529 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:00.496496 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" event={"ID":"a6b1c406-e931-4645-9fca-97ee5da046e1","Type":"ContainerStarted","Data":"c98342bc26f5c1df31667a5ea8aff843b67af094dfbb29844bb515aaf2f5b8bc"} Jul 09 11:12:01.294181 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:01.293997 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4hjlg" Jul 09 11:12:01.294362 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:12:01.294281 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4hjlg" podUID="00faf136-9553-489b-a648-408d89197e53" Jul 09 11:12:02.293584 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:02.293554 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bvn2" Jul 09 11:12:02.294104 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:02.293554 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sg6nt" Jul 09 11:12:02.294104 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:12:02.293714 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bvn2" podUID="91432ff0-fa58-460b-a6d6-3a14a9f5f295" Jul 09 11:12:02.294104 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:12:02.293777 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sg6nt" podUID="aefb1aae-31d5-4c63-8245-3b4544e21e49" Jul 09 11:12:03.294451 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:03.294252 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4hjlg" Jul 09 11:12:03.295247 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:12:03.294551 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4hjlg" podUID="00faf136-9553-489b-a648-408d89197e53" Jul 09 11:12:03.503891 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:03.503855 2550 generic.go:356] "Generic (PLEG): container finished" podID="30d995da-9325-4192-9695-ad8ae483a809" containerID="345aa950a418e168affff3eed4688277958269cf7f596bbaa8e17c1f40d789e5" exitCode=0 Jul 09 11:12:03.504082 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:03.503909 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n2nj6" event={"ID":"30d995da-9325-4192-9695-ad8ae483a809","Type":"ContainerDied","Data":"345aa950a418e168affff3eed4688277958269cf7f596bbaa8e17c1f40d789e5"} Jul 09 11:12:03.507652 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:03.507622 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" event={"ID":"a6b1c406-e931-4645-9fca-97ee5da046e1","Type":"ContainerStarted","Data":"9aa68a48f0b11e94d881f89a4b19f4eb5fe4f4ef63d4f7121b6451fb52c0f793"} Jul 09 11:12:03.507990 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:03.507958 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:12:03.508092 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:03.508039 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:12:03.508092 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:03.508089 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:12:03.523190 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:03.523167 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:12:03.523321 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:03.523264 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:12:03.551351 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:03.551240 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" podStartSLOduration=9.914917023 podStartE2EDuration="28.551226213s" totalImagesPullingTime="18.63630919s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 11:11:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:12:03.549939916 +0000 UTC m=+28.752259078" watchObservedRunningTime="2026-07-09 11:12:03.551226213 +0000 UTC m=+28.753545361" Jul 09 11:12:04.293759 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:04.293722 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sg6nt" Jul 09 11:12:04.293759 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:04.293755 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bvn2" Jul 09 11:12:04.294117 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:12:04.293857 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sg6nt" podUID="aefb1aae-31d5-4c63-8245-3b4544e21e49" Jul 09 11:12:04.294117 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:12:04.294001 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bvn2" podUID="91432ff0-fa58-460b-a6d6-3a14a9f5f295" Jul 09 11:12:04.511927 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:04.511894 2550 generic.go:356] "Generic (PLEG): container finished" podID="30d995da-9325-4192-9695-ad8ae483a809" containerID="f2e3eca9f8942a4bb533e3ada40369a7766de1046ec300f4af5e3e301fb41bd4" exitCode=0 Jul 09 11:12:04.512365 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:04.511980 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n2nj6" event={"ID":"30d995da-9325-4192-9695-ad8ae483a809","Type":"ContainerDied","Data":"f2e3eca9f8942a4bb533e3ada40369a7766de1046ec300f4af5e3e301fb41bd4"} Jul 09 11:12:04.780162 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:04.780129 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-sg6nt"] Jul 09 11:12:04.780317 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:04.780264 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sg6nt" Jul 09 11:12:04.780404 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:12:04.780380 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sg6nt" podUID="aefb1aae-31d5-4c63-8245-3b4544e21e49" Jul 09 11:12:04.783290 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:04.783263 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6bvn2"] Jul 09 11:12:04.783444 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:04.783409 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bvn2" Jul 09 11:12:04.783524 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:12:04.783499 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bvn2" podUID="91432ff0-fa58-460b-a6d6-3a14a9f5f295" Jul 09 11:12:04.783882 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:04.783836 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4hjlg"] Jul 09 11:12:04.783975 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:04.783963 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4hjlg" Jul 09 11:12:04.784066 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:12:04.784041 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4hjlg" podUID="00faf136-9553-489b-a648-408d89197e53" Jul 09 11:12:06.293947 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:06.293783 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4hjlg" Jul 09 11:12:06.294298 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:06.293820 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sg6nt" Jul 09 11:12:06.294298 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:12:06.294023 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4hjlg" podUID="00faf136-9553-489b-a648-408d89197e53" Jul 09 11:12:06.294298 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:06.293847 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bvn2" Jul 09 11:12:06.294298 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:12:06.294131 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sg6nt" podUID="aefb1aae-31d5-4c63-8245-3b4544e21e49" Jul 09 11:12:06.294298 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:12:06.294210 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bvn2" podUID="91432ff0-fa58-460b-a6d6-3a14a9f5f295" Jul 09 11:12:06.518863 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:06.518830 2550 generic.go:356] "Generic (PLEG): container finished" podID="30d995da-9325-4192-9695-ad8ae483a809" containerID="73b39f4a0f34004adf02d0841fcd685ef713e0588e8372d64d917d4f923b43d9" exitCode=0 Jul 09 11:12:06.519018 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:06.518886 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n2nj6" event={"ID":"30d995da-9325-4192-9695-ad8ae483a809","Type":"ContainerDied","Data":"73b39f4a0f34004adf02d0841fcd685ef713e0588e8372d64d917d4f923b43d9"} Jul 09 11:12:08.293892 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:08.293856 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4hjlg" Jul 09 11:12:08.293892 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:08.293893 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bvn2" Jul 09 11:12:08.293892 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:08.293898 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sg6nt" Jul 09 11:12:08.294515 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:12:08.293977 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4hjlg" podUID="00faf136-9553-489b-a648-408d89197e53" Jul 09 11:12:08.294515 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:12:08.294123 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6bvn2" podUID="91432ff0-fa58-460b-a6d6-3a14a9f5f295" Jul 09 11:12:08.294515 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:12:08.294209 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sg6nt" podUID="aefb1aae-31d5-4c63-8245-3b4544e21e49" Jul 09 11:12:08.919199 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:08.919159 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91432ff0-fa58-460b-a6d6-3a14a9f5f295-metrics-certs\") pod \"network-metrics-daemon-6bvn2\" (UID: \"91432ff0-fa58-460b-a6d6-3a14a9f5f295\") " pod="openshift-multus/network-metrics-daemon-6bvn2" Jul 09 11:12:08.919508 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:08.919206 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aefb1aae-31d5-4c63-8245-3b4544e21e49-original-pull-secret\") pod \"global-pull-secret-syncer-sg6nt\" (UID: \"aefb1aae-31d5-4c63-8245-3b4544e21e49\") " pod="kube-system/global-pull-secret-syncer-sg6nt" Jul 09 11:12:08.919508 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:12:08.919342 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 11:12:08.919508 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:12:08.919343 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 11:12:08.919508 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:12:08.919410 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91432ff0-fa58-460b-a6d6-3a14a9f5f295-metrics-certs podName:91432ff0-fa58-460b-a6d6-3a14a9f5f295 nodeName:}" failed. No retries permitted until 2026-07-09 11:12:40.919390796 +0000 UTC m=+66.121709927 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/91432ff0-fa58-460b-a6d6-3a14a9f5f295-metrics-certs") pod "network-metrics-daemon-6bvn2" (UID: "91432ff0-fa58-460b-a6d6-3a14a9f5f295") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 11:12:08.919508 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:12:08.919436 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aefb1aae-31d5-4c63-8245-3b4544e21e49-original-pull-secret podName:aefb1aae-31d5-4c63-8245-3b4544e21e49 nodeName:}" failed. No retries permitted until 2026-07-09 11:12:40.919425253 +0000 UTC m=+66.121744379 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/aefb1aae-31d5-4c63-8245-3b4544e21e49-original-pull-secret") pod "global-pull-secret-syncer-sg6nt" (UID: "aefb1aae-31d5-4c63-8245-3b4544e21e49") : object "kube-system"/"original-pull-secret" not registered Jul 09 11:12:09.120627 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:09.120596 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnrk8\" (UniqueName: \"kubernetes.io/projected/00faf136-9553-489b-a648-408d89197e53-kube-api-access-bnrk8\") pod \"network-check-target-4hjlg\" (UID: \"00faf136-9553-489b-a648-408d89197e53\") " pod="openshift-network-diagnostics/network-check-target-4hjlg" Jul 09 11:12:09.120809 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:12:09.120739 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 11:12:09.120809 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:12:09.120755 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 11:12:09.120809 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:12:09.120764 2550 projected.go:196] Error preparing data for projected volume kube-api-access-bnrk8 for pod openshift-network-diagnostics/network-check-target-4hjlg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 11:12:09.120933 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:12:09.120820 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/00faf136-9553-489b-a648-408d89197e53-kube-api-access-bnrk8 podName:00faf136-9553-489b-a648-408d89197e53 nodeName:}" failed. No retries permitted until 2026-07-09 11:12:41.120804682 +0000 UTC m=+66.323123809 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-bnrk8" (UniqueName: "kubernetes.io/projected/00faf136-9553-489b-a648-408d89197e53-kube-api-access-bnrk8") pod "network-check-target-4hjlg" (UID: "00faf136-9553-489b-a648-408d89197e53") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 11:12:09.611944 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:09.611868 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeReady" Jul 09 11:12:09.612408 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:09.611997 2550 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 09 11:12:09.664768 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:09.664736 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vsfwj"] Jul 09 11:12:09.667255 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:09.667228 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vsfwj" Jul 09 11:12:09.668241 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:09.668218 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-96bj9"] Jul 09 11:12:09.670124 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:09.669855 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-96bj9" Jul 09 11:12:09.671575 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:09.671547 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 09 11:12:09.671687 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:09.671677 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 09 11:12:09.671807 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:09.671792 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6pmqk\"" Jul 09 11:12:09.672052 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:09.672035 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-8jsqk\"" Jul 09 11:12:09.672546 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:09.672528 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 09 11:12:09.672641 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:09.672587 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 09 11:12:09.672641 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:09.672624 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 09 11:12:09.682122 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:09.682030 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vsfwj"] Jul 09 11:12:09.683810 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:09.683640 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-96bj9"] Jul 09 11:12:09.825400 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:09.825364 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6rc4\" (UniqueName: \"kubernetes.io/projected/330acdbe-92a6-4ad6-b11a-e941fc7217fc-kube-api-access-w6rc4\") pod \"ingress-canary-96bj9\" (UID: \"330acdbe-92a6-4ad6-b11a-e941fc7217fc\") " pod="openshift-ingress-canary/ingress-canary-96bj9" Jul 09 11:12:09.825596 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:09.825419 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/127cd1f0-bf8a-4bae-94fa-d21833536f5f-tmp-dir\") pod \"dns-default-vsfwj\" (UID: \"127cd1f0-bf8a-4bae-94fa-d21833536f5f\") " pod="openshift-dns/dns-default-vsfwj" Jul 09 11:12:09.825596 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:09.825488 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/127cd1f0-bf8a-4bae-94fa-d21833536f5f-metrics-tls\") pod \"dns-default-vsfwj\" (UID: \"127cd1f0-bf8a-4bae-94fa-d21833536f5f\") " pod="openshift-dns/dns-default-vsfwj" Jul 09 11:12:09.825596 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:09.825508 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsjnm\" (UniqueName: \"kubernetes.io/projected/127cd1f0-bf8a-4bae-94fa-d21833536f5f-kube-api-access-qsjnm\") pod \"dns-default-vsfwj\" (UID: \"127cd1f0-bf8a-4bae-94fa-d21833536f5f\") " pod="openshift-dns/dns-default-vsfwj" Jul 09 11:12:09.825596 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:09.825531 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/330acdbe-92a6-4ad6-b11a-e941fc7217fc-cert\") pod \"ingress-canary-96bj9\" (UID: \"330acdbe-92a6-4ad6-b11a-e941fc7217fc\") " pod="openshift-ingress-canary/ingress-canary-96bj9" Jul 09 11:12:09.825778 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:09.825588 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/127cd1f0-bf8a-4bae-94fa-d21833536f5f-config-volume\") pod \"dns-default-vsfwj\" (UID: \"127cd1f0-bf8a-4bae-94fa-d21833536f5f\") " pod="openshift-dns/dns-default-vsfwj" Jul 09 11:12:09.926682 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:09.926576 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6rc4\" (UniqueName: \"kubernetes.io/projected/330acdbe-92a6-4ad6-b11a-e941fc7217fc-kube-api-access-w6rc4\") pod \"ingress-canary-96bj9\" (UID: \"330acdbe-92a6-4ad6-b11a-e941fc7217fc\") " pod="openshift-ingress-canary/ingress-canary-96bj9" Jul 09 11:12:09.926831 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:09.926739 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/127cd1f0-bf8a-4bae-94fa-d21833536f5f-tmp-dir\") pod \"dns-default-vsfwj\" (UID: \"127cd1f0-bf8a-4bae-94fa-d21833536f5f\") " pod="openshift-dns/dns-default-vsfwj" Jul 09 11:12:09.926831 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:09.926798 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/127cd1f0-bf8a-4bae-94fa-d21833536f5f-metrics-tls\") pod \"dns-default-vsfwj\" (UID: \"127cd1f0-bf8a-4bae-94fa-d21833536f5f\") " pod="openshift-dns/dns-default-vsfwj" Jul 09 11:12:09.926831 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:09.926818 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsjnm\" (UniqueName: \"kubernetes.io/projected/127cd1f0-bf8a-4bae-94fa-d21833536f5f-kube-api-access-qsjnm\") pod \"dns-default-vsfwj\" (UID: \"127cd1f0-bf8a-4bae-94fa-d21833536f5f\") " pod="openshift-dns/dns-default-vsfwj" Jul 09 11:12:09.926986 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:09.926834 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/330acdbe-92a6-4ad6-b11a-e941fc7217fc-cert\") pod \"ingress-canary-96bj9\" (UID: \"330acdbe-92a6-4ad6-b11a-e941fc7217fc\") " pod="openshift-ingress-canary/ingress-canary-96bj9" Jul 09 11:12:09.926986 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:09.926859 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/127cd1f0-bf8a-4bae-94fa-d21833536f5f-config-volume\") pod \"dns-default-vsfwj\" (UID: \"127cd1f0-bf8a-4bae-94fa-d21833536f5f\") " pod="openshift-dns/dns-default-vsfwj" Jul 09 11:12:09.927141 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:12:09.927116 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 11:12:09.927141 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:09.927126 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/127cd1f0-bf8a-4bae-94fa-d21833536f5f-tmp-dir\") pod \"dns-default-vsfwj\" (UID: \"127cd1f0-bf8a-4bae-94fa-d21833536f5f\") " pod="openshift-dns/dns-default-vsfwj" Jul 09 11:12:09.927416 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:12:09.927182 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/127cd1f0-bf8a-4bae-94fa-d21833536f5f-metrics-tls podName:127cd1f0-bf8a-4bae-94fa-d21833536f5f nodeName:}" failed. No retries permitted until 2026-07-09 11:12:10.427161538 +0000 UTC m=+35.629480664 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/127cd1f0-bf8a-4bae-94fa-d21833536f5f-metrics-tls") pod "dns-default-vsfwj" (UID: "127cd1f0-bf8a-4bae-94fa-d21833536f5f") : secret "dns-default-metrics-tls" not found Jul 09 11:12:09.927561 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:09.927534 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/127cd1f0-bf8a-4bae-94fa-d21833536f5f-config-volume\") pod \"dns-default-vsfwj\" (UID: \"127cd1f0-bf8a-4bae-94fa-d21833536f5f\") " pod="openshift-dns/dns-default-vsfwj" Jul 09 11:12:09.931940 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:09.931915 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/330acdbe-92a6-4ad6-b11a-e941fc7217fc-cert\") pod \"ingress-canary-96bj9\" (UID: \"330acdbe-92a6-4ad6-b11a-e941fc7217fc\") " pod="openshift-ingress-canary/ingress-canary-96bj9" Jul 09 11:12:09.939038 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:09.938988 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsjnm\" (UniqueName: \"kubernetes.io/projected/127cd1f0-bf8a-4bae-94fa-d21833536f5f-kube-api-access-qsjnm\") pod \"dns-default-vsfwj\" (UID: \"127cd1f0-bf8a-4bae-94fa-d21833536f5f\") " pod="openshift-dns/dns-default-vsfwj" Jul 09 11:12:09.939199 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:09.939179 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6rc4\" (UniqueName: \"kubernetes.io/projected/330acdbe-92a6-4ad6-b11a-e941fc7217fc-kube-api-access-w6rc4\") pod \"ingress-canary-96bj9\" (UID: \"330acdbe-92a6-4ad6-b11a-e941fc7217fc\") " pod="openshift-ingress-canary/ingress-canary-96bj9" Jul 09 11:12:09.987625 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:09.987586 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-96bj9" Jul 09 11:12:10.294602 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:10.294554 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sg6nt" Jul 09 11:12:10.294602 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:10.294591 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bvn2" Jul 09 11:12:10.294858 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:10.294555 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4hjlg" Jul 09 11:12:10.297496 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:10.297472 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 11:12:10.297646 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:10.297628 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dn2v5\"" Jul 09 11:12:10.298224 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:10.298204 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-h6b4g\"" Jul 09 11:12:10.298351 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:10.298264 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 11:12:10.298351 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:10.298264 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 11:12:10.298351 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:10.298270 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 11:12:10.430685 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:10.430639 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/127cd1f0-bf8a-4bae-94fa-d21833536f5f-metrics-tls\") pod \"dns-default-vsfwj\" (UID: \"127cd1f0-bf8a-4bae-94fa-d21833536f5f\") " pod="openshift-dns/dns-default-vsfwj" Jul 09 11:12:10.430854 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:12:10.430778 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 11:12:10.430854 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:12:10.430840 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/127cd1f0-bf8a-4bae-94fa-d21833536f5f-metrics-tls podName:127cd1f0-bf8a-4bae-94fa-d21833536f5f nodeName:}" failed. No retries permitted until 2026-07-09 11:12:11.430820101 +0000 UTC m=+36.633139236 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/127cd1f0-bf8a-4bae-94fa-d21833536f5f-metrics-tls") pod "dns-default-vsfwj" (UID: "127cd1f0-bf8a-4bae-94fa-d21833536f5f") : secret "dns-default-metrics-tls" not found Jul 09 11:12:11.099054 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:11.098816 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-96bj9"] Jul 09 11:12:11.104557 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:12:11.104520 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod330acdbe_92a6_4ad6_b11a_e941fc7217fc.slice/crio-2e1cb18cd1f5344a0fe892787f2f4b99f0142ca5d212263488ed4b833e78060d WatchSource:0}: Error finding container 2e1cb18cd1f5344a0fe892787f2f4b99f0142ca5d212263488ed4b833e78060d: Status 404 returned error can't find the container with id 2e1cb18cd1f5344a0fe892787f2f4b99f0142ca5d212263488ed4b833e78060d Jul 09 11:12:11.438555 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:11.438515 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/127cd1f0-bf8a-4bae-94fa-d21833536f5f-metrics-tls\") pod \"dns-default-vsfwj\" (UID: \"127cd1f0-bf8a-4bae-94fa-d21833536f5f\") " pod="openshift-dns/dns-default-vsfwj" Jul 09 11:12:11.438730 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:12:11.438670 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 11:12:11.438777 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:12:11.438736 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/127cd1f0-bf8a-4bae-94fa-d21833536f5f-metrics-tls podName:127cd1f0-bf8a-4bae-94fa-d21833536f5f nodeName:}" failed. No retries permitted until 2026-07-09 11:12:13.438720175 +0000 UTC m=+38.641039301 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/127cd1f0-bf8a-4bae-94fa-d21833536f5f-metrics-tls") pod "dns-default-vsfwj" (UID: "127cd1f0-bf8a-4bae-94fa-d21833536f5f") : secret "dns-default-metrics-tls" not found Jul 09 11:12:11.533140 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:11.533109 2550 generic.go:356] "Generic (PLEG): container finished" podID="30d995da-9325-4192-9695-ad8ae483a809" containerID="09f0b70e4d46524aaba12a41caff53e53aba40eb022e939de1c6f88b1d41ff84" exitCode=0 Jul 09 11:12:11.533289 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:11.533188 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n2nj6" event={"ID":"30d995da-9325-4192-9695-ad8ae483a809","Type":"ContainerDied","Data":"09f0b70e4d46524aaba12a41caff53e53aba40eb022e939de1c6f88b1d41ff84"} Jul 09 11:12:11.534412 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:11.534389 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-96bj9" event={"ID":"330acdbe-92a6-4ad6-b11a-e941fc7217fc","Type":"ContainerStarted","Data":"2e1cb18cd1f5344a0fe892787f2f4b99f0142ca5d212263488ed4b833e78060d"} Jul 09 11:12:12.540221 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:12.540185 2550 generic.go:356] "Generic (PLEG): container finished" podID="30d995da-9325-4192-9695-ad8ae483a809" containerID="460044d3d3c827f8cc6ad08075f4072a0267ddb1a6325f99797e7c010434e02b" exitCode=0 Jul 09 11:12:12.540675 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:12.540257 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n2nj6" event={"ID":"30d995da-9325-4192-9695-ad8ae483a809","Type":"ContainerDied","Data":"460044d3d3c827f8cc6ad08075f4072a0267ddb1a6325f99797e7c010434e02b"} Jul 09 11:12:13.453243 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:13.453203 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/127cd1f0-bf8a-4bae-94fa-d21833536f5f-metrics-tls\") pod \"dns-default-vsfwj\" (UID: \"127cd1f0-bf8a-4bae-94fa-d21833536f5f\") " pod="openshift-dns/dns-default-vsfwj" Jul 09 11:12:13.453464 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:12:13.453358 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 11:12:13.453464 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:12:13.453428 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/127cd1f0-bf8a-4bae-94fa-d21833536f5f-metrics-tls podName:127cd1f0-bf8a-4bae-94fa-d21833536f5f nodeName:}" failed. No retries permitted until 2026-07-09 11:12:17.453410051 +0000 UTC m=+42.655729181 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/127cd1f0-bf8a-4bae-94fa-d21833536f5f-metrics-tls") pod "dns-default-vsfwj" (UID: "127cd1f0-bf8a-4bae-94fa-d21833536f5f") : secret "dns-default-metrics-tls" not found Jul 09 11:12:13.546463 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:13.546425 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n2nj6" event={"ID":"30d995da-9325-4192-9695-ad8ae483a809","Type":"ContainerStarted","Data":"68a6789d8276405f593255c4b759a9840c537c1b1a746c7cd26efc77278f1505"} Jul 09 11:12:13.570723 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:13.570651 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-n2nj6" podStartSLOduration=8.101593343 podStartE2EDuration="38.570630388s" totalImagesPullingTime="30.469037045s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 11:11:35 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:12:13.569273482 +0000 UTC m=+38.771592658" watchObservedRunningTime="2026-07-09 11:12:13.570630388 +0000 UTC m=+38.772949539" Jul 09 11:12:14.550976 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:14.550878 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-96bj9" event={"ID":"330acdbe-92a6-4ad6-b11a-e941fc7217fc","Type":"ContainerStarted","Data":"fd9aa7eba49aea465939751b6cd74ab98a26c805a078a939fb1626a08c90cece"} Jul 09 11:12:14.569771 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:14.569712 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-96bj9" podStartSLOduration=2.382679279 podStartE2EDuration="5.569672787s" totalImagesPullingTime="3.186993508s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 11:12:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:12:14.569008305 +0000 UTC m=+39.771327455" watchObservedRunningTime="2026-07-09 11:12:14.569672787 +0000 UTC m=+39.771991914" Jul 09 11:12:17.485130 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:17.485098 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/127cd1f0-bf8a-4bae-94fa-d21833536f5f-metrics-tls\") pod \"dns-default-vsfwj\" (UID: \"127cd1f0-bf8a-4bae-94fa-d21833536f5f\") " pod="openshift-dns/dns-default-vsfwj" Jul 09 11:12:17.485566 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:12:17.485253 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 11:12:17.485566 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:12:17.485315 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/127cd1f0-bf8a-4bae-94fa-d21833536f5f-metrics-tls podName:127cd1f0-bf8a-4bae-94fa-d21833536f5f nodeName:}" failed. No retries permitted until 2026-07-09 11:12:25.485298905 +0000 UTC m=+50.687618033 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/127cd1f0-bf8a-4bae-94fa-d21833536f5f-metrics-tls") pod "dns-default-vsfwj" (UID: "127cd1f0-bf8a-4bae-94fa-d21833536f5f") : secret "dns-default-metrics-tls" not found Jul 09 11:12:25.543977 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:25.543941 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/127cd1f0-bf8a-4bae-94fa-d21833536f5f-metrics-tls\") pod \"dns-default-vsfwj\" (UID: \"127cd1f0-bf8a-4bae-94fa-d21833536f5f\") " pod="openshift-dns/dns-default-vsfwj" Jul 09 11:12:25.544462 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:12:25.544120 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 11:12:25.544462 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:12:25.544198 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/127cd1f0-bf8a-4bae-94fa-d21833536f5f-metrics-tls podName:127cd1f0-bf8a-4bae-94fa-d21833536f5f nodeName:}" failed. No retries permitted until 2026-07-09 11:12:41.544182002 +0000 UTC m=+66.746501129 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/127cd1f0-bf8a-4bae-94fa-d21833536f5f-metrics-tls") pod "dns-default-vsfwj" (UID: "127cd1f0-bf8a-4bae-94fa-d21833536f5f") : secret "dns-default-metrics-tls" not found Jul 09 11:12:35.524468 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:35.524437 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fr9h4" Jul 09 11:12:40.953828 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:40.953793 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91432ff0-fa58-460b-a6d6-3a14a9f5f295-metrics-certs\") pod \"network-metrics-daemon-6bvn2\" (UID: \"91432ff0-fa58-460b-a6d6-3a14a9f5f295\") " pod="openshift-multus/network-metrics-daemon-6bvn2" Jul 09 11:12:40.953828 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:40.953833 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aefb1aae-31d5-4c63-8245-3b4544e21e49-original-pull-secret\") pod \"global-pull-secret-syncer-sg6nt\" (UID: \"aefb1aae-31d5-4c63-8245-3b4544e21e49\") " pod="kube-system/global-pull-secret-syncer-sg6nt" Jul 09 11:12:40.956703 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:40.956684 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 11:12:40.956754 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:40.956719 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 11:12:40.964898 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:12:40.964876 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 09 11:12:40.964948 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:12:40.964938 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91432ff0-fa58-460b-a6d6-3a14a9f5f295-metrics-certs podName:91432ff0-fa58-460b-a6d6-3a14a9f5f295 nodeName:}" failed. No retries permitted until 2026-07-09 11:13:44.964920974 +0000 UTC m=+130.167240101 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/91432ff0-fa58-460b-a6d6-3a14a9f5f295-metrics-certs") pod "network-metrics-daemon-6bvn2" (UID: "91432ff0-fa58-460b-a6d6-3a14a9f5f295") : secret "metrics-daemon-secret" not found Jul 09 11:12:40.968157 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:40.968130 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aefb1aae-31d5-4c63-8245-3b4544e21e49-original-pull-secret\") pod \"global-pull-secret-syncer-sg6nt\" (UID: \"aefb1aae-31d5-4c63-8245-3b4544e21e49\") " pod="kube-system/global-pull-secret-syncer-sg6nt" Jul 09 11:12:41.155091 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:41.155048 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnrk8\" (UniqueName: \"kubernetes.io/projected/00faf136-9553-489b-a648-408d89197e53-kube-api-access-bnrk8\") pod \"network-check-target-4hjlg\" (UID: \"00faf136-9553-489b-a648-408d89197e53\") " pod="openshift-network-diagnostics/network-check-target-4hjlg" Jul 09 11:12:41.157745 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:41.157727 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 11:12:41.167882 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:41.167860 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 11:12:41.178279 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:41.178244 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnrk8\" (UniqueName: \"kubernetes.io/projected/00faf136-9553-489b-a648-408d89197e53-kube-api-access-bnrk8\") pod \"network-check-target-4hjlg\" (UID: \"00faf136-9553-489b-a648-408d89197e53\") " pod="openshift-network-diagnostics/network-check-target-4hjlg" Jul 09 11:12:41.207110 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:41.207021 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sg6nt" Jul 09 11:12:41.218112 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:41.218090 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dn2v5\"" Jul 09 11:12:41.226316 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:41.226282 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4hjlg" Jul 09 11:12:41.339626 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:41.339595 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-sg6nt"] Jul 09 11:12:41.344058 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:12:41.344028 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaefb1aae_31d5_4c63_8245_3b4544e21e49.slice/crio-cc37d234ba047fffbe453d2cac86af99f87a16fa6d7b43f0105b26eb0003c653 WatchSource:0}: Error finding container cc37d234ba047fffbe453d2cac86af99f87a16fa6d7b43f0105b26eb0003c653: Status 404 returned error can't find the container with id cc37d234ba047fffbe453d2cac86af99f87a16fa6d7b43f0105b26eb0003c653 Jul 09 11:12:41.357963 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:41.357939 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4hjlg"] Jul 09 11:12:41.360983 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:12:41.360963 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00faf136_9553_489b_a648_408d89197e53.slice/crio-3c90e1f92cc3b72fbe3236cbd91daefbd618ce41d513a34908c3fe4669d162ab WatchSource:0}: Error finding container 3c90e1f92cc3b72fbe3236cbd91daefbd618ce41d513a34908c3fe4669d162ab: Status 404 returned error can't find the container with id 3c90e1f92cc3b72fbe3236cbd91daefbd618ce41d513a34908c3fe4669d162ab Jul 09 11:12:41.558436 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:41.558402 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/127cd1f0-bf8a-4bae-94fa-d21833536f5f-metrics-tls\") pod \"dns-default-vsfwj\" (UID: \"127cd1f0-bf8a-4bae-94fa-d21833536f5f\") " pod="openshift-dns/dns-default-vsfwj" Jul 09 11:12:41.558597 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:12:41.558527 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 11:12:41.558597 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:12:41.558586 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/127cd1f0-bf8a-4bae-94fa-d21833536f5f-metrics-tls podName:127cd1f0-bf8a-4bae-94fa-d21833536f5f nodeName:}" failed. No retries permitted until 2026-07-09 11:13:13.558569854 +0000 UTC m=+98.760888982 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/127cd1f0-bf8a-4bae-94fa-d21833536f5f-metrics-tls") pod "dns-default-vsfwj" (UID: "127cd1f0-bf8a-4bae-94fa-d21833536f5f") : secret "dns-default-metrics-tls" not found Jul 09 11:12:41.612402 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:41.612359 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-sg6nt" event={"ID":"aefb1aae-31d5-4c63-8245-3b4544e21e49","Type":"ContainerStarted","Data":"cc37d234ba047fffbe453d2cac86af99f87a16fa6d7b43f0105b26eb0003c653"} Jul 09 11:12:41.613420 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:41.613397 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4hjlg" event={"ID":"00faf136-9553-489b-a648-408d89197e53","Type":"ContainerStarted","Data":"3c90e1f92cc3b72fbe3236cbd91daefbd618ce41d513a34908c3fe4669d162ab"} Jul 09 11:12:46.629518 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:46.629485 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-sg6nt" event={"ID":"aefb1aae-31d5-4c63-8245-3b4544e21e49","Type":"ContainerStarted","Data":"09a4ce5cfc1490808927e76178fb93f8dcb2f2d70bee41855c63e052349753ac"} Jul 09 11:12:46.630960 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:46.630933 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4hjlg" event={"ID":"00faf136-9553-489b-a648-408d89197e53","Type":"ContainerStarted","Data":"7e3d4aaf918f1d8cc0840efab14bf1e7d5076b37c85bd7f26e25a0a4a6b04a02"} Jul 09 11:12:46.631060 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:46.630996 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-4hjlg" Jul 09 11:12:46.646491 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:46.646438 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-sg6nt" podStartSLOduration=66.936392463 podStartE2EDuration="1m11.64642247s" totalImagesPullingTime="4.710030007s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 11:11:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:12:46.646230963 +0000 UTC m=+71.848550113" watchObservedRunningTime="2026-07-09 11:12:46.64642247 +0000 UTC m=+71.848741667" Jul 09 11:12:46.664252 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:12:46.664199 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-4hjlg" podStartSLOduration=66.973724877 podStartE2EDuration="1m11.6641846s" totalImagesPullingTime="4.690459723s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 11:11:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:12:46.663605065 +0000 UTC m=+71.865924216" watchObservedRunningTime="2026-07-09 11:12:46.6641846 +0000 UTC m=+71.866503748" Jul 09 11:13:10.876729 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:13:10.876597 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cd7b44b58-vbqtn"] Jul 09 11:13:10.878990 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:13:10.878966 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cd7b44b58-vbqtn" Jul 09 11:13:10.881732 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:13:10.881709 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 09 11:13:10.882897 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:13:10.882874 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 09 11:13:10.883013 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:13:10.882878 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 09 11:13:10.883013 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:13:10.882882 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 09 11:13:10.894468 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:13:10.894385 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cd7b44b58-vbqtn"] Jul 09 11:13:10.937985 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:13:10.937953 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a7fac224-c8f9-4600-a295-3d3ce8e5ed48-tmp\") pod \"klusterlet-addon-workmgr-6cd7b44b58-vbqtn\" (UID: \"a7fac224-c8f9-4600-a295-3d3ce8e5ed48\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cd7b44b58-vbqtn" Jul 09 11:13:10.938151 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:13:10.938000 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a7fac224-c8f9-4600-a295-3d3ce8e5ed48-klusterlet-config\") pod \"klusterlet-addon-workmgr-6cd7b44b58-vbqtn\" (UID: \"a7fac224-c8f9-4600-a295-3d3ce8e5ed48\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cd7b44b58-vbqtn" Jul 09 11:13:10.938151 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:13:10.938020 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmpc6\" (UniqueName: \"kubernetes.io/projected/a7fac224-c8f9-4600-a295-3d3ce8e5ed48-kube-api-access-gmpc6\") pod \"klusterlet-addon-workmgr-6cd7b44b58-vbqtn\" (UID: \"a7fac224-c8f9-4600-a295-3d3ce8e5ed48\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cd7b44b58-vbqtn" Jul 09 11:13:11.038676 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:13:11.038637 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a7fac224-c8f9-4600-a295-3d3ce8e5ed48-tmp\") pod \"klusterlet-addon-workmgr-6cd7b44b58-vbqtn\" (UID: \"a7fac224-c8f9-4600-a295-3d3ce8e5ed48\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cd7b44b58-vbqtn" Jul 09 11:13:11.038848 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:13:11.038697 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a7fac224-c8f9-4600-a295-3d3ce8e5ed48-klusterlet-config\") pod \"klusterlet-addon-workmgr-6cd7b44b58-vbqtn\" (UID: \"a7fac224-c8f9-4600-a295-3d3ce8e5ed48\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cd7b44b58-vbqtn" Jul 09 11:13:11.038848 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:13:11.038723 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmpc6\" (UniqueName: \"kubernetes.io/projected/a7fac224-c8f9-4600-a295-3d3ce8e5ed48-kube-api-access-gmpc6\") pod \"klusterlet-addon-workmgr-6cd7b44b58-vbqtn\" (UID: \"a7fac224-c8f9-4600-a295-3d3ce8e5ed48\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cd7b44b58-vbqtn" Jul 09 11:13:11.039036 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:13:11.039019 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a7fac224-c8f9-4600-a295-3d3ce8e5ed48-tmp\") pod \"klusterlet-addon-workmgr-6cd7b44b58-vbqtn\" (UID: \"a7fac224-c8f9-4600-a295-3d3ce8e5ed48\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cd7b44b58-vbqtn" Jul 09 11:13:11.041180 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:13:11.041159 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a7fac224-c8f9-4600-a295-3d3ce8e5ed48-klusterlet-config\") pod \"klusterlet-addon-workmgr-6cd7b44b58-vbqtn\" (UID: \"a7fac224-c8f9-4600-a295-3d3ce8e5ed48\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cd7b44b58-vbqtn" Jul 09 11:13:11.048414 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:13:11.048391 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmpc6\" (UniqueName: \"kubernetes.io/projected/a7fac224-c8f9-4600-a295-3d3ce8e5ed48-kube-api-access-gmpc6\") pod \"klusterlet-addon-workmgr-6cd7b44b58-vbqtn\" (UID: \"a7fac224-c8f9-4600-a295-3d3ce8e5ed48\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cd7b44b58-vbqtn" Jul 09 11:13:11.187847 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:13:11.187748 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cd7b44b58-vbqtn" Jul 09 11:13:11.312938 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:13:11.310598 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cd7b44b58-vbqtn"] Jul 09 11:13:11.315851 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:13:11.315822 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7fac224_c8f9_4600_a295_3d3ce8e5ed48.slice/crio-c7439e4745735396fe7afe41ec59521277f4dcfc5d0a1ae152f4a2fddb65c530 WatchSource:0}: Error finding container c7439e4745735396fe7afe41ec59521277f4dcfc5d0a1ae152f4a2fddb65c530: Status 404 returned error can't find the container with id c7439e4745735396fe7afe41ec59521277f4dcfc5d0a1ae152f4a2fddb65c530 Jul 09 11:13:11.694223 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:13:11.694191 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cd7b44b58-vbqtn" event={"ID":"a7fac224-c8f9-4600-a295-3d3ce8e5ed48","Type":"ContainerStarted","Data":"c7439e4745735396fe7afe41ec59521277f4dcfc5d0a1ae152f4a2fddb65c530"} Jul 09 11:13:13.655312 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:13:13.655277 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/127cd1f0-bf8a-4bae-94fa-d21833536f5f-metrics-tls\") pod \"dns-default-vsfwj\" (UID: \"127cd1f0-bf8a-4bae-94fa-d21833536f5f\") " pod="openshift-dns/dns-default-vsfwj" Jul 09 11:13:13.655762 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:13:13.655450 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 11:13:13.655762 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:13:13.655532 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/127cd1f0-bf8a-4bae-94fa-d21833536f5f-metrics-tls podName:127cd1f0-bf8a-4bae-94fa-d21833536f5f nodeName:}" failed. No retries permitted until 2026-07-09 11:14:17.655510785 +0000 UTC m=+162.857829919 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/127cd1f0-bf8a-4bae-94fa-d21833536f5f-metrics-tls") pod "dns-default-vsfwj" (UID: "127cd1f0-bf8a-4bae-94fa-d21833536f5f") : secret "dns-default-metrics-tls" not found Jul 09 11:13:15.706458 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:13:15.706423 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cd7b44b58-vbqtn" event={"ID":"a7fac224-c8f9-4600-a295-3d3ce8e5ed48","Type":"ContainerStarted","Data":"e84e48e32242a614178d6f61613a78e1db8aaa3046c1dcb82030c129a48e1a50"} Jul 09 11:13:15.706879 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:13:15.706635 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cd7b44b58-vbqtn" Jul 09 11:13:15.708322 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:13:15.708302 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cd7b44b58-vbqtn" Jul 09 11:13:15.723447 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:13:15.723404 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cd7b44b58-vbqtn" podStartSLOduration=2.084692857 podStartE2EDuration="5.723390465s" totalImagesPullingTime="3.638697608s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 11:13:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:13:15.72301352 +0000 UTC m=+100.925332669" watchObservedRunningTime="2026-07-09 11:13:15.723390465 +0000 UTC m=+100.925709613" Jul 09 11:13:17.636473 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:13:17.636444 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-4hjlg" Jul 09 11:13:44.968268 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:13:44.968231 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91432ff0-fa58-460b-a6d6-3a14a9f5f295-metrics-certs\") pod \"network-metrics-daemon-6bvn2\" (UID: \"91432ff0-fa58-460b-a6d6-3a14a9f5f295\") " pod="openshift-multus/network-metrics-daemon-6bvn2" Jul 09 11:13:44.968748 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:13:44.968400 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 09 11:13:44.968748 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:13:44.968476 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91432ff0-fa58-460b-a6d6-3a14a9f5f295-metrics-certs podName:91432ff0-fa58-460b-a6d6-3a14a9f5f295 nodeName:}" failed. No retries permitted until 2026-07-09 11:15:46.968456557 +0000 UTC m=+252.170775703 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/91432ff0-fa58-460b-a6d6-3a14a9f5f295-metrics-certs") pod "network-metrics-daemon-6bvn2" (UID: "91432ff0-fa58-460b-a6d6-3a14a9f5f295") : secret "metrics-daemon-secret" not found Jul 09 11:14:12.679983 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:14:12.679940 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-vsfwj" podUID="127cd1f0-bf8a-4bae-94fa-d21833536f5f" Jul 09 11:14:12.838447 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:14:12.838418 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vsfwj" Jul 09 11:14:13.322853 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:14:13.322814 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-6bvn2" podUID="91432ff0-fa58-460b-a6d6-3a14a9f5f295" Jul 09 11:14:17.685655 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:14:17.685614 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/127cd1f0-bf8a-4bae-94fa-d21833536f5f-metrics-tls\") pod \"dns-default-vsfwj\" (UID: \"127cd1f0-bf8a-4bae-94fa-d21833536f5f\") " pod="openshift-dns/dns-default-vsfwj" Jul 09 11:14:17.686085 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:14:17.685775 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 11:14:17.686085 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:14:17.685851 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/127cd1f0-bf8a-4bae-94fa-d21833536f5f-metrics-tls podName:127cd1f0-bf8a-4bae-94fa-d21833536f5f nodeName:}" failed. No retries permitted until 2026-07-09 11:16:19.685832989 +0000 UTC m=+284.888152119 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/127cd1f0-bf8a-4bae-94fa-d21833536f5f-metrics-tls") pod "dns-default-vsfwj" (UID: "127cd1f0-bf8a-4bae-94fa-d21833536f5f") : secret "dns-default-metrics-tls" not found Jul 09 11:14:24.293716 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:14:24.293655 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bvn2" Jul 09 11:15:15.707802 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:15.707752 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cd7b44b58-vbqtn" podUID="a7fac224-c8f9-4600-a295-3d3ce8e5ed48" containerName="acm-agent" probeResult="failure" output="Get \"http://10.133.0.8:8000/readyz\": dial tcp 10.133.0.8:8000: connect: connection refused" Jul 09 11:15:15.989186 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:15.989100 2550 generic.go:356] "Generic (PLEG): container finished" podID="a7fac224-c8f9-4600-a295-3d3ce8e5ed48" containerID="e84e48e32242a614178d6f61613a78e1db8aaa3046c1dcb82030c129a48e1a50" exitCode=1 Jul 09 11:15:15.989186 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:15.989140 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cd7b44b58-vbqtn" event={"ID":"a7fac224-c8f9-4600-a295-3d3ce8e5ed48","Type":"ContainerDied","Data":"e84e48e32242a614178d6f61613a78e1db8aaa3046c1dcb82030c129a48e1a50"} Jul 09 11:15:15.989511 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:15.989494 2550 scope.go:117] "RemoveContainer" containerID="e84e48e32242a614178d6f61613a78e1db8aaa3046c1dcb82030c129a48e1a50" Jul 09 11:15:16.992853 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:16.992821 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cd7b44b58-vbqtn" event={"ID":"a7fac224-c8f9-4600-a295-3d3ce8e5ed48","Type":"ContainerStarted","Data":"e71feaa8a1cff09ffb0a03a6b98d7cbf4c0fe8377e80bc805405754088f18ff8"} Jul 09 11:15:16.993230 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:16.993093 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cd7b44b58-vbqtn" Jul 09 11:15:16.993736 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:16.993719 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6cd7b44b58-vbqtn" Jul 09 11:15:46.981095 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:46.981056 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91432ff0-fa58-460b-a6d6-3a14a9f5f295-metrics-certs\") pod \"network-metrics-daemon-6bvn2\" (UID: \"91432ff0-fa58-460b-a6d6-3a14a9f5f295\") " pod="openshift-multus/network-metrics-daemon-6bvn2" Jul 09 11:15:46.981699 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:15:46.981223 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 09 11:15:46.981699 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:15:46.981321 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91432ff0-fa58-460b-a6d6-3a14a9f5f295-metrics-certs podName:91432ff0-fa58-460b-a6d6-3a14a9f5f295 nodeName:}" failed. No retries permitted until 2026-07-09 11:17:48.981297518 +0000 UTC m=+374.183616646 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/91432ff0-fa58-460b-a6d6-3a14a9f5f295-metrics-certs") pod "network-metrics-daemon-6bvn2" (UID: "91432ff0-fa58-460b-a6d6-3a14a9f5f295") : secret "metrics-daemon-secret" not found Jul 09 11:15:49.129153 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.129119 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-hr5fb"] Jul 09 11:15:49.131009 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.130987 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-hr5fb" Jul 09 11:15:49.133731 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.133707 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 09 11:15:49.133868 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.133713 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 09 11:15:49.133868 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.133798 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 09 11:15:49.133868 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.133832 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 09 11:15:49.134031 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.133993 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-699bccf5c4-zkf4x"] Jul 09 11:15:49.135596 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.135583 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" Jul 09 11:15:49.137683 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.137663 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 09 11:15:49.137979 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.137961 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 09 11:15:49.137979 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.137974 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 09 11:15:49.153699 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.153683 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-6sg55\"" Jul 09 11:15:49.158969 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.158949 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-hr5fb"] Jul 09 11:15:49.161958 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.161941 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-5q9mj\"" Jul 09 11:15:49.167482 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.167463 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 09 11:15:49.168197 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.168167 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-699bccf5c4-zkf4x"] Jul 09 11:15:49.194600 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.194575 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/511c08dd-43b1-485c-b275-df75627c0441-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-hr5fb\" (UID: \"511c08dd-43b1-485c-b275-df75627c0441\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-hr5fb" Jul 09 11:15:49.194703 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.194617 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/48f05f33-fe91-4449-a9a8-a03a95d45006-bound-sa-token\") pod \"image-registry-699bccf5c4-zkf4x\" (UID: \"48f05f33-fe91-4449-a9a8-a03a95d45006\") " pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" Jul 09 11:15:49.194703 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.194641 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/48f05f33-fe91-4449-a9a8-a03a95d45006-registry-certificates\") pod \"image-registry-699bccf5c4-zkf4x\" (UID: \"48f05f33-fe91-4449-a9a8-a03a95d45006\") " pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" Jul 09 11:15:49.194703 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.194665 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/511c08dd-43b1-485c-b275-df75627c0441-config\") pod \"service-ca-operator-5d46d9ffd4-hr5fb\" (UID: \"511c08dd-43b1-485c-b275-df75627c0441\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-hr5fb" Jul 09 11:15:49.194703 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.194683 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/48f05f33-fe91-4449-a9a8-a03a95d45006-trusted-ca\") pod \"image-registry-699bccf5c4-zkf4x\" (UID: \"48f05f33-fe91-4449-a9a8-a03a95d45006\") " pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" Jul 09 11:15:49.194833 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.194752 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8k8bg\" (UniqueName: \"kubernetes.io/projected/511c08dd-43b1-485c-b275-df75627c0441-kube-api-access-8k8bg\") pod \"service-ca-operator-5d46d9ffd4-hr5fb\" (UID: \"511c08dd-43b1-485c-b275-df75627c0441\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-hr5fb" Jul 09 11:15:49.194833 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.194776 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/48f05f33-fe91-4449-a9a8-a03a95d45006-ca-trust-extracted\") pod \"image-registry-699bccf5c4-zkf4x\" (UID: \"48f05f33-fe91-4449-a9a8-a03a95d45006\") " pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" Jul 09 11:15:49.194833 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.194793 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/48f05f33-fe91-4449-a9a8-a03a95d45006-installation-pull-secrets\") pod \"image-registry-699bccf5c4-zkf4x\" (UID: \"48f05f33-fe91-4449-a9a8-a03a95d45006\") " pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" Jul 09 11:15:49.194833 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.194809 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkwr2\" (UniqueName: \"kubernetes.io/projected/48f05f33-fe91-4449-a9a8-a03a95d45006-kube-api-access-hkwr2\") pod \"image-registry-699bccf5c4-zkf4x\" (UID: \"48f05f33-fe91-4449-a9a8-a03a95d45006\") " pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" Jul 09 11:15:49.194833 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.194829 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/48f05f33-fe91-4449-a9a8-a03a95d45006-image-registry-private-configuration\") pod \"image-registry-699bccf5c4-zkf4x\" (UID: \"48f05f33-fe91-4449-a9a8-a03a95d45006\") " pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" Jul 09 11:15:49.194978 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.194847 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48f05f33-fe91-4449-a9a8-a03a95d45006-registry-tls\") pod \"image-registry-699bccf5c4-zkf4x\" (UID: \"48f05f33-fe91-4449-a9a8-a03a95d45006\") " pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" Jul 09 11:15:49.231399 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.231373 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-7wm65"] Jul 09 11:15:49.233263 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.233244 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-98ghf"] Jul 09 11:15:49.233383 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.233366 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-7wm65" Jul 09 11:15:49.234896 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.234881 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-98ghf" Jul 09 11:15:49.251899 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.251877 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 09 11:15:49.251997 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.251877 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 09 11:15:49.251997 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.251956 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 09 11:15:49.252104 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.252048 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-q8dgc\"" Jul 09 11:15:49.252156 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.252130 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-kc76t\"" Jul 09 11:15:49.252317 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.252302 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 09 11:15:49.252420 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.252382 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 09 11:15:49.252420 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.252399 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 09 11:15:49.253859 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.253830 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-98ghf"] Jul 09 11:15:49.261068 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.261033 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-7wm65"] Jul 09 11:15:49.262553 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.262538 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 09 11:15:49.295430 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.295399 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8k8bg\" (UniqueName: \"kubernetes.io/projected/511c08dd-43b1-485c-b275-df75627c0441-kube-api-access-8k8bg\") pod \"service-ca-operator-5d46d9ffd4-hr5fb\" (UID: \"511c08dd-43b1-485c-b275-df75627c0441\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-hr5fb" Jul 09 11:15:49.295611 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.295436 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/48f05f33-fe91-4449-a9a8-a03a95d45006-ca-trust-extracted\") pod \"image-registry-699bccf5c4-zkf4x\" (UID: \"48f05f33-fe91-4449-a9a8-a03a95d45006\") " pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" Jul 09 11:15:49.295611 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.295460 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/48f05f33-fe91-4449-a9a8-a03a95d45006-installation-pull-secrets\") pod \"image-registry-699bccf5c4-zkf4x\" (UID: \"48f05f33-fe91-4449-a9a8-a03a95d45006\") " pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" Jul 09 11:15:49.295611 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.295481 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkwr2\" (UniqueName: \"kubernetes.io/projected/48f05f33-fe91-4449-a9a8-a03a95d45006-kube-api-access-hkwr2\") pod \"image-registry-699bccf5c4-zkf4x\" (UID: \"48f05f33-fe91-4449-a9a8-a03a95d45006\") " pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" Jul 09 11:15:49.295611 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.295506 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/48f05f33-fe91-4449-a9a8-a03a95d45006-image-registry-private-configuration\") pod \"image-registry-699bccf5c4-zkf4x\" (UID: \"48f05f33-fe91-4449-a9a8-a03a95d45006\") " pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" Jul 09 11:15:49.295611 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.295532 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fccccfad-9a68-4bd9-a44f-af22e4339085-config\") pod \"console-operator-68b757fb67-98ghf\" (UID: \"fccccfad-9a68-4bd9-a44f-af22e4339085\") " pod="openshift-console-operator/console-operator-68b757fb67-98ghf" Jul 09 11:15:49.295611 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.295559 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48f05f33-fe91-4449-a9a8-a03a95d45006-registry-tls\") pod \"image-registry-699bccf5c4-zkf4x\" (UID: \"48f05f33-fe91-4449-a9a8-a03a95d45006\") " pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" Jul 09 11:15:49.295611 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.295585 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tbb9\" (UniqueName: \"kubernetes.io/projected/fccccfad-9a68-4bd9-a44f-af22e4339085-kube-api-access-5tbb9\") pod \"console-operator-68b757fb67-98ghf\" (UID: \"fccccfad-9a68-4bd9-a44f-af22e4339085\") " pod="openshift-console-operator/console-operator-68b757fb67-98ghf" Jul 09 11:15:49.296117 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:15:49.295670 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 11:15:49.296117 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:15:49.295685 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-699bccf5c4-zkf4x: secret "image-registry-tls" not found Jul 09 11:15:49.296117 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.295668 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/511c08dd-43b1-485c-b275-df75627c0441-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-hr5fb\" (UID: \"511c08dd-43b1-485c-b275-df75627c0441\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-hr5fb" Jul 09 11:15:49.296117 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:15:49.295753 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48f05f33-fe91-4449-a9a8-a03a95d45006-registry-tls podName:48f05f33-fe91-4449-a9a8-a03a95d45006 nodeName:}" failed. No retries permitted until 2026-07-09 11:15:49.795730718 +0000 UTC m=+254.998049863 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/48f05f33-fe91-4449-a9a8-a03a95d45006-registry-tls") pod "image-registry-699bccf5c4-zkf4x" (UID: "48f05f33-fe91-4449-a9a8-a03a95d45006") : secret "image-registry-tls" not found Jul 09 11:15:49.296117 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.295823 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/48f05f33-fe91-4449-a9a8-a03a95d45006-bound-sa-token\") pod \"image-registry-699bccf5c4-zkf4x\" (UID: \"48f05f33-fe91-4449-a9a8-a03a95d45006\") " pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" Jul 09 11:15:49.296117 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.295858 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/48f05f33-fe91-4449-a9a8-a03a95d45006-registry-certificates\") pod \"image-registry-699bccf5c4-zkf4x\" (UID: \"48f05f33-fe91-4449-a9a8-a03a95d45006\") " pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" Jul 09 11:15:49.296117 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.295862 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/48f05f33-fe91-4449-a9a8-a03a95d45006-ca-trust-extracted\") pod \"image-registry-699bccf5c4-zkf4x\" (UID: \"48f05f33-fe91-4449-a9a8-a03a95d45006\") " pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" Jul 09 11:15:49.296117 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.295887 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fccccfad-9a68-4bd9-a44f-af22e4339085-serving-cert\") pod \"console-operator-68b757fb67-98ghf\" (UID: \"fccccfad-9a68-4bd9-a44f-af22e4339085\") " pod="openshift-console-operator/console-operator-68b757fb67-98ghf" Jul 09 11:15:49.296117 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.295915 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/511c08dd-43b1-485c-b275-df75627c0441-config\") pod \"service-ca-operator-5d46d9ffd4-hr5fb\" (UID: \"511c08dd-43b1-485c-b275-df75627c0441\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-hr5fb" Jul 09 11:15:49.296117 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.295939 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/48f05f33-fe91-4449-a9a8-a03a95d45006-trusted-ca\") pod \"image-registry-699bccf5c4-zkf4x\" (UID: \"48f05f33-fe91-4449-a9a8-a03a95d45006\") " pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" Jul 09 11:15:49.296117 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.295964 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7dfm\" (UniqueName: \"kubernetes.io/projected/97cba2fd-d06f-4c64-82e1-9e9d65be3908-kube-api-access-s7dfm\") pod \"volume-data-source-validator-5d456fdbcd-7wm65\" (UID: \"97cba2fd-d06f-4c64-82e1-9e9d65be3908\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-7wm65" Jul 09 11:15:49.296117 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.295997 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fccccfad-9a68-4bd9-a44f-af22e4339085-trusted-ca\") pod \"console-operator-68b757fb67-98ghf\" (UID: \"fccccfad-9a68-4bd9-a44f-af22e4339085\") " pod="openshift-console-operator/console-operator-68b757fb67-98ghf" Jul 09 11:15:49.296934 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.296508 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/511c08dd-43b1-485c-b275-df75627c0441-config\") pod \"service-ca-operator-5d46d9ffd4-hr5fb\" (UID: \"511c08dd-43b1-485c-b275-df75627c0441\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-hr5fb" Jul 09 11:15:49.297004 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.296976 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/48f05f33-fe91-4449-a9a8-a03a95d45006-trusted-ca\") pod \"image-registry-699bccf5c4-zkf4x\" (UID: \"48f05f33-fe91-4449-a9a8-a03a95d45006\") " pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" Jul 09 11:15:49.297179 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.297156 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/48f05f33-fe91-4449-a9a8-a03a95d45006-registry-certificates\") pod \"image-registry-699bccf5c4-zkf4x\" (UID: \"48f05f33-fe91-4449-a9a8-a03a95d45006\") " pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" Jul 09 11:15:49.298209 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.298179 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/511c08dd-43b1-485c-b275-df75627c0441-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-hr5fb\" (UID: \"511c08dd-43b1-485c-b275-df75627c0441\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-hr5fb" Jul 09 11:15:49.298306 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.298285 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/48f05f33-fe91-4449-a9a8-a03a95d45006-installation-pull-secrets\") pod \"image-registry-699bccf5c4-zkf4x\" (UID: \"48f05f33-fe91-4449-a9a8-a03a95d45006\") " pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" Jul 09 11:15:49.298361 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.298304 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/48f05f33-fe91-4449-a9a8-a03a95d45006-image-registry-private-configuration\") pod \"image-registry-699bccf5c4-zkf4x\" (UID: \"48f05f33-fe91-4449-a9a8-a03a95d45006\") " pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" Jul 09 11:15:49.314561 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.314531 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k8bg\" (UniqueName: \"kubernetes.io/projected/511c08dd-43b1-485c-b275-df75627c0441-kube-api-access-8k8bg\") pod \"service-ca-operator-5d46d9ffd4-hr5fb\" (UID: \"511c08dd-43b1-485c-b275-df75627c0441\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-hr5fb" Jul 09 11:15:49.314686 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.314613 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkwr2\" (UniqueName: \"kubernetes.io/projected/48f05f33-fe91-4449-a9a8-a03a95d45006-kube-api-access-hkwr2\") pod \"image-registry-699bccf5c4-zkf4x\" (UID: \"48f05f33-fe91-4449-a9a8-a03a95d45006\") " pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" Jul 09 11:15:49.314865 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.314849 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/48f05f33-fe91-4449-a9a8-a03a95d45006-bound-sa-token\") pod \"image-registry-699bccf5c4-zkf4x\" (UID: \"48f05f33-fe91-4449-a9a8-a03a95d45006\") " pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" Jul 09 11:15:49.396407 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.396294 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fccccfad-9a68-4bd9-a44f-af22e4339085-config\") pod \"console-operator-68b757fb67-98ghf\" (UID: \"fccccfad-9a68-4bd9-a44f-af22e4339085\") " pod="openshift-console-operator/console-operator-68b757fb67-98ghf" Jul 09 11:15:49.396407 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.396364 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tbb9\" (UniqueName: \"kubernetes.io/projected/fccccfad-9a68-4bd9-a44f-af22e4339085-kube-api-access-5tbb9\") pod \"console-operator-68b757fb67-98ghf\" (UID: \"fccccfad-9a68-4bd9-a44f-af22e4339085\") " pod="openshift-console-operator/console-operator-68b757fb67-98ghf" Jul 09 11:15:49.396407 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.396402 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fccccfad-9a68-4bd9-a44f-af22e4339085-serving-cert\") pod \"console-operator-68b757fb67-98ghf\" (UID: \"fccccfad-9a68-4bd9-a44f-af22e4339085\") " pod="openshift-console-operator/console-operator-68b757fb67-98ghf" Jul 09 11:15:49.396682 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.396420 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7dfm\" (UniqueName: \"kubernetes.io/projected/97cba2fd-d06f-4c64-82e1-9e9d65be3908-kube-api-access-s7dfm\") pod \"volume-data-source-validator-5d456fdbcd-7wm65\" (UID: \"97cba2fd-d06f-4c64-82e1-9e9d65be3908\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-7wm65" Jul 09 11:15:49.396682 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.396438 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fccccfad-9a68-4bd9-a44f-af22e4339085-trusted-ca\") pod \"console-operator-68b757fb67-98ghf\" (UID: \"fccccfad-9a68-4bd9-a44f-af22e4339085\") " pod="openshift-console-operator/console-operator-68b757fb67-98ghf" Jul 09 11:15:49.397059 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.397033 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fccccfad-9a68-4bd9-a44f-af22e4339085-config\") pod \"console-operator-68b757fb67-98ghf\" (UID: \"fccccfad-9a68-4bd9-a44f-af22e4339085\") " pod="openshift-console-operator/console-operator-68b757fb67-98ghf" Jul 09 11:15:49.397217 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.397199 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fccccfad-9a68-4bd9-a44f-af22e4339085-trusted-ca\") pod \"console-operator-68b757fb67-98ghf\" (UID: \"fccccfad-9a68-4bd9-a44f-af22e4339085\") " pod="openshift-console-operator/console-operator-68b757fb67-98ghf" Jul 09 11:15:49.398619 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.398588 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fccccfad-9a68-4bd9-a44f-af22e4339085-serving-cert\") pod \"console-operator-68b757fb67-98ghf\" (UID: \"fccccfad-9a68-4bd9-a44f-af22e4339085\") " pod="openshift-console-operator/console-operator-68b757fb67-98ghf" Jul 09 11:15:49.410146 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.410119 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tbb9\" (UniqueName: \"kubernetes.io/projected/fccccfad-9a68-4bd9-a44f-af22e4339085-kube-api-access-5tbb9\") pod \"console-operator-68b757fb67-98ghf\" (UID: \"fccccfad-9a68-4bd9-a44f-af22e4339085\") " pod="openshift-console-operator/console-operator-68b757fb67-98ghf" Jul 09 11:15:49.411946 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.411926 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7dfm\" (UniqueName: \"kubernetes.io/projected/97cba2fd-d06f-4c64-82e1-9e9d65be3908-kube-api-access-s7dfm\") pod \"volume-data-source-validator-5d456fdbcd-7wm65\" (UID: \"97cba2fd-d06f-4c64-82e1-9e9d65be3908\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-7wm65" Jul 09 11:15:49.441236 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.441202 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-hr5fb" Jul 09 11:15:49.542760 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.542729 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-7wm65" Jul 09 11:15:49.547440 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.547418 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-98ghf" Jul 09 11:15:49.565809 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.565778 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-hr5fb"] Jul 09 11:15:49.569906 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:15:49.569872 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod511c08dd_43b1_485c_b275_df75627c0441.slice/crio-fba5f2327526a4b4feb62544e142c80897c974004d44610f61a54c075f03b382 WatchSource:0}: Error finding container fba5f2327526a4b4feb62544e142c80897c974004d44610f61a54c075f03b382: Status 404 returned error can't find the container with id fba5f2327526a4b4feb62544e142c80897c974004d44610f61a54c075f03b382 Jul 09 11:15:49.690065 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.690036 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-98ghf"] Jul 09 11:15:49.693221 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:15:49.693192 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfccccfad_9a68_4bd9_a44f_af22e4339085.slice/crio-56b21319585b5a162cd28ba741dac29f38f38c42d4aaf543924387dcaf916aef WatchSource:0}: Error finding container 56b21319585b5a162cd28ba741dac29f38f38c42d4aaf543924387dcaf916aef: Status 404 returned error can't find the container with id 56b21319585b5a162cd28ba741dac29f38f38c42d4aaf543924387dcaf916aef Jul 09 11:15:49.712361 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.712316 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-7wm65"] Jul 09 11:15:49.715216 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:15:49.715193 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97cba2fd_d06f_4c64_82e1_9e9d65be3908.slice/crio-f88a6d8a300c9b177ffced6f6ca8544dede25b18262602d996eef1b86025f607 WatchSource:0}: Error finding container f88a6d8a300c9b177ffced6f6ca8544dede25b18262602d996eef1b86025f607: Status 404 returned error can't find the container with id f88a6d8a300c9b177ffced6f6ca8544dede25b18262602d996eef1b86025f607 Jul 09 11:15:49.801963 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:49.801929 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48f05f33-fe91-4449-a9a8-a03a95d45006-registry-tls\") pod \"image-registry-699bccf5c4-zkf4x\" (UID: \"48f05f33-fe91-4449-a9a8-a03a95d45006\") " pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" Jul 09 11:15:49.802098 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:15:49.802078 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 11:15:49.802098 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:15:49.802095 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-699bccf5c4-zkf4x: secret "image-registry-tls" not found Jul 09 11:15:49.802183 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:15:49.802148 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48f05f33-fe91-4449-a9a8-a03a95d45006-registry-tls podName:48f05f33-fe91-4449-a9a8-a03a95d45006 nodeName:}" failed. No retries permitted until 2026-07-09 11:15:50.802130246 +0000 UTC m=+256.004449384 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/48f05f33-fe91-4449-a9a8-a03a95d45006-registry-tls") pod "image-registry-699bccf5c4-zkf4x" (UID: "48f05f33-fe91-4449-a9a8-a03a95d45006") : secret "image-registry-tls" not found Jul 09 11:15:50.067540 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:50.067498 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-7wm65" event={"ID":"97cba2fd-d06f-4c64-82e1-9e9d65be3908","Type":"ContainerStarted","Data":"f88a6d8a300c9b177ffced6f6ca8544dede25b18262602d996eef1b86025f607"} Jul 09 11:15:50.068419 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:50.068401 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-98ghf" event={"ID":"fccccfad-9a68-4bd9-a44f-af22e4339085","Type":"ContainerStarted","Data":"56b21319585b5a162cd28ba741dac29f38f38c42d4aaf543924387dcaf916aef"} Jul 09 11:15:50.069285 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:50.069267 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-hr5fb" event={"ID":"511c08dd-43b1-485c-b275-df75627c0441","Type":"ContainerStarted","Data":"fba5f2327526a4b4feb62544e142c80897c974004d44610f61a54c075f03b382"} Jul 09 11:15:50.810668 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:50.810613 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48f05f33-fe91-4449-a9a8-a03a95d45006-registry-tls\") pod \"image-registry-699bccf5c4-zkf4x\" (UID: \"48f05f33-fe91-4449-a9a8-a03a95d45006\") " pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" Jul 09 11:15:50.811127 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:15:50.810750 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 11:15:50.811127 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:15:50.810773 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-699bccf5c4-zkf4x: secret "image-registry-tls" not found Jul 09 11:15:50.811127 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:15:50.810838 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48f05f33-fe91-4449-a9a8-a03a95d45006-registry-tls podName:48f05f33-fe91-4449-a9a8-a03a95d45006 nodeName:}" failed. No retries permitted until 2026-07-09 11:15:52.810817842 +0000 UTC m=+258.013136976 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/48f05f33-fe91-4449-a9a8-a03a95d45006-registry-tls") pod "image-registry-699bccf5c4-zkf4x" (UID: "48f05f33-fe91-4449-a9a8-a03a95d45006") : secret "image-registry-tls" not found Jul 09 11:15:52.077057 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:52.077028 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-7wm65" event={"ID":"97cba2fd-d06f-4c64-82e1-9e9d65be3908","Type":"ContainerStarted","Data":"cdd89fc394641962ed6d33bd2de3ff52d288363bc939801ca750fd50e4296447"} Jul 09 11:15:52.102518 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:52.102473 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-7wm65" podStartSLOduration=1.579488735 podStartE2EDuration="3.102456396s" totalImagesPullingTime="1.522967661s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 11:15:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:15:52.102449043 +0000 UTC m=+257.304768191" watchObservedRunningTime="2026-07-09 11:15:52.102456396 +0000 UTC m=+257.304775544" Jul 09 11:15:52.826271 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:52.826235 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48f05f33-fe91-4449-a9a8-a03a95d45006-registry-tls\") pod \"image-registry-699bccf5c4-zkf4x\" (UID: \"48f05f33-fe91-4449-a9a8-a03a95d45006\") " pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" Jul 09 11:15:52.826439 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:15:52.826388 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 11:15:52.826439 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:15:52.826407 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-699bccf5c4-zkf4x: secret "image-registry-tls" not found Jul 09 11:15:52.826510 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:15:52.826459 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48f05f33-fe91-4449-a9a8-a03a95d45006-registry-tls podName:48f05f33-fe91-4449-a9a8-a03a95d45006 nodeName:}" failed. No retries permitted until 2026-07-09 11:15:56.826443854 +0000 UTC m=+262.028762981 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/48f05f33-fe91-4449-a9a8-a03a95d45006-registry-tls") pod "image-registry-699bccf5c4-zkf4x" (UID: "48f05f33-fe91-4449-a9a8-a03a95d45006") : secret "image-registry-tls" not found Jul 09 11:15:53.081924 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:53.081840 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-98ghf_fccccfad-9a68-4bd9-a44f-af22e4339085/console-operator/0.log" Jul 09 11:15:53.081924 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:53.081888 2550 generic.go:356] "Generic (PLEG): container finished" podID="fccccfad-9a68-4bd9-a44f-af22e4339085" containerID="5b9c10651de630a1175dcdc942e6310c2f481f6d5cb91ab3a9ebb325ec9bab91" exitCode=255 Jul 09 11:15:53.082437 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:53.081965 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-98ghf" event={"ID":"fccccfad-9a68-4bd9-a44f-af22e4339085","Type":"ContainerDied","Data":"5b9c10651de630a1175dcdc942e6310c2f481f6d5cb91ab3a9ebb325ec9bab91"} Jul 09 11:15:53.082437 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:53.082214 2550 scope.go:117] "RemoveContainer" containerID="5b9c10651de630a1175dcdc942e6310c2f481f6d5cb91ab3a9ebb325ec9bab91" Jul 09 11:15:53.083551 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:53.083524 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-hr5fb" event={"ID":"511c08dd-43b1-485c-b275-df75627c0441","Type":"ContainerStarted","Data":"629acc6f7b3af80efe6bf9bb9afb9161299b939e83252dacabd6baeb3666714b"} Jul 09 11:15:53.255477 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:53.255430 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-hr5fb" podStartSLOduration=0.995971021 podStartE2EDuration="4.255414303s" totalImagesPullingTime="3.259443282s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 11:15:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:15:53.253496056 +0000 UTC m=+258.455815224" watchObservedRunningTime="2026-07-09 11:15:53.255414303 +0000 UTC m=+258.457733452" Jul 09 11:15:53.303580 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:53.303549 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-l7t8g"] Jul 09 11:15:53.305318 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:53.305301 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-l7t8g" Jul 09 11:15:53.308878 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:53.308858 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 09 11:15:53.308995 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:53.308931 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-4pfpp\"" Jul 09 11:15:53.311699 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:53.311679 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 09 11:15:53.326024 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:53.325995 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-l7t8g"] Jul 09 11:15:53.432270 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:53.432182 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62dtp\" (UniqueName: \"kubernetes.io/projected/bbe87e2e-7cb1-4822-a5f1-245774196a39-kube-api-access-62dtp\") pod \"migrator-d57d58cc9-l7t8g\" (UID: \"bbe87e2e-7cb1-4822-a5f1-245774196a39\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-l7t8g" Jul 09 11:15:53.533324 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:53.533296 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62dtp\" (UniqueName: \"kubernetes.io/projected/bbe87e2e-7cb1-4822-a5f1-245774196a39-kube-api-access-62dtp\") pod \"migrator-d57d58cc9-l7t8g\" (UID: \"bbe87e2e-7cb1-4822-a5f1-245774196a39\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-l7t8g" Jul 09 11:15:53.550232 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:53.550199 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-tmb84"] Jul 09 11:15:53.552112 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:53.552094 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tmb84" Jul 09 11:15:53.552859 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:53.552835 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-62dtp\" (UniqueName: \"kubernetes.io/projected/bbe87e2e-7cb1-4822-a5f1-245774196a39-kube-api-access-62dtp\") pod \"migrator-d57d58cc9-l7t8g\" (UID: \"bbe87e2e-7cb1-4822-a5f1-245774196a39\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-l7t8g" Jul 09 11:15:53.554763 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:53.554745 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 09 11:15:53.555054 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:53.555042 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 09 11:15:53.555145 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:53.555129 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 09 11:15:53.555468 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:53.555454 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8hzfb\"" Jul 09 11:15:53.555696 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:53.555681 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 09 11:15:53.574703 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:53.574678 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tmb84"] Jul 09 11:15:53.615608 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:53.615579 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-l7t8g" Jul 09 11:15:53.634749 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:53.634718 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/110148eb-6117-4ca1-b49c-911089247193-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tmb84\" (UID: \"110148eb-6117-4ca1-b49c-911089247193\") " pod="openshift-insights/insights-runtime-extractor-tmb84" Jul 09 11:15:53.634860 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:53.634794 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/110148eb-6117-4ca1-b49c-911089247193-crio-socket\") pod \"insights-runtime-extractor-tmb84\" (UID: \"110148eb-6117-4ca1-b49c-911089247193\") " pod="openshift-insights/insights-runtime-extractor-tmb84" Jul 09 11:15:53.634860 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:53.634849 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7549\" (UniqueName: \"kubernetes.io/projected/110148eb-6117-4ca1-b49c-911089247193-kube-api-access-t7549\") pod \"insights-runtime-extractor-tmb84\" (UID: \"110148eb-6117-4ca1-b49c-911089247193\") " pod="openshift-insights/insights-runtime-extractor-tmb84" Jul 09 11:15:53.634984 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:53.634921 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/110148eb-6117-4ca1-b49c-911089247193-data-volume\") pod \"insights-runtime-extractor-tmb84\" (UID: \"110148eb-6117-4ca1-b49c-911089247193\") " pod="openshift-insights/insights-runtime-extractor-tmb84" Jul 09 11:15:53.635035 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:53.635001 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/110148eb-6117-4ca1-b49c-911089247193-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tmb84\" (UID: \"110148eb-6117-4ca1-b49c-911089247193\") " pod="openshift-insights/insights-runtime-extractor-tmb84" Jul 09 11:15:53.736121 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:53.736077 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/110148eb-6117-4ca1-b49c-911089247193-crio-socket\") pod \"insights-runtime-extractor-tmb84\" (UID: \"110148eb-6117-4ca1-b49c-911089247193\") " pod="openshift-insights/insights-runtime-extractor-tmb84" Jul 09 11:15:53.736270 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:53.736130 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7549\" (UniqueName: \"kubernetes.io/projected/110148eb-6117-4ca1-b49c-911089247193-kube-api-access-t7549\") pod \"insights-runtime-extractor-tmb84\" (UID: \"110148eb-6117-4ca1-b49c-911089247193\") " pod="openshift-insights/insights-runtime-extractor-tmb84" Jul 09 11:15:53.736270 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:53.736148 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/110148eb-6117-4ca1-b49c-911089247193-data-volume\") pod \"insights-runtime-extractor-tmb84\" (UID: \"110148eb-6117-4ca1-b49c-911089247193\") " pod="openshift-insights/insights-runtime-extractor-tmb84" Jul 09 11:15:53.736270 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:53.736185 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/110148eb-6117-4ca1-b49c-911089247193-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tmb84\" (UID: \"110148eb-6117-4ca1-b49c-911089247193\") " pod="openshift-insights/insights-runtime-extractor-tmb84" Jul 09 11:15:53.736270 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:53.736210 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/110148eb-6117-4ca1-b49c-911089247193-crio-socket\") pod \"insights-runtime-extractor-tmb84\" (UID: \"110148eb-6117-4ca1-b49c-911089247193\") " pod="openshift-insights/insights-runtime-extractor-tmb84" Jul 09 11:15:53.736270 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:15:53.736246 2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 11:15:53.736453 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:15:53.736313 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/110148eb-6117-4ca1-b49c-911089247193-kube-rbac-proxy-cm podName:110148eb-6117-4ca1-b49c-911089247193 nodeName:}" failed. No retries permitted until 2026-07-09 11:15:54.23629464 +0000 UTC m=+259.438613789 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/110148eb-6117-4ca1-b49c-911089247193-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tmb84" (UID: "110148eb-6117-4ca1-b49c-911089247193") : configmap "kube-rbac-proxy" not found Jul 09 11:15:53.736453 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:53.736368 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/110148eb-6117-4ca1-b49c-911089247193-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tmb84\" (UID: \"110148eb-6117-4ca1-b49c-911089247193\") " pod="openshift-insights/insights-runtime-extractor-tmb84" Jul 09 11:15:53.736544 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:15:53.736465 2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 11:15:53.736544 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:15:53.736503 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/110148eb-6117-4ca1-b49c-911089247193-insights-runtime-extractor-tls podName:110148eb-6117-4ca1-b49c-911089247193 nodeName:}" failed. No retries permitted until 2026-07-09 11:15:54.236490487 +0000 UTC m=+259.438809619 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/110148eb-6117-4ca1-b49c-911089247193-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tmb84" (UID: "110148eb-6117-4ca1-b49c-911089247193") : secret "insights-runtime-extractor-tls" not found Jul 09 11:15:53.736544 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:53.736502 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/110148eb-6117-4ca1-b49c-911089247193-data-volume\") pod \"insights-runtime-extractor-tmb84\" (UID: \"110148eb-6117-4ca1-b49c-911089247193\") " pod="openshift-insights/insights-runtime-extractor-tmb84" Jul 09 11:15:53.748871 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:53.748849 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7549\" (UniqueName: \"kubernetes.io/projected/110148eb-6117-4ca1-b49c-911089247193-kube-api-access-t7549\") pod \"insights-runtime-extractor-tmb84\" (UID: \"110148eb-6117-4ca1-b49c-911089247193\") " pod="openshift-insights/insights-runtime-extractor-tmb84" Jul 09 11:15:53.752043 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:53.752023 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-l7t8g"] Jul 09 11:15:53.754464 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:15:53.754441 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbe87e2e_7cb1_4822_a5f1_245774196a39.slice/crio-6d73074d292663d5bad0904b007be78ff80486558f8d2a97135a468516b099a3 WatchSource:0}: Error finding container 6d73074d292663d5bad0904b007be78ff80486558f8d2a97135a468516b099a3: Status 404 returned error can't find the container with id 6d73074d292663d5bad0904b007be78ff80486558f8d2a97135a468516b099a3 Jul 09 11:15:54.088388 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:54.088324 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-98ghf_fccccfad-9a68-4bd9-a44f-af22e4339085/console-operator/1.log" Jul 09 11:15:54.088832 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:54.088759 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-98ghf_fccccfad-9a68-4bd9-a44f-af22e4339085/console-operator/0.log" Jul 09 11:15:54.088832 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:54.088800 2550 generic.go:356] "Generic (PLEG): container finished" podID="fccccfad-9a68-4bd9-a44f-af22e4339085" containerID="944eeaf7283a02ce9a03130c8e1f5f564f6d08359b69f8144b7e13421dc73359" exitCode=255 Jul 09 11:15:54.088939 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:54.088878 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-98ghf" event={"ID":"fccccfad-9a68-4bd9-a44f-af22e4339085","Type":"ContainerDied","Data":"944eeaf7283a02ce9a03130c8e1f5f564f6d08359b69f8144b7e13421dc73359"} Jul 09 11:15:54.088939 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:54.088926 2550 scope.go:117] "RemoveContainer" containerID="5b9c10651de630a1175dcdc942e6310c2f481f6d5cb91ab3a9ebb325ec9bab91" Jul 09 11:15:54.089164 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:54.089141 2550 scope.go:117] "RemoveContainer" containerID="944eeaf7283a02ce9a03130c8e1f5f564f6d08359b69f8144b7e13421dc73359" Jul 09 11:15:54.089382 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:15:54.089358 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-98ghf_openshift-console-operator(fccccfad-9a68-4bd9-a44f-af22e4339085)\"" pod="openshift-console-operator/console-operator-68b757fb67-98ghf" podUID="fccccfad-9a68-4bd9-a44f-af22e4339085" Jul 09 11:15:54.090005 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:54.089984 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-l7t8g" event={"ID":"bbe87e2e-7cb1-4822-a5f1-245774196a39","Type":"ContainerStarted","Data":"6d73074d292663d5bad0904b007be78ff80486558f8d2a97135a468516b099a3"} Jul 09 11:15:54.241015 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:54.240983 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/110148eb-6117-4ca1-b49c-911089247193-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tmb84\" (UID: \"110148eb-6117-4ca1-b49c-911089247193\") " pod="openshift-insights/insights-runtime-extractor-tmb84" Jul 09 11:15:54.241152 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:54.241033 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/110148eb-6117-4ca1-b49c-911089247193-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tmb84\" (UID: \"110148eb-6117-4ca1-b49c-911089247193\") " pod="openshift-insights/insights-runtime-extractor-tmb84" Jul 09 11:15:54.241152 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:15:54.241103 2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 11:15:54.241152 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:15:54.241130 2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 11:15:54.241265 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:15:54.241167 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/110148eb-6117-4ca1-b49c-911089247193-kube-rbac-proxy-cm podName:110148eb-6117-4ca1-b49c-911089247193 nodeName:}" failed. No retries permitted until 2026-07-09 11:15:55.241150354 +0000 UTC m=+260.443469480 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/110148eb-6117-4ca1-b49c-911089247193-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tmb84" (UID: "110148eb-6117-4ca1-b49c-911089247193") : configmap "kube-rbac-proxy" not found Jul 09 11:15:54.241265 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:15:54.241183 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/110148eb-6117-4ca1-b49c-911089247193-insights-runtime-extractor-tls podName:110148eb-6117-4ca1-b49c-911089247193 nodeName:}" failed. No retries permitted until 2026-07-09 11:15:55.241176467 +0000 UTC m=+260.443495594 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/110148eb-6117-4ca1-b49c-911089247193-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tmb84" (UID: "110148eb-6117-4ca1-b49c-911089247193") : secret "insights-runtime-extractor-tls" not found Jul 09 11:15:55.094666 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:55.094640 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-98ghf_fccccfad-9a68-4bd9-a44f-af22e4339085/console-operator/1.log" Jul 09 11:15:55.095010 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:55.094952 2550 scope.go:117] "RemoveContainer" containerID="944eeaf7283a02ce9a03130c8e1f5f564f6d08359b69f8144b7e13421dc73359" Jul 09 11:15:55.095105 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:15:55.095088 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-98ghf_openshift-console-operator(fccccfad-9a68-4bd9-a44f-af22e4339085)\"" pod="openshift-console-operator/console-operator-68b757fb67-98ghf" podUID="fccccfad-9a68-4bd9-a44f-af22e4339085" Jul 09 11:15:55.250268 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:55.250234 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/110148eb-6117-4ca1-b49c-911089247193-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tmb84\" (UID: \"110148eb-6117-4ca1-b49c-911089247193\") " pod="openshift-insights/insights-runtime-extractor-tmb84" Jul 09 11:15:55.250458 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:55.250301 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/110148eb-6117-4ca1-b49c-911089247193-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tmb84\" (UID: \"110148eb-6117-4ca1-b49c-911089247193\") " pod="openshift-insights/insights-runtime-extractor-tmb84" Jul 09 11:15:55.250458 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:15:55.250374 2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 11:15:55.250458 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:15:55.250445 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/110148eb-6117-4ca1-b49c-911089247193-kube-rbac-proxy-cm podName:110148eb-6117-4ca1-b49c-911089247193 nodeName:}" failed. No retries permitted until 2026-07-09 11:15:57.250429727 +0000 UTC m=+262.452748857 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/110148eb-6117-4ca1-b49c-911089247193-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tmb84" (UID: "110148eb-6117-4ca1-b49c-911089247193") : configmap "kube-rbac-proxy" not found Jul 09 11:15:55.250458 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:15:55.250446 2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 11:15:55.250601 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:15:55.250493 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/110148eb-6117-4ca1-b49c-911089247193-insights-runtime-extractor-tls podName:110148eb-6117-4ca1-b49c-911089247193 nodeName:}" failed. No retries permitted until 2026-07-09 11:15:57.250481019 +0000 UTC m=+262.452800146 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/110148eb-6117-4ca1-b49c-911089247193-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tmb84" (UID: "110148eb-6117-4ca1-b49c-911089247193") : secret "insights-runtime-extractor-tls" not found Jul 09 11:15:56.099660 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:56.099624 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-l7t8g" event={"ID":"bbe87e2e-7cb1-4822-a5f1-245774196a39","Type":"ContainerStarted","Data":"d794f1bc54c09cfa6c2940ac1184b4143a0ba03a16ef43ac940fe858f7b081f0"} Jul 09 11:15:56.100028 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:56.099707 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-l7t8g" event={"ID":"bbe87e2e-7cb1-4822-a5f1-245774196a39","Type":"ContainerStarted","Data":"73065db773c03f783a3ee84eac893b0d85cabd31458c193b8f2b2faf77ca438b"} Jul 09 11:15:56.122510 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:56.122460 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-l7t8g" podStartSLOduration=1.8116989879999998 podStartE2EDuration="3.122444001s" totalImagesPullingTime="1.310745013s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 11:15:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:15:56.120943604 +0000 UTC m=+261.323262753" watchObservedRunningTime="2026-07-09 11:15:56.122444001 +0000 UTC m=+261.324763149" Jul 09 11:15:56.490231 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:56.490151 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wsfmn_8117b6cb-dfab-462d-ae86-50a342e88a9a/dns-node-resolver/0.log" Jul 09 11:15:56.864628 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:56.864595 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48f05f33-fe91-4449-a9a8-a03a95d45006-registry-tls\") pod \"image-registry-699bccf5c4-zkf4x\" (UID: \"48f05f33-fe91-4449-a9a8-a03a95d45006\") " pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" Jul 09 11:15:56.864805 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:15:56.864749 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 11:15:56.864805 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:15:56.864766 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-699bccf5c4-zkf4x: secret "image-registry-tls" not found Jul 09 11:15:56.864897 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:15:56.864823 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48f05f33-fe91-4449-a9a8-a03a95d45006-registry-tls podName:48f05f33-fe91-4449-a9a8-a03a95d45006 nodeName:}" failed. No retries permitted until 2026-07-09 11:16:04.864806187 +0000 UTC m=+270.067125328 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/48f05f33-fe91-4449-a9a8-a03a95d45006-registry-tls") pod "image-registry-699bccf5c4-zkf4x" (UID: "48f05f33-fe91-4449-a9a8-a03a95d45006") : secret "image-registry-tls" not found Jul 09 11:15:57.087284 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:57.087258 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ppz4r_8207614d-56bb-42cd-a5e6-d7fd5da8dac1/node-ca/0.log" Jul 09 11:15:57.268394 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:57.268305 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/110148eb-6117-4ca1-b49c-911089247193-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tmb84\" (UID: \"110148eb-6117-4ca1-b49c-911089247193\") " pod="openshift-insights/insights-runtime-extractor-tmb84" Jul 09 11:15:57.268828 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:15:57.268427 2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 11:15:57.268828 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:57.268452 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/110148eb-6117-4ca1-b49c-911089247193-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tmb84\" (UID: \"110148eb-6117-4ca1-b49c-911089247193\") " pod="openshift-insights/insights-runtime-extractor-tmb84" Jul 09 11:15:57.268828 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:15:57.268486 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/110148eb-6117-4ca1-b49c-911089247193-kube-rbac-proxy-cm podName:110148eb-6117-4ca1-b49c-911089247193 nodeName:}" failed. No retries permitted until 2026-07-09 11:16:01.268471071 +0000 UTC m=+266.470790198 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/110148eb-6117-4ca1-b49c-911089247193-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tmb84" (UID: "110148eb-6117-4ca1-b49c-911089247193") : configmap "kube-rbac-proxy" not found Jul 09 11:15:57.268828 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:15:57.268537 2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 11:15:57.268828 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:15:57.268581 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/110148eb-6117-4ca1-b49c-911089247193-insights-runtime-extractor-tls podName:110148eb-6117-4ca1-b49c-911089247193 nodeName:}" failed. No retries permitted until 2026-07-09 11:16:01.268569353 +0000 UTC m=+266.470888479 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/110148eb-6117-4ca1-b49c-911089247193-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tmb84" (UID: "110148eb-6117-4ca1-b49c-911089247193") : secret "insights-runtime-extractor-tls" not found Jul 09 11:15:58.088938 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:58.088911 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-96bj9_330acdbe-92a6-4ad6-b11a-e941fc7217fc/serve-healthcheck-canary/0.log" Jul 09 11:15:58.507267 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:58.507190 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-l7t8g_bbe87e2e-7cb1-4822-a5f1-245774196a39/migrator/0.log" Jul 09 11:15:58.707833 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:58.707803 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-l7t8g_bbe87e2e-7cb1-4822-a5f1-245774196a39/graceful-termination/0.log" Jul 09 11:15:59.547651 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:59.547618 2550 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-98ghf" Jul 09 11:15:59.548146 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:59.547676 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-98ghf" Jul 09 11:15:59.548146 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:15:59.548106 2550 scope.go:117] "RemoveContainer" containerID="944eeaf7283a02ce9a03130c8e1f5f564f6d08359b69f8144b7e13421dc73359" Jul 09 11:15:59.548310 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:15:59.548286 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-98ghf_openshift-console-operator(fccccfad-9a68-4bd9-a44f-af22e4339085)\"" pod="openshift-console-operator/console-operator-68b757fb67-98ghf" podUID="fccccfad-9a68-4bd9-a44f-af22e4339085" Jul 09 11:16:01.300219 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:01.300189 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/110148eb-6117-4ca1-b49c-911089247193-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tmb84\" (UID: \"110148eb-6117-4ca1-b49c-911089247193\") " pod="openshift-insights/insights-runtime-extractor-tmb84" Jul 09 11:16:01.300668 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:01.300294 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/110148eb-6117-4ca1-b49c-911089247193-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tmb84\" (UID: \"110148eb-6117-4ca1-b49c-911089247193\") " pod="openshift-insights/insights-runtime-extractor-tmb84" Jul 09 11:16:01.300668 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:16:01.300364 2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 11:16:01.300668 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:16:01.300423 2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 11:16:01.300668 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:16:01.300442 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/110148eb-6117-4ca1-b49c-911089247193-insights-runtime-extractor-tls podName:110148eb-6117-4ca1-b49c-911089247193 nodeName:}" failed. No retries permitted until 2026-07-09 11:16:09.300421374 +0000 UTC m=+274.502740505 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/110148eb-6117-4ca1-b49c-911089247193-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tmb84" (UID: "110148eb-6117-4ca1-b49c-911089247193") : secret "insights-runtime-extractor-tls" not found Jul 09 11:16:01.300668 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:16:01.300462 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/110148eb-6117-4ca1-b49c-911089247193-kube-rbac-proxy-cm podName:110148eb-6117-4ca1-b49c-911089247193 nodeName:}" failed. No retries permitted until 2026-07-09 11:16:09.300450887 +0000 UTC m=+274.502770023 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/110148eb-6117-4ca1-b49c-911089247193-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tmb84" (UID: "110148eb-6117-4ca1-b49c-911089247193") : configmap "kube-rbac-proxy" not found Jul 09 11:16:04.931098 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:04.931060 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48f05f33-fe91-4449-a9a8-a03a95d45006-registry-tls\") pod \"image-registry-699bccf5c4-zkf4x\" (UID: \"48f05f33-fe91-4449-a9a8-a03a95d45006\") " pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" Jul 09 11:16:04.933455 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:04.933435 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48f05f33-fe91-4449-a9a8-a03a95d45006-registry-tls\") pod \"image-registry-699bccf5c4-zkf4x\" (UID: \"48f05f33-fe91-4449-a9a8-a03a95d45006\") " pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" Jul 09 11:16:05.046258 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:05.046221 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" Jul 09 11:16:05.179953 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:05.179931 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-699bccf5c4-zkf4x"] Jul 09 11:16:05.182446 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:16:05.182381 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48f05f33_fe91_4449_a9a8_a03a95d45006.slice/crio-39d85b45cd98ae697cbaab02c48c95b88eb5192d76b57468adee16fc48f2d927 WatchSource:0}: Error finding container 39d85b45cd98ae697cbaab02c48c95b88eb5192d76b57468adee16fc48f2d927: Status 404 returned error can't find the container with id 39d85b45cd98ae697cbaab02c48c95b88eb5192d76b57468adee16fc48f2d927 Jul 09 11:16:06.130662 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:06.130629 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" event={"ID":"48f05f33-fe91-4449-a9a8-a03a95d45006","Type":"ContainerStarted","Data":"8ff900be7ab964099a8fb5d4a39374f0277dea63a071fe02c02b75c6f8ee7882"} Jul 09 11:16:06.131050 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:06.130677 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" event={"ID":"48f05f33-fe91-4449-a9a8-a03a95d45006","Type":"ContainerStarted","Data":"39d85b45cd98ae697cbaab02c48c95b88eb5192d76b57468adee16fc48f2d927"} Jul 09 11:16:06.131050 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:06.130751 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" Jul 09 11:16:09.367608 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:09.367566 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/110148eb-6117-4ca1-b49c-911089247193-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tmb84\" (UID: \"110148eb-6117-4ca1-b49c-911089247193\") " pod="openshift-insights/insights-runtime-extractor-tmb84" Jul 09 11:16:09.367608 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:09.367620 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/110148eb-6117-4ca1-b49c-911089247193-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tmb84\" (UID: \"110148eb-6117-4ca1-b49c-911089247193\") " pod="openshift-insights/insights-runtime-extractor-tmb84" Jul 09 11:16:09.368180 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:16:09.367715 2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 11:16:09.368180 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:16:09.367780 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/110148eb-6117-4ca1-b49c-911089247193-insights-runtime-extractor-tls podName:110148eb-6117-4ca1-b49c-911089247193 nodeName:}" failed. No retries permitted until 2026-07-09 11:16:25.367767141 +0000 UTC m=+290.570086269 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/110148eb-6117-4ca1-b49c-911089247193-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tmb84" (UID: "110148eb-6117-4ca1-b49c-911089247193") : secret "insights-runtime-extractor-tls" not found Jul 09 11:16:09.368289 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:09.368218 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/110148eb-6117-4ca1-b49c-911089247193-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tmb84\" (UID: \"110148eb-6117-4ca1-b49c-911089247193\") " pod="openshift-insights/insights-runtime-extractor-tmb84" Jul 09 11:16:12.294176 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:12.294138 2550 scope.go:117] "RemoveContainer" containerID="944eeaf7283a02ce9a03130c8e1f5f564f6d08359b69f8144b7e13421dc73359" Jul 09 11:16:13.153061 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:13.153034 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-98ghf_fccccfad-9a68-4bd9-a44f-af22e4339085/console-operator/2.log" Jul 09 11:16:13.153384 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:13.153372 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-98ghf_fccccfad-9a68-4bd9-a44f-af22e4339085/console-operator/1.log" Jul 09 11:16:13.153460 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:13.153399 2550 generic.go:356] "Generic (PLEG): container finished" podID="fccccfad-9a68-4bd9-a44f-af22e4339085" containerID="b818cbf1a28804f91441f4359e6e4f58172947388d1c62e84f18a24cd2e16dee" exitCode=255 Jul 09 11:16:13.153460 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:13.153440 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-98ghf" event={"ID":"fccccfad-9a68-4bd9-a44f-af22e4339085","Type":"ContainerDied","Data":"b818cbf1a28804f91441f4359e6e4f58172947388d1c62e84f18a24cd2e16dee"} Jul 09 11:16:13.153531 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:13.153480 2550 scope.go:117] "RemoveContainer" containerID="944eeaf7283a02ce9a03130c8e1f5f564f6d08359b69f8144b7e13421dc73359" Jul 09 11:16:13.153882 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:13.153861 2550 scope.go:117] "RemoveContainer" containerID="b818cbf1a28804f91441f4359e6e4f58172947388d1c62e84f18a24cd2e16dee" Jul 09 11:16:13.154073 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:16:13.154053 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-98ghf_openshift-console-operator(fccccfad-9a68-4bd9-a44f-af22e4339085)\"" pod="openshift-console-operator/console-operator-68b757fb67-98ghf" podUID="fccccfad-9a68-4bd9-a44f-af22e4339085" Jul 09 11:16:13.193306 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:13.193260 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" podStartSLOduration=24.193245228 podStartE2EDuration="24.193245228s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 11:15:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:16:06.19260765 +0000 UTC m=+271.394926798" watchObservedRunningTime="2026-07-09 11:16:13.193245228 +0000 UTC m=+278.395564376" Jul 09 11:16:14.158010 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:14.157983 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-98ghf_fccccfad-9a68-4bd9-a44f-af22e4339085/console-operator/2.log" Jul 09 11:16:15.839316 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:16:15.839268 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-vsfwj" podUID="127cd1f0-bf8a-4bae-94fa-d21833536f5f" Jul 09 11:16:16.164046 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:16.163974 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vsfwj" Jul 09 11:16:19.548412 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:19.548367 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-98ghf" Jul 09 11:16:19.548911 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:19.548434 2550 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-98ghf" Jul 09 11:16:19.548911 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:19.548744 2550 scope.go:117] "RemoveContainer" containerID="b818cbf1a28804f91441f4359e6e4f58172947388d1c62e84f18a24cd2e16dee" Jul 09 11:16:19.548911 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:16:19.548881 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-98ghf_openshift-console-operator(fccccfad-9a68-4bd9-a44f-af22e4339085)\"" pod="openshift-console-operator/console-operator-68b757fb67-98ghf" podUID="fccccfad-9a68-4bd9-a44f-af22e4339085" Jul 09 11:16:19.746113 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:19.746082 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/127cd1f0-bf8a-4bae-94fa-d21833536f5f-metrics-tls\") pod \"dns-default-vsfwj\" (UID: \"127cd1f0-bf8a-4bae-94fa-d21833536f5f\") " pod="openshift-dns/dns-default-vsfwj" Jul 09 11:16:19.748392 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:19.748356 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/127cd1f0-bf8a-4bae-94fa-d21833536f5f-metrics-tls\") pod \"dns-default-vsfwj\" (UID: \"127cd1f0-bf8a-4bae-94fa-d21833536f5f\") " pod="openshift-dns/dns-default-vsfwj" Jul 09 11:16:19.767985 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:19.767959 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6pmqk\"" Jul 09 11:16:19.776323 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:19.776302 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vsfwj" Jul 09 11:16:19.911838 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:19.911792 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vsfwj"] Jul 09 11:16:19.914590 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:16:19.914565 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod127cd1f0_bf8a_4bae_94fa_d21833536f5f.slice/crio-8b6962d4c1bd771799d7b6d2158da1b83b17ec47bf3810190c13322ed0115e61 WatchSource:0}: Error finding container 8b6962d4c1bd771799d7b6d2158da1b83b17ec47bf3810190c13322ed0115e61: Status 404 returned error can't find the container with id 8b6962d4c1bd771799d7b6d2158da1b83b17ec47bf3810190c13322ed0115e61 Jul 09 11:16:20.176797 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:20.176714 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vsfwj" event={"ID":"127cd1f0-bf8a-4bae-94fa-d21833536f5f","Type":"ContainerStarted","Data":"8b6962d4c1bd771799d7b6d2158da1b83b17ec47bf3810190c13322ed0115e61"} Jul 09 11:16:20.687604 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:20.687567 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-699bccf5c4-zkf4x"] Jul 09 11:16:20.693450 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:20.693399 2550 patch_prober.go:28] interesting pod/image-registry-699bccf5c4-zkf4x container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 09 11:16:20.693636 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:20.693467 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" podUID="48f05f33-fe91-4449-a9a8-a03a95d45006" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 11:16:20.762527 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:20.762492 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7448f9db94-86hhs"] Jul 09 11:16:20.767779 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:20.767760 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7448f9db94-86hhs" Jul 09 11:16:20.793430 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:20.793386 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7448f9db94-86hhs"] Jul 09 11:16:20.854814 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:20.854779 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9ttw\" (UniqueName: \"kubernetes.io/projected/02acda78-f3fd-42ff-8e98-5f9131421fbc-kube-api-access-l9ttw\") pod \"image-registry-7448f9db94-86hhs\" (UID: \"02acda78-f3fd-42ff-8e98-5f9131421fbc\") " pod="openshift-image-registry/image-registry-7448f9db94-86hhs" Jul 09 11:16:20.854991 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:20.854823 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/02acda78-f3fd-42ff-8e98-5f9131421fbc-registry-tls\") pod \"image-registry-7448f9db94-86hhs\" (UID: \"02acda78-f3fd-42ff-8e98-5f9131421fbc\") " pod="openshift-image-registry/image-registry-7448f9db94-86hhs" Jul 09 11:16:20.854991 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:20.854864 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/02acda78-f3fd-42ff-8e98-5f9131421fbc-installation-pull-secrets\") pod \"image-registry-7448f9db94-86hhs\" (UID: \"02acda78-f3fd-42ff-8e98-5f9131421fbc\") " pod="openshift-image-registry/image-registry-7448f9db94-86hhs" Jul 09 11:16:20.854991 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:20.854914 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/02acda78-f3fd-42ff-8e98-5f9131421fbc-ca-trust-extracted\") pod \"image-registry-7448f9db94-86hhs\" (UID: \"02acda78-f3fd-42ff-8e98-5f9131421fbc\") " pod="openshift-image-registry/image-registry-7448f9db94-86hhs" Jul 09 11:16:20.854991 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:20.854938 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/02acda78-f3fd-42ff-8e98-5f9131421fbc-registry-certificates\") pod \"image-registry-7448f9db94-86hhs\" (UID: \"02acda78-f3fd-42ff-8e98-5f9131421fbc\") " pod="openshift-image-registry/image-registry-7448f9db94-86hhs" Jul 09 11:16:20.854991 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:20.854965 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/02acda78-f3fd-42ff-8e98-5f9131421fbc-trusted-ca\") pod \"image-registry-7448f9db94-86hhs\" (UID: \"02acda78-f3fd-42ff-8e98-5f9131421fbc\") " pod="openshift-image-registry/image-registry-7448f9db94-86hhs" Jul 09 11:16:20.855176 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:20.855021 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/02acda78-f3fd-42ff-8e98-5f9131421fbc-bound-sa-token\") pod \"image-registry-7448f9db94-86hhs\" (UID: \"02acda78-f3fd-42ff-8e98-5f9131421fbc\") " pod="openshift-image-registry/image-registry-7448f9db94-86hhs" Jul 09 11:16:20.855176 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:20.855069 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/02acda78-f3fd-42ff-8e98-5f9131421fbc-image-registry-private-configuration\") pod \"image-registry-7448f9db94-86hhs\" (UID: \"02acda78-f3fd-42ff-8e98-5f9131421fbc\") " pod="openshift-image-registry/image-registry-7448f9db94-86hhs" Jul 09 11:16:20.955874 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:20.955781 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/02acda78-f3fd-42ff-8e98-5f9131421fbc-registry-tls\") pod \"image-registry-7448f9db94-86hhs\" (UID: \"02acda78-f3fd-42ff-8e98-5f9131421fbc\") " pod="openshift-image-registry/image-registry-7448f9db94-86hhs" Jul 09 11:16:20.955874 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:20.955850 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/02acda78-f3fd-42ff-8e98-5f9131421fbc-installation-pull-secrets\") pod \"image-registry-7448f9db94-86hhs\" (UID: \"02acda78-f3fd-42ff-8e98-5f9131421fbc\") " pod="openshift-image-registry/image-registry-7448f9db94-86hhs" Jul 09 11:16:20.956105 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:20.955904 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/02acda78-f3fd-42ff-8e98-5f9131421fbc-ca-trust-extracted\") pod \"image-registry-7448f9db94-86hhs\" (UID: \"02acda78-f3fd-42ff-8e98-5f9131421fbc\") " pod="openshift-image-registry/image-registry-7448f9db94-86hhs" Jul 09 11:16:20.956105 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:20.955934 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/02acda78-f3fd-42ff-8e98-5f9131421fbc-registry-certificates\") pod \"image-registry-7448f9db94-86hhs\" (UID: \"02acda78-f3fd-42ff-8e98-5f9131421fbc\") " pod="openshift-image-registry/image-registry-7448f9db94-86hhs" Jul 09 11:16:20.956105 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:20.955964 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/02acda78-f3fd-42ff-8e98-5f9131421fbc-trusted-ca\") pod \"image-registry-7448f9db94-86hhs\" (UID: \"02acda78-f3fd-42ff-8e98-5f9131421fbc\") " pod="openshift-image-registry/image-registry-7448f9db94-86hhs" Jul 09 11:16:20.956105 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:20.955999 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/02acda78-f3fd-42ff-8e98-5f9131421fbc-bound-sa-token\") pod \"image-registry-7448f9db94-86hhs\" (UID: \"02acda78-f3fd-42ff-8e98-5f9131421fbc\") " pod="openshift-image-registry/image-registry-7448f9db94-86hhs" Jul 09 11:16:20.956105 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:20.956035 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/02acda78-f3fd-42ff-8e98-5f9131421fbc-image-registry-private-configuration\") pod \"image-registry-7448f9db94-86hhs\" (UID: \"02acda78-f3fd-42ff-8e98-5f9131421fbc\") " pod="openshift-image-registry/image-registry-7448f9db94-86hhs" Jul 09 11:16:20.956105 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:20.956070 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9ttw\" (UniqueName: \"kubernetes.io/projected/02acda78-f3fd-42ff-8e98-5f9131421fbc-kube-api-access-l9ttw\") pod \"image-registry-7448f9db94-86hhs\" (UID: \"02acda78-f3fd-42ff-8e98-5f9131421fbc\") " pod="openshift-image-registry/image-registry-7448f9db94-86hhs" Jul 09 11:16:20.957570 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:20.956943 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/02acda78-f3fd-42ff-8e98-5f9131421fbc-ca-trust-extracted\") pod \"image-registry-7448f9db94-86hhs\" (UID: \"02acda78-f3fd-42ff-8e98-5f9131421fbc\") " pod="openshift-image-registry/image-registry-7448f9db94-86hhs" Jul 09 11:16:20.957570 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:20.957198 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/02acda78-f3fd-42ff-8e98-5f9131421fbc-trusted-ca\") pod \"image-registry-7448f9db94-86hhs\" (UID: \"02acda78-f3fd-42ff-8e98-5f9131421fbc\") " pod="openshift-image-registry/image-registry-7448f9db94-86hhs" Jul 09 11:16:20.957570 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:20.957533 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/02acda78-f3fd-42ff-8e98-5f9131421fbc-registry-certificates\") pod \"image-registry-7448f9db94-86hhs\" (UID: \"02acda78-f3fd-42ff-8e98-5f9131421fbc\") " pod="openshift-image-registry/image-registry-7448f9db94-86hhs" Jul 09 11:16:20.958899 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:20.958872 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/02acda78-f3fd-42ff-8e98-5f9131421fbc-registry-tls\") pod \"image-registry-7448f9db94-86hhs\" (UID: \"02acda78-f3fd-42ff-8e98-5f9131421fbc\") " pod="openshift-image-registry/image-registry-7448f9db94-86hhs" Jul 09 11:16:20.959016 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:20.958946 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/02acda78-f3fd-42ff-8e98-5f9131421fbc-installation-pull-secrets\") pod \"image-registry-7448f9db94-86hhs\" (UID: \"02acda78-f3fd-42ff-8e98-5f9131421fbc\") " pod="openshift-image-registry/image-registry-7448f9db94-86hhs" Jul 09 11:16:20.959082 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:20.959052 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/02acda78-f3fd-42ff-8e98-5f9131421fbc-image-registry-private-configuration\") pod \"image-registry-7448f9db94-86hhs\" (UID: \"02acda78-f3fd-42ff-8e98-5f9131421fbc\") " pod="openshift-image-registry/image-registry-7448f9db94-86hhs" Jul 09 11:16:20.986596 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:20.986569 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9ttw\" (UniqueName: \"kubernetes.io/projected/02acda78-f3fd-42ff-8e98-5f9131421fbc-kube-api-access-l9ttw\") pod \"image-registry-7448f9db94-86hhs\" (UID: \"02acda78-f3fd-42ff-8e98-5f9131421fbc\") " pod="openshift-image-registry/image-registry-7448f9db94-86hhs" Jul 09 11:16:21.007697 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:21.007665 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/02acda78-f3fd-42ff-8e98-5f9131421fbc-bound-sa-token\") pod \"image-registry-7448f9db94-86hhs\" (UID: \"02acda78-f3fd-42ff-8e98-5f9131421fbc\") " pod="openshift-image-registry/image-registry-7448f9db94-86hhs" Jul 09 11:16:21.079829 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:21.079793 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7448f9db94-86hhs" Jul 09 11:16:21.254117 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:21.254038 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7448f9db94-86hhs"] Jul 09 11:16:21.621639 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:16:21.621606 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02acda78_f3fd_42ff_8e98_5f9131421fbc.slice/crio-527b7fd5e52d6fb1be3265c67f00a01f74af32521bbc33b798c5f0a427725319 WatchSource:0}: Error finding container 527b7fd5e52d6fb1be3265c67f00a01f74af32521bbc33b798c5f0a427725319: Status 404 returned error can't find the container with id 527b7fd5e52d6fb1be3265c67f00a01f74af32521bbc33b798c5f0a427725319 Jul 09 11:16:22.185615 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:22.185575 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7448f9db94-86hhs" event={"ID":"02acda78-f3fd-42ff-8e98-5f9131421fbc","Type":"ContainerStarted","Data":"2d1ca41f095430809ba9c8bdc1ecef42a73397d7f045ea378e1fc0c00fea6669"} Jul 09 11:16:22.186095 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:22.185681 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7448f9db94-86hhs" Jul 09 11:16:22.186095 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:22.185704 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7448f9db94-86hhs" event={"ID":"02acda78-f3fd-42ff-8e98-5f9131421fbc","Type":"ContainerStarted","Data":"527b7fd5e52d6fb1be3265c67f00a01f74af32521bbc33b798c5f0a427725319"} Jul 09 11:16:22.187237 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:22.187213 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vsfwj" event={"ID":"127cd1f0-bf8a-4bae-94fa-d21833536f5f","Type":"ContainerStarted","Data":"d934eb974d9555ff465599260466f90785703336a3f1d32e85d8c43d0b576beb"} Jul 09 11:16:22.187340 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:22.187259 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vsfwj" event={"ID":"127cd1f0-bf8a-4bae-94fa-d21833536f5f","Type":"ContainerStarted","Data":"30bec00cf50a67a8fce1c4dceab627c65984cd875c431f55d319b3568d587a1d"} Jul 09 11:16:22.187391 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:22.187371 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vsfwj" Jul 09 11:16:22.241960 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:22.241910 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7448f9db94-86hhs" podStartSLOduration=2.241894878 podStartE2EDuration="2.241894878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 11:16:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:16:22.241205 +0000 UTC m=+287.443524167" watchObservedRunningTime="2026-07-09 11:16:22.241894878 +0000 UTC m=+287.444214026" Jul 09 11:16:22.285430 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:22.285382 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-vsfwj" podStartSLOduration=251.529772782 podStartE2EDuration="4m13.285364783s" totalImagesPullingTime="1.755592001s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 11:12:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:16:22.281749396 +0000 UTC m=+287.484068544" watchObservedRunningTime="2026-07-09 11:16:22.285364783 +0000 UTC m=+287.487683931" Jul 09 11:16:25.394599 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:25.394561 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/110148eb-6117-4ca1-b49c-911089247193-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tmb84\" (UID: \"110148eb-6117-4ca1-b49c-911089247193\") " pod="openshift-insights/insights-runtime-extractor-tmb84" Jul 09 11:16:25.396758 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:25.396742 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/110148eb-6117-4ca1-b49c-911089247193-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tmb84\" (UID: \"110148eb-6117-4ca1-b49c-911089247193\") " pod="openshift-insights/insights-runtime-extractor-tmb84" Jul 09 11:16:25.666094 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:25.666008 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tmb84" Jul 09 11:16:25.837042 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:25.837017 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tmb84"] Jul 09 11:16:25.839448 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:16:25.839423 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod110148eb_6117_4ca1_b49c_911089247193.slice/crio-96d6beb111cc2496122dce04392dac6c5d7e2b3e0a5e4c767dae3e62fb316255 WatchSource:0}: Error finding container 96d6beb111cc2496122dce04392dac6c5d7e2b3e0a5e4c767dae3e62fb316255: Status 404 returned error can't find the container with id 96d6beb111cc2496122dce04392dac6c5d7e2b3e0a5e4c767dae3e62fb316255 Jul 09 11:16:26.200804 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:26.200774 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tmb84" event={"ID":"110148eb-6117-4ca1-b49c-911089247193","Type":"ContainerStarted","Data":"96d6beb111cc2496122dce04392dac6c5d7e2b3e0a5e4c767dae3e62fb316255"} Jul 09 11:16:27.294494 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:16:27.294441 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-6bvn2" podUID="91432ff0-fa58-460b-a6d6-3a14a9f5f295" Jul 09 11:16:30.216618 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:30.216582 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tmb84" event={"ID":"110148eb-6117-4ca1-b49c-911089247193","Type":"ContainerStarted","Data":"eee4832b6744a37fe86b08e40cc6c68840d26e49ab5ea48230352ecb71771f18"} Jul 09 11:16:30.692861 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:30.692824 2550 patch_prober.go:28] interesting pod/image-registry-699bccf5c4-zkf4x container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 09 11:16:30.693045 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:30.692909 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" podUID="48f05f33-fe91-4449-a9a8-a03a95d45006" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 11:16:31.222853 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:31.222799 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tmb84" event={"ID":"110148eb-6117-4ca1-b49c-911089247193","Type":"ContainerStarted","Data":"f27ebd5c3155cbb032fb97a49fcf2ef44d6f9f0c7cc17d519dd2e283b859e375"} Jul 09 11:16:32.192817 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:32.192780 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vsfwj" Jul 09 11:16:32.294408 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:32.294379 2550 scope.go:117] "RemoveContainer" containerID="b818cbf1a28804f91441f4359e6e4f58172947388d1c62e84f18a24cd2e16dee" Jul 09 11:16:32.294896 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:16:32.294541 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-98ghf_openshift-console-operator(fccccfad-9a68-4bd9-a44f-af22e4339085)\"" pod="openshift-console-operator/console-operator-68b757fb67-98ghf" podUID="fccccfad-9a68-4bd9-a44f-af22e4339085" Jul 09 11:16:33.232636 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:33.232602 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tmb84" event={"ID":"110148eb-6117-4ca1-b49c-911089247193","Type":"ContainerStarted","Data":"c9f21d603fba42319d368b0a72ca5d71ef7a5f26c6e5cffd4334b6e7c39a5798"} Jul 09 11:16:33.296627 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:33.296575 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-tmb84" podStartSLOduration=33.582521254 podStartE2EDuration="40.296280452s" totalImagesPullingTime="6.713759198s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 11:15:53 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:16:33.291259427 +0000 UTC m=+298.493578578" watchObservedRunningTime="2026-07-09 11:16:33.296280452 +0000 UTC m=+298.498599583" Jul 09 11:16:35.163660 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:35.163632 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-98ghf_fccccfad-9a68-4bd9-a44f-af22e4339085/console-operator/2.log" Jul 09 11:16:35.164081 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:35.163814 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-98ghf_fccccfad-9a68-4bd9-a44f-af22e4339085/console-operator/2.log" Jul 09 11:16:35.174179 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:35.174155 2550 kubelet.go:1648] "Image garbage collection succeeded" Jul 09 11:16:37.298598 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.298572 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-skljh"] Jul 09 11:16:37.301795 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.301779 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-skljh" Jul 09 11:16:37.304533 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.304511 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 09 11:16:37.310128 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.310108 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 09 11:16:37.310227 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.310136 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-2tsfd\"" Jul 09 11:16:37.310973 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.310956 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 09 11:16:37.310973 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.310967 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 09 11:16:37.311114 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.311021 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 09 11:16:37.311114 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.311043 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 09 11:16:37.325858 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.325825 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-skljh"] Jul 09 11:16:37.371848 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.371813 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-f62dk"] Jul 09 11:16:37.374921 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.374903 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-f62dk" Jul 09 11:16:37.377771 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:16:37.377744 2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"node-exporter-f62dk\" is forbidden: User \"system:node:ip-10-0-143-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-143-142.ec2.internal' and this object" podUID="0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55" pod="openshift-monitoring/node-exporter-f62dk" Jul 09 11:16:37.377864 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:16:37.377772 2550 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"node-exporter-tls\" is forbidden: User \"system:node:ip-10-0-143-142.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-143-142.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" type="*v1.Secret" Jul 09 11:16:37.378095 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:16:37.378078 2550 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"node-exporter-accelerators-collector-config\" is forbidden: User \"system:node:ip-10-0-143-142.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-143-142.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" type="*v1.ConfigMap" Jul 09 11:16:37.378141 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:16:37.378073 2550 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"node-exporter-kube-rbac-proxy-config\" is forbidden: User \"system:node:ip-10-0-143-142.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-143-142.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" type="*v1.Secret" Jul 09 11:16:37.378452 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:16:37.378430 2550 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"node-exporter-dockercfg-kn2d9\" is forbidden: User \"system:node:ip-10-0-143-142.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-143-142.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-kn2d9\"" type="*v1.Secret" Jul 09 11:16:37.386910 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.386887 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d692q\" (UniqueName: \"kubernetes.io/projected/8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f-kube-api-access-d692q\") pod \"kube-state-metrics-7c9dbb8b7d-skljh\" (UID: \"8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-skljh" Jul 09 11:16:37.387021 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.386928 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-skljh\" (UID: \"8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-skljh" Jul 09 11:16:37.387021 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.386955 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-skljh\" (UID: \"8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-skljh" Jul 09 11:16:37.387021 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.386979 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-skljh\" (UID: \"8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-skljh" Jul 09 11:16:37.387021 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.387013 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-skljh\" (UID: \"8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-skljh" Jul 09 11:16:37.387197 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.387040 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-skljh\" (UID: \"8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-skljh" Jul 09 11:16:37.487996 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.487961 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55-root\") pod \"node-exporter-f62dk\" (UID: \"0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55\") " pod="openshift-monitoring/node-exporter-f62dk" Jul 09 11:16:37.487996 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.487998 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55-node-exporter-tls\") pod \"node-exporter-f62dk\" (UID: \"0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55\") " pod="openshift-monitoring/node-exporter-f62dk" Jul 09 11:16:37.488250 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.488069 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d692q\" (UniqueName: \"kubernetes.io/projected/8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f-kube-api-access-d692q\") pod \"kube-state-metrics-7c9dbb8b7d-skljh\" (UID: \"8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-skljh" Jul 09 11:16:37.488250 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.488097 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55-node-exporter-accelerators-collector-config\") pod \"node-exporter-f62dk\" (UID: \"0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55\") " pod="openshift-monitoring/node-exporter-f62dk" Jul 09 11:16:37.488250 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.488170 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5tgg\" (UniqueName: \"kubernetes.io/projected/0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55-kube-api-access-c5tgg\") pod \"node-exporter-f62dk\" (UID: \"0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55\") " pod="openshift-monitoring/node-exporter-f62dk" Jul 09 11:16:37.488250 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.488218 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-skljh\" (UID: \"8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-skljh" Jul 09 11:16:37.488558 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.488255 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-skljh\" (UID: \"8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-skljh" Jul 09 11:16:37.488558 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.488274 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-skljh\" (UID: \"8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-skljh" Jul 09 11:16:37.488558 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.488289 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55-metrics-client-ca\") pod \"node-exporter-f62dk\" (UID: \"0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55\") " pod="openshift-monitoring/node-exporter-f62dk" Jul 09 11:16:37.488558 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.488349 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-skljh\" (UID: \"8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-skljh" Jul 09 11:16:37.488558 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.488376 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-skljh\" (UID: \"8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-skljh" Jul 09 11:16:37.488558 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.488400 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55-sys\") pod \"node-exporter-f62dk\" (UID: \"0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55\") " pod="openshift-monitoring/node-exporter-f62dk" Jul 09 11:16:37.488558 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.488418 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f62dk\" (UID: \"0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55\") " pod="openshift-monitoring/node-exporter-f62dk" Jul 09 11:16:37.488558 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.488443 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55-node-exporter-textfile\") pod \"node-exporter-f62dk\" (UID: \"0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55\") " pod="openshift-monitoring/node-exporter-f62dk" Jul 09 11:16:37.488558 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.488502 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55-node-exporter-wtmp\") pod \"node-exporter-f62dk\" (UID: \"0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55\") " pod="openshift-monitoring/node-exporter-f62dk" Jul 09 11:16:37.488992 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.488761 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-skljh\" (UID: \"8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-skljh" Jul 09 11:16:37.489067 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.489042 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-skljh\" (UID: \"8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-skljh" Jul 09 11:16:37.489120 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.489065 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-skljh\" (UID: \"8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-skljh" Jul 09 11:16:37.490768 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.490742 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-skljh\" (UID: \"8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-skljh" Jul 09 11:16:37.490873 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.490802 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-skljh\" (UID: \"8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-skljh" Jul 09 11:16:37.499833 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.499802 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d692q\" (UniqueName: \"kubernetes.io/projected/8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f-kube-api-access-d692q\") pod \"kube-state-metrics-7c9dbb8b7d-skljh\" (UID: \"8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-skljh" Jul 09 11:16:37.589032 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.588945 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55-metrics-client-ca\") pod \"node-exporter-f62dk\" (UID: \"0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55\") " pod="openshift-monitoring/node-exporter-f62dk" Jul 09 11:16:37.589032 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.588999 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55-sys\") pod \"node-exporter-f62dk\" (UID: \"0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55\") " pod="openshift-monitoring/node-exporter-f62dk" Jul 09 11:16:37.589032 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.589019 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f62dk\" (UID: \"0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55\") " pod="openshift-monitoring/node-exporter-f62dk" Jul 09 11:16:37.589258 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.589089 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55-sys\") pod \"node-exporter-f62dk\" (UID: \"0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55\") " pod="openshift-monitoring/node-exporter-f62dk" Jul 09 11:16:37.589258 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.589145 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55-node-exporter-textfile\") pod \"node-exporter-f62dk\" (UID: \"0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55\") " pod="openshift-monitoring/node-exporter-f62dk" Jul 09 11:16:37.589258 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.589179 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55-node-exporter-wtmp\") pod \"node-exporter-f62dk\" (UID: \"0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55\") " pod="openshift-monitoring/node-exporter-f62dk" Jul 09 11:16:37.589258 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.589223 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55-root\") pod \"node-exporter-f62dk\" (UID: \"0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55\") " pod="openshift-monitoring/node-exporter-f62dk" Jul 09 11:16:37.589258 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.589256 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55-node-exporter-tls\") pod \"node-exporter-f62dk\" (UID: \"0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55\") " pod="openshift-monitoring/node-exporter-f62dk" Jul 09 11:16:37.589487 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.589313 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55-node-exporter-accelerators-collector-config\") pod \"node-exporter-f62dk\" (UID: \"0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55\") " pod="openshift-monitoring/node-exporter-f62dk" Jul 09 11:16:37.589487 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.589353 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55-root\") pod \"node-exporter-f62dk\" (UID: \"0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55\") " pod="openshift-monitoring/node-exporter-f62dk" Jul 09 11:16:37.589487 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.589370 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5tgg\" (UniqueName: \"kubernetes.io/projected/0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55-kube-api-access-c5tgg\") pod \"node-exporter-f62dk\" (UID: \"0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55\") " pod="openshift-monitoring/node-exporter-f62dk" Jul 09 11:16:37.589487 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.589410 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55-node-exporter-wtmp\") pod \"node-exporter-f62dk\" (UID: \"0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55\") " pod="openshift-monitoring/node-exporter-f62dk" Jul 09 11:16:37.589611 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.589534 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55-node-exporter-textfile\") pod \"node-exporter-f62dk\" (UID: \"0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55\") " pod="openshift-monitoring/node-exporter-f62dk" Jul 09 11:16:37.589611 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.589566 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55-metrics-client-ca\") pod \"node-exporter-f62dk\" (UID: \"0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55\") " pod="openshift-monitoring/node-exporter-f62dk" Jul 09 11:16:37.597849 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.597827 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5tgg\" (UniqueName: \"kubernetes.io/projected/0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55-kube-api-access-c5tgg\") pod \"node-exporter-f62dk\" (UID: \"0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55\") " pod="openshift-monitoring/node-exporter-f62dk" Jul 09 11:16:37.609990 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.609970 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-skljh" Jul 09 11:16:37.757108 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.757024 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-skljh"] Jul 09 11:16:37.759697 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:16:37.759671 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c947a1f_aa08_45aa_9a5d_ae3f7fa40a4f.slice/crio-0e8884841275b5d61bd03feb6cda179976501020ba06cbae840d863ee1e9eac2 WatchSource:0}: Error finding container 0e8884841275b5d61bd03feb6cda179976501020ba06cbae840d863ee1e9eac2: Status 404 returned error can't find the container with id 0e8884841275b5d61bd03feb6cda179976501020ba06cbae840d863ee1e9eac2 Jul 09 11:16:37.761572 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:37.761555 2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 11:16:38.250458 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:38.250427 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-skljh" event={"ID":"8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f","Type":"ContainerStarted","Data":"0e8884841275b5d61bd03feb6cda179976501020ba06cbae840d863ee1e9eac2"} Jul 09 11:16:38.301384 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:38.301354 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 09 11:16:38.311686 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:38.311661 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f62dk\" (UID: \"0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55\") " pod="openshift-monitoring/node-exporter-f62dk" Jul 09 11:16:38.590212 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:16:38.590177 2550 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: failed to sync secret cache: timed out waiting for the condition Jul 09 11:16:38.590415 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:16:38.590257 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55-node-exporter-tls podName:0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55 nodeName:}" failed. No retries permitted until 2026-07-09 11:16:39.090239735 +0000 UTC m=+304.292558866 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55-node-exporter-tls") pod "node-exporter-f62dk" (UID: "0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55") : failed to sync secret cache: timed out waiting for the condition Jul 09 11:16:38.590415 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:16:38.590177 2550 configmap.go:193] Couldn't get configMap openshift-monitoring/node-exporter-accelerators-collector-config: failed to sync configmap cache: timed out waiting for the condition Jul 09 11:16:38.590415 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:16:38.590324 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55-node-exporter-accelerators-collector-config podName:0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55 nodeName:}" failed. No retries permitted until 2026-07-09 11:16:39.090312393 +0000 UTC m=+304.292631524 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-accelerators-collector-config" (UniqueName: "kubernetes.io/configmap/0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55-node-exporter-accelerators-collector-config") pod "node-exporter-f62dk" (UID: "0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55") : failed to sync configmap cache: timed out waiting for the condition Jul 09 11:16:38.649427 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:38.649393 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 09 11:16:38.867897 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:38.867817 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 09 11:16:38.976068 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:38.976036 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-kn2d9\"" Jul 09 11:16:39.104635 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:39.104605 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55-node-exporter-tls\") pod \"node-exporter-f62dk\" (UID: \"0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55\") " pod="openshift-monitoring/node-exporter-f62dk" Jul 09 11:16:39.104760 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:39.104665 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55-node-exporter-accelerators-collector-config\") pod \"node-exporter-f62dk\" (UID: \"0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55\") " pod="openshift-monitoring/node-exporter-f62dk" Jul 09 11:16:39.105186 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:39.105169 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55-node-exporter-accelerators-collector-config\") pod \"node-exporter-f62dk\" (UID: \"0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55\") " pod="openshift-monitoring/node-exporter-f62dk" Jul 09 11:16:39.106885 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:39.106864 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55-node-exporter-tls\") pod \"node-exporter-f62dk\" (UID: \"0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55\") " pod="openshift-monitoring/node-exporter-f62dk" Jul 09 11:16:39.183876 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:39.183822 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-f62dk" Jul 09 11:16:39.201537 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:16:39.201509 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0eb1f5d9_f0d7_48d3_be2b_65ccfcccbf55.slice/crio-7d9827c1626c1f1c09985b60cdf04d7c775ce6ce137abf19bba74f6a5cce08ca WatchSource:0}: Error finding container 7d9827c1626c1f1c09985b60cdf04d7c775ce6ce137abf19bba74f6a5cce08ca: Status 404 returned error can't find the container with id 7d9827c1626c1f1c09985b60cdf04d7c775ce6ce137abf19bba74f6a5cce08ca Jul 09 11:16:39.256846 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:39.256804 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-skljh" event={"ID":"8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f","Type":"ContainerStarted","Data":"0a847a18cc403de086ae06286643d2989b4dc2ceb04e7735900636c7647e6ee4"} Jul 09 11:16:39.257007 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:39.256878 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-skljh" event={"ID":"8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f","Type":"ContainerStarted","Data":"e8e1cd355822f349d820e83c0953e7b40abb04c815c71f881f356b20aaa47779"} Jul 09 11:16:39.257007 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:39.256907 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-skljh" event={"ID":"8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f","Type":"ContainerStarted","Data":"0ec524d30b468350eecc4674d647e20085c34e4a4288b8fc82e18d0ac95711ac"} Jul 09 11:16:39.258539 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:39.258513 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f62dk" event={"ID":"0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55","Type":"ContainerStarted","Data":"7d9827c1626c1f1c09985b60cdf04d7c775ce6ce137abf19bba74f6a5cce08ca"} Jul 09 11:16:39.291670 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:39.291616 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-skljh" podStartSLOduration=1.045343654 podStartE2EDuration="2.291594917s" totalImagesPullingTime="1.246251263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 11:16:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:16:39.291126816 +0000 UTC m=+304.493445964" watchObservedRunningTime="2026-07-09 11:16:39.291594917 +0000 UTC m=+304.493914067" Jul 09 11:16:39.294241 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:39.294215 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bvn2" Jul 09 11:16:40.264187 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:40.264140 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f62dk" event={"ID":"0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55","Type":"ContainerStarted","Data":"0de49892a00c3f94dce430e97c45f040930d4176cc3e78a0820968369998dd1e"} Jul 09 11:16:40.692388 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:40.692360 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" Jul 09 11:16:41.269252 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:41.269219 2550 generic.go:356] "Generic (PLEG): container finished" podID="0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55" containerID="0de49892a00c3f94dce430e97c45f040930d4176cc3e78a0820968369998dd1e" exitCode=0 Jul 09 11:16:41.269650 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:41.269310 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f62dk" event={"ID":"0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55","Type":"ContainerDied","Data":"0de49892a00c3f94dce430e97c45f040930d4176cc3e78a0820968369998dd1e"} Jul 09 11:16:42.275258 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:42.275225 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f62dk" event={"ID":"0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55","Type":"ContainerStarted","Data":"5c49f92f51600582685ffa338d8da78cd946a9fb85e61249b3efa842676b8649"} Jul 09 11:16:42.275675 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:42.275276 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f62dk" event={"ID":"0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55","Type":"ContainerStarted","Data":"1cef8a370c73e52505599ac7da7dfc61621c5bacf730db0a8ac67ff5c38dbf9d"} Jul 09 11:16:42.346259 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:42.346197 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-f62dk" podStartSLOduration=4.342501394 podStartE2EDuration="5.346175317s" totalImagesPullingTime="938.664695ms" totalInitContainerRuntime="65.009228ms" isStatefulPod=true podCreationTimestamp="2026-07-09 11:16:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:16:42.34508596 +0000 UTC m=+307.547405100" watchObservedRunningTime="2026-07-09 11:16:42.346175317 +0000 UTC m=+307.548494476" Jul 09 11:16:43.195036 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.195003 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7448f9db94-86hhs" Jul 09 11:16:43.294256 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.294229 2550 scope.go:117] "RemoveContainer" containerID="b818cbf1a28804f91441f4359e6e4f58172947388d1c62e84f18a24cd2e16dee" Jul 09 11:16:43.794689 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.794654 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 11:16:43.798318 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.798298 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.801245 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.801224 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 09 11:16:43.801679 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.801632 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 09 11:16:43.802251 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.802229 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 09 11:16:43.802691 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.802542 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 09 11:16:43.802691 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.802573 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 09 11:16:43.802691 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.802586 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 09 11:16:43.802691 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.802674 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 09 11:16:43.802892 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.802725 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7bllvd22d2kub\"" Jul 09 11:16:43.803570 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.803548 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 09 11:16:43.803705 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.803552 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 09 11:16:43.804522 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.804504 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 09 11:16:43.805506 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.805324 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 09 11:16:43.828867 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.828847 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-pgx9k\"" Jul 09 11:16:43.831406 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.831390 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 09 11:16:43.831512 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.831486 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 09 11:16:43.835457 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.835440 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 09 11:16:43.843678 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.843654 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0e08c463-1b72-4e78-9e08-203aa65af9cf-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.843781 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.843686 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-config\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.843781 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.843708 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0e08c463-1b72-4e78-9e08-203aa65af9cf-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.843781 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.843738 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.843897 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.843777 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.843897 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.843824 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e08c463-1b72-4e78-9e08-203aa65af9cf-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.843897 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.843842 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0e08c463-1b72-4e78-9e08-203aa65af9cf-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.843897 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.843881 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e08c463-1b72-4e78-9e08-203aa65af9cf-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.844027 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.843928 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0e08c463-1b72-4e78-9e08-203aa65af9cf-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.844027 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.843947 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.844027 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.843964 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-web-config\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.844027 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.843978 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.844154 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.844042 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0e08c463-1b72-4e78-9e08-203aa65af9cf-config-out\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.844154 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.844063 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.844154 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.844078 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.844154 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.844113 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0e08c463-1b72-4e78-9e08-203aa65af9cf-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.844154 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.844137 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.844154 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.844151 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbdz4\" (UniqueName: \"kubernetes.io/projected/0e08c463-1b72-4e78-9e08-203aa65af9cf-kube-api-access-wbdz4\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.844354 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.844170 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e08c463-1b72-4e78-9e08-203aa65af9cf-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.844354 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.844208 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e08c463-1b72-4e78-9e08-203aa65af9cf-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.849175 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.849159 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 09 11:16:43.862474 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.862453 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 11:16:43.944824 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.944791 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0e08c463-1b72-4e78-9e08-203aa65af9cf-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.944824 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.944828 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.945090 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.944849 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-web-config\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.945090 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.944865 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.945090 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.944889 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0e08c463-1b72-4e78-9e08-203aa65af9cf-config-out\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.945090 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.944906 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.945090 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.944930 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.945344 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.945087 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0e08c463-1b72-4e78-9e08-203aa65af9cf-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.945732 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.945705 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0e08c463-1b72-4e78-9e08-203aa65af9cf-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.945844 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.945764 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.945844 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.945798 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbdz4\" (UniqueName: \"kubernetes.io/projected/0e08c463-1b72-4e78-9e08-203aa65af9cf-kube-api-access-wbdz4\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.945957 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.945850 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e08c463-1b72-4e78-9e08-203aa65af9cf-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.945957 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.945911 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e08c463-1b72-4e78-9e08-203aa65af9cf-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.946061 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.945965 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0e08c463-1b72-4e78-9e08-203aa65af9cf-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.946061 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.945999 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-config\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.946061 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.946035 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0e08c463-1b72-4e78-9e08-203aa65af9cf-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.946220 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.946077 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.946220 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.946102 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.946220 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.946131 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e08c463-1b72-4e78-9e08-203aa65af9cf-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.946220 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.946163 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0e08c463-1b72-4e78-9e08-203aa65af9cf-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.946220 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.946197 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e08c463-1b72-4e78-9e08-203aa65af9cf-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.947221 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.947193 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e08c463-1b72-4e78-9e08-203aa65af9cf-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.947516 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.947493 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0e08c463-1b72-4e78-9e08-203aa65af9cf-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.948023 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.947998 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.948179 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.948145 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e08c463-1b72-4e78-9e08-203aa65af9cf-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.948727 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.948706 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.948988 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.948966 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0e08c463-1b72-4e78-9e08-203aa65af9cf-config-out\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.949309 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.949281 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.949510 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.949315 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.949780 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.949754 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e08c463-1b72-4e78-9e08-203aa65af9cf-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.950300 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.950258 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.951242 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.951211 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-config\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.951485 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.951445 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0e08c463-1b72-4e78-9e08-203aa65af9cf-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.951592 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.951384 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-web-config\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.951699 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.951678 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.951879 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.951777 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0e08c463-1b72-4e78-9e08-203aa65af9cf-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.951879 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.951844 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e08c463-1b72-4e78-9e08-203aa65af9cf-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.952573 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.952555 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.952821 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.952805 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0e08c463-1b72-4e78-9e08-203aa65af9cf-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:43.956939 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:43.956916 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbdz4\" (UniqueName: \"kubernetes.io/projected/0e08c463-1b72-4e78-9e08-203aa65af9cf-kube-api-access-wbdz4\") pod \"prometheus-k8s-0\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:44.107811 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:44.107725 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:44.265486 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:44.265457 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 11:16:44.265862 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:16:44.265840 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e08c463_1b72_4e78_9e08_203aa65af9cf.slice/crio-5284e9cac5c258b2f2471b7078e65b078c6137a6aad7ec477a16b7fe566c6d15 WatchSource:0}: Error finding container 5284e9cac5c258b2f2471b7078e65b078c6137a6aad7ec477a16b7fe566c6d15: Status 404 returned error can't find the container with id 5284e9cac5c258b2f2471b7078e65b078c6137a6aad7ec477a16b7fe566c6d15 Jul 09 11:16:44.284014 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:44.283993 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-98ghf_fccccfad-9a68-4bd9-a44f-af22e4339085/console-operator/2.log" Jul 09 11:16:44.284135 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:44.284080 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-98ghf" event={"ID":"fccccfad-9a68-4bd9-a44f-af22e4339085","Type":"ContainerStarted","Data":"6d73e9ed49f35f72a6d6a2d7a72783725a47279aa756c42d08724e0e35e7e3eb"} Jul 09 11:16:44.284439 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:44.284382 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-98ghf" Jul 09 11:16:44.285439 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:44.285413 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e08c463-1b72-4e78-9e08-203aa65af9cf","Type":"ContainerStarted","Data":"5284e9cac5c258b2f2471b7078e65b078c6137a6aad7ec477a16b7fe566c6d15"} Jul 09 11:16:44.289412 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:44.289393 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-98ghf" Jul 09 11:16:44.347462 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:44.347401 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-98ghf" podStartSLOduration=53.004316467 podStartE2EDuration="55.34737875s" totalImagesPullingTime="2.343062283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 11:15:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:16:44.316487483 +0000 UTC m=+309.518806633" watchObservedRunningTime="2026-07-09 11:16:44.34737875 +0000 UTC m=+309.549697900" Jul 09 11:16:45.711445 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:45.711315 2550 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" podUID="48f05f33-fe91-4449-a9a8-a03a95d45006" containerName="registry" containerID="cri-o://8ff900be7ab964099a8fb5d4a39374f0277dea63a071fe02c02b75c6f8ee7882" gracePeriod=30 Jul 09 11:16:45.943831 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:45.943805 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" Jul 09 11:16:46.062095 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:46.062066 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/48f05f33-fe91-4449-a9a8-a03a95d45006-bound-sa-token\") pod \"48f05f33-fe91-4449-a9a8-a03a95d45006\" (UID: \"48f05f33-fe91-4449-a9a8-a03a95d45006\") " Jul 09 11:16:46.062251 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:46.062130 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/48f05f33-fe91-4449-a9a8-a03a95d45006-ca-trust-extracted\") pod \"48f05f33-fe91-4449-a9a8-a03a95d45006\" (UID: \"48f05f33-fe91-4449-a9a8-a03a95d45006\") " Jul 09 11:16:46.062251 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:46.062154 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/48f05f33-fe91-4449-a9a8-a03a95d45006-installation-pull-secrets\") pod \"48f05f33-fe91-4449-a9a8-a03a95d45006\" (UID: \"48f05f33-fe91-4449-a9a8-a03a95d45006\") " Jul 09 11:16:46.062251 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:46.062180 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48f05f33-fe91-4449-a9a8-a03a95d45006-registry-tls\") pod \"48f05f33-fe91-4449-a9a8-a03a95d45006\" (UID: \"48f05f33-fe91-4449-a9a8-a03a95d45006\") " Jul 09 11:16:46.062251 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:46.062207 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hkwr2\" (UniqueName: \"kubernetes.io/projected/48f05f33-fe91-4449-a9a8-a03a95d45006-kube-api-access-hkwr2\") pod \"48f05f33-fe91-4449-a9a8-a03a95d45006\" (UID: \"48f05f33-fe91-4449-a9a8-a03a95d45006\") " Jul 09 11:16:46.062251 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:46.062240 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/48f05f33-fe91-4449-a9a8-a03a95d45006-trusted-ca\") pod \"48f05f33-fe91-4449-a9a8-a03a95d45006\" (UID: \"48f05f33-fe91-4449-a9a8-a03a95d45006\") " Jul 09 11:16:46.062515 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:46.062271 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/48f05f33-fe91-4449-a9a8-a03a95d45006-registry-certificates\") pod \"48f05f33-fe91-4449-a9a8-a03a95d45006\" (UID: \"48f05f33-fe91-4449-a9a8-a03a95d45006\") " Jul 09 11:16:46.062515 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:46.062350 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/48f05f33-fe91-4449-a9a8-a03a95d45006-image-registry-private-configuration\") pod \"48f05f33-fe91-4449-a9a8-a03a95d45006\" (UID: \"48f05f33-fe91-4449-a9a8-a03a95d45006\") " Jul 09 11:16:46.062854 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:46.062823 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48f05f33-fe91-4449-a9a8-a03a95d45006-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "48f05f33-fe91-4449-a9a8-a03a95d45006" (UID: "48f05f33-fe91-4449-a9a8-a03a95d45006"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 11:16:46.062854 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:46.062836 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48f05f33-fe91-4449-a9a8-a03a95d45006-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "48f05f33-fe91-4449-a9a8-a03a95d45006" (UID: "48f05f33-fe91-4449-a9a8-a03a95d45006"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 11:16:46.064634 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:46.064606 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48f05f33-fe91-4449-a9a8-a03a95d45006-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "48f05f33-fe91-4449-a9a8-a03a95d45006" (UID: "48f05f33-fe91-4449-a9a8-a03a95d45006"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 11:16:46.064831 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:46.064803 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48f05f33-fe91-4449-a9a8-a03a95d45006-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "48f05f33-fe91-4449-a9a8-a03a95d45006" (UID: "48f05f33-fe91-4449-a9a8-a03a95d45006"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 11:16:46.064933 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:46.064881 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48f05f33-fe91-4449-a9a8-a03a95d45006-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "48f05f33-fe91-4449-a9a8-a03a95d45006" (UID: "48f05f33-fe91-4449-a9a8-a03a95d45006"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 11:16:46.064933 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:46.064890 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48f05f33-fe91-4449-a9a8-a03a95d45006-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "48f05f33-fe91-4449-a9a8-a03a95d45006" (UID: "48f05f33-fe91-4449-a9a8-a03a95d45006"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 11:16:46.065144 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:46.065128 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48f05f33-fe91-4449-a9a8-a03a95d45006-kube-api-access-hkwr2" (OuterVolumeSpecName: "kube-api-access-hkwr2") pod "48f05f33-fe91-4449-a9a8-a03a95d45006" (UID: "48f05f33-fe91-4449-a9a8-a03a95d45006"). InnerVolumeSpecName "kube-api-access-hkwr2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 11:16:46.070551 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:46.070530 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48f05f33-fe91-4449-a9a8-a03a95d45006-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "48f05f33-fe91-4449-a9a8-a03a95d45006" (UID: "48f05f33-fe91-4449-a9a8-a03a95d45006"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 11:16:46.163905 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:46.163867 2550 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/48f05f33-fe91-4449-a9a8-a03a95d45006-ca-trust-extracted\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:16:46.163905 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:46.163898 2550 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/48f05f33-fe91-4449-a9a8-a03a95d45006-installation-pull-secrets\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:16:46.163905 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:46.163907 2550 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48f05f33-fe91-4449-a9a8-a03a95d45006-registry-tls\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:16:46.163905 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:46.163916 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hkwr2\" (UniqueName: \"kubernetes.io/projected/48f05f33-fe91-4449-a9a8-a03a95d45006-kube-api-access-hkwr2\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:16:46.164156 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:46.163926 2550 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/48f05f33-fe91-4449-a9a8-a03a95d45006-trusted-ca\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:16:46.164156 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:46.163934 2550 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/48f05f33-fe91-4449-a9a8-a03a95d45006-registry-certificates\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:16:46.164156 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:46.163943 2550 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/48f05f33-fe91-4449-a9a8-a03a95d45006-image-registry-private-configuration\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:16:46.164156 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:46.163952 2550 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/48f05f33-fe91-4449-a9a8-a03a95d45006-bound-sa-token\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:16:46.293269 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:46.293234 2550 generic.go:356] "Generic (PLEG): container finished" podID="48f05f33-fe91-4449-a9a8-a03a95d45006" containerID="8ff900be7ab964099a8fb5d4a39374f0277dea63a071fe02c02b75c6f8ee7882" exitCode=0 Jul 09 11:16:46.293485 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:46.293285 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" Jul 09 11:16:46.293485 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:46.293350 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" event={"ID":"48f05f33-fe91-4449-a9a8-a03a95d45006","Type":"ContainerDied","Data":"8ff900be7ab964099a8fb5d4a39374f0277dea63a071fe02c02b75c6f8ee7882"} Jul 09 11:16:46.293485 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:46.293408 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-699bccf5c4-zkf4x" event={"ID":"48f05f33-fe91-4449-a9a8-a03a95d45006","Type":"ContainerDied","Data":"39d85b45cd98ae697cbaab02c48c95b88eb5192d76b57468adee16fc48f2d927"} Jul 09 11:16:46.293485 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:46.293444 2550 scope.go:117] "RemoveContainer" containerID="8ff900be7ab964099a8fb5d4a39374f0277dea63a071fe02c02b75c6f8ee7882" Jul 09 11:16:46.294913 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:46.294880 2550 generic.go:356] "Generic (PLEG): container finished" podID="0e08c463-1b72-4e78-9e08-203aa65af9cf" containerID="8d7d636d20680ccf47960c76e9389f66f7eb18a5c0bf9a697ae267b1f99a921f" exitCode=0 Jul 09 11:16:46.295029 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:46.294972 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e08c463-1b72-4e78-9e08-203aa65af9cf","Type":"ContainerDied","Data":"8d7d636d20680ccf47960c76e9389f66f7eb18a5c0bf9a697ae267b1f99a921f"} Jul 09 11:16:46.302105 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:46.301993 2550 scope.go:117] "RemoveContainer" containerID="8ff900be7ab964099a8fb5d4a39374f0277dea63a071fe02c02b75c6f8ee7882" Jul 09 11:16:46.302409 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:16:46.302382 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ff900be7ab964099a8fb5d4a39374f0277dea63a071fe02c02b75c6f8ee7882\": container with ID starting with 8ff900be7ab964099a8fb5d4a39374f0277dea63a071fe02c02b75c6f8ee7882 not found: ID does not exist" containerID="8ff900be7ab964099a8fb5d4a39374f0277dea63a071fe02c02b75c6f8ee7882" Jul 09 11:16:46.302489 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:46.302421 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ff900be7ab964099a8fb5d4a39374f0277dea63a071fe02c02b75c6f8ee7882"} err="failed to get container status \"8ff900be7ab964099a8fb5d4a39374f0277dea63a071fe02c02b75c6f8ee7882\": rpc error: code = NotFound desc = could not find container \"8ff900be7ab964099a8fb5d4a39374f0277dea63a071fe02c02b75c6f8ee7882\": container with ID starting with 8ff900be7ab964099a8fb5d4a39374f0277dea63a071fe02c02b75c6f8ee7882 not found: ID does not exist" Jul 09 11:16:46.362268 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:46.362188 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-699bccf5c4-zkf4x"] Jul 09 11:16:46.373594 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:46.373562 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-699bccf5c4-zkf4x"] Jul 09 11:16:47.300191 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:47.300152 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48f05f33-fe91-4449-a9a8-a03a95d45006" path="/var/lib/kubelet/pods/48f05f33-fe91-4449-a9a8-a03a95d45006/volumes" Jul 09 11:16:50.314863 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:50.314834 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e08c463-1b72-4e78-9e08-203aa65af9cf","Type":"ContainerStarted","Data":"a476b724478297f551c3741a66c5b2d758b82f2d1c2ddb80fd6f0d26a54a4ad6"} Jul 09 11:16:50.315249 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:50.314941 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e08c463-1b72-4e78-9e08-203aa65af9cf","Type":"ContainerStarted","Data":"94b6d87719ae0d9ab237ea42bccaafc7d69756f692cf2a1cd4d0021a84831c82"} Jul 09 11:16:52.325981 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:52.325948 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e08c463-1b72-4e78-9e08-203aa65af9cf","Type":"ContainerStarted","Data":"cb302e76cfe94cd357825a527f567ebcf5bc26cf5e873995fbe715b7eb16f734"} Jul 09 11:16:52.326432 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:52.326024 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e08c463-1b72-4e78-9e08-203aa65af9cf","Type":"ContainerStarted","Data":"13c54bebe1a0b513df7134d180cbc7079c04852ac840215a3f1387f38fa38645"} Jul 09 11:16:52.326432 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:52.326064 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e08c463-1b72-4e78-9e08-203aa65af9cf","Type":"ContainerStarted","Data":"d013e13d73d977bff46b25451a8102750e10a4840130840d5ceb416f8e2e00a9"} Jul 09 11:16:52.326432 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:52.326103 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e08c463-1b72-4e78-9e08-203aa65af9cf","Type":"ContainerStarted","Data":"43c42e707c0095f6682932fbf02b2cd0a03fdb2b095c5c09f6c86d1bbe653a8a"} Jul 09 11:16:52.381986 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:52.381930 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.977098006 podStartE2EDuration="9.38191322s" totalImagesPullingTime="6.386963025s" totalInitContainerRuntime="17.852189ms" isStatefulPod=false podCreationTimestamp="2026-07-09 11:16:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:16:52.381021207 +0000 UTC m=+317.583340374" watchObservedRunningTime="2026-07-09 11:16:52.38191322 +0000 UTC m=+317.584232369" Jul 09 11:16:54.108168 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:54.108125 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:16:58.348283 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:58.348250 2550 generic.go:356] "Generic (PLEG): container finished" podID="511c08dd-43b1-485c-b275-df75627c0441" containerID="629acc6f7b3af80efe6bf9bb9afb9161299b939e83252dacabd6baeb3666714b" exitCode=0 Jul 09 11:16:58.348691 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:58.348324 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-hr5fb" event={"ID":"511c08dd-43b1-485c-b275-df75627c0441","Type":"ContainerDied","Data":"629acc6f7b3af80efe6bf9bb9afb9161299b939e83252dacabd6baeb3666714b"} Jul 09 11:16:58.348691 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:58.348628 2550 scope.go:117] "RemoveContainer" containerID="629acc6f7b3af80efe6bf9bb9afb9161299b939e83252dacabd6baeb3666714b" Jul 09 11:16:59.354118 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:16:59.354084 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-hr5fb" event={"ID":"511c08dd-43b1-485c-b275-df75627c0441","Type":"ContainerStarted","Data":"200e7a83d5d85870a0f15d179e5bb8d6c557016404a72e9c05ec00a4fa2a460d"} Jul 09 11:17:44.108427 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:17:44.108389 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:17:44.127262 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:17:44.127235 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:17:44.529234 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:17:44.529159 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:17:49.008091 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:17:49.008047 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91432ff0-fa58-460b-a6d6-3a14a9f5f295-metrics-certs\") pod \"network-metrics-daemon-6bvn2\" (UID: \"91432ff0-fa58-460b-a6d6-3a14a9f5f295\") " pod="openshift-multus/network-metrics-daemon-6bvn2" Jul 09 11:17:49.010298 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:17:49.010276 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91432ff0-fa58-460b-a6d6-3a14a9f5f295-metrics-certs\") pod \"network-metrics-daemon-6bvn2\" (UID: \"91432ff0-fa58-460b-a6d6-3a14a9f5f295\") " pod="openshift-multus/network-metrics-daemon-6bvn2" Jul 09 11:17:49.197391 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:17:49.197356 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-h6b4g\"" Jul 09 11:17:49.205624 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:17:49.205606 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6bvn2" Jul 09 11:17:49.346868 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:17:49.346711 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6bvn2"] Jul 09 11:17:49.349494 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:17:49.349460 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91432ff0_fa58_460b_a6d6_3a14a9f5f295.slice/crio-d74264656c9fd341a753801598c56a1794d3241147ef27f5d360da46cf97d170 WatchSource:0}: Error finding container d74264656c9fd341a753801598c56a1794d3241147ef27f5d360da46cf97d170: Status 404 returned error can't find the container with id d74264656c9fd341a753801598c56a1794d3241147ef27f5d360da46cf97d170 Jul 09 11:17:49.532453 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:17:49.532421 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6bvn2" event={"ID":"91432ff0-fa58-460b-a6d6-3a14a9f5f295","Type":"ContainerStarted","Data":"d74264656c9fd341a753801598c56a1794d3241147ef27f5d360da46cf97d170"} Jul 09 11:17:51.540250 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:17:51.540219 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6bvn2" event={"ID":"91432ff0-fa58-460b-a6d6-3a14a9f5f295","Type":"ContainerStarted","Data":"304682001a8c68cf0f02452aa7de541df74953c88e50f98e40b48f6c2dbad6ea"} Jul 09 11:17:51.540654 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:17:51.540273 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6bvn2" event={"ID":"91432ff0-fa58-460b-a6d6-3a14a9f5f295","Type":"ContainerStarted","Data":"270bf478f0b50b15b995392b76f8971285c2bcee76072ce8e6f4aaa6f27165dd"} Jul 09 11:17:51.562461 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:17:51.562414 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-6bvn2" podStartSLOduration=375.366481146 podStartE2EDuration="6m16.562396362s" totalImagesPullingTime="1.195915216s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 11:11:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:17:51.560113027 +0000 UTC m=+376.762432191" watchObservedRunningTime="2026-07-09 11:17:51.562396362 +0000 UTC m=+376.764715511" Jul 09 11:18:02.120575 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.120534 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 11:18:02.121147 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.120913 2550 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0e08c463-1b72-4e78-9e08-203aa65af9cf" containerName="prometheus" containerID="cri-o://94b6d87719ae0d9ab237ea42bccaafc7d69756f692cf2a1cd4d0021a84831c82" gracePeriod=600 Jul 09 11:18:02.121147 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.120939 2550 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0e08c463-1b72-4e78-9e08-203aa65af9cf" containerName="thanos-sidecar" containerID="cri-o://43c42e707c0095f6682932fbf02b2cd0a03fdb2b095c5c09f6c86d1bbe653a8a" gracePeriod=600 Jul 09 11:18:02.121147 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.120981 2550 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0e08c463-1b72-4e78-9e08-203aa65af9cf" containerName="kube-rbac-proxy-thanos" containerID="cri-o://cb302e76cfe94cd357825a527f567ebcf5bc26cf5e873995fbe715b7eb16f734" gracePeriod=600 Jul 09 11:18:02.121147 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.121004 2550 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0e08c463-1b72-4e78-9e08-203aa65af9cf" containerName="kube-rbac-proxy-web" containerID="cri-o://d013e13d73d977bff46b25451a8102750e10a4840130840d5ceb416f8e2e00a9" gracePeriod=600 Jul 09 11:18:02.121147 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.120917 2550 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0e08c463-1b72-4e78-9e08-203aa65af9cf" containerName="kube-rbac-proxy" containerID="cri-o://13c54bebe1a0b513df7134d180cbc7079c04852ac840215a3f1387f38fa38645" gracePeriod=600 Jul 09 11:18:02.121147 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.121105 2550 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0e08c463-1b72-4e78-9e08-203aa65af9cf" containerName="config-reloader" containerID="cri-o://a476b724478297f551c3741a66c5b2d758b82f2d1c2ddb80fd6f0d26a54a4ad6" gracePeriod=600 Jul 09 11:18:02.365998 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.365971 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.410871 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.410795 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0e08c463-1b72-4e78-9e08-203aa65af9cf-prometheus-k8s-rulefiles-0\") pod \"0e08c463-1b72-4e78-9e08-203aa65af9cf\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " Jul 09 11:18:02.410871 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.410830 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0e08c463-1b72-4e78-9e08-203aa65af9cf-prometheus-k8s-db\") pod \"0e08c463-1b72-4e78-9e08-203aa65af9cf\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " Jul 09 11:18:02.410871 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.410851 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e08c463-1b72-4e78-9e08-203aa65af9cf-configmap-kubelet-serving-ca-bundle\") pod \"0e08c463-1b72-4e78-9e08-203aa65af9cf\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " Jul 09 11:18:02.411131 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.410892 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0e08c463-1b72-4e78-9e08-203aa65af9cf-tls-assets\") pod \"0e08c463-1b72-4e78-9e08-203aa65af9cf\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " Jul 09 11:18:02.411131 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.410923 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-secret-metrics-client-certs\") pod \"0e08c463-1b72-4e78-9e08-203aa65af9cf\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " Jul 09 11:18:02.411131 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.410948 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0e08c463-1b72-4e78-9e08-203aa65af9cf-prometheus-k8s-rulefiles-1\") pod \"0e08c463-1b72-4e78-9e08-203aa65af9cf\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " Jul 09 11:18:02.411131 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.410971 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-secret-prometheus-k8s-tls\") pod \"0e08c463-1b72-4e78-9e08-203aa65af9cf\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " Jul 09 11:18:02.411131 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.411002 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e08c463-1b72-4e78-9e08-203aa65af9cf-configmap-metrics-client-ca\") pod \"0e08c463-1b72-4e78-9e08-203aa65af9cf\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " Jul 09 11:18:02.411131 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.411034 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e08c463-1b72-4e78-9e08-203aa65af9cf-configmap-serving-certs-ca-bundle\") pod \"0e08c463-1b72-4e78-9e08-203aa65af9cf\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " Jul 09 11:18:02.411131 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.411076 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e08c463-1b72-4e78-9e08-203aa65af9cf-prometheus-trusted-ca-bundle\") pod \"0e08c463-1b72-4e78-9e08-203aa65af9cf\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " Jul 09 11:18:02.411131 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.411109 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0e08c463-1b72-4e78-9e08-203aa65af9cf-prometheus-k8s-rulefiles-2\") pod \"0e08c463-1b72-4e78-9e08-203aa65af9cf\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " Jul 09 11:18:02.411131 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.411134 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0e08c463-1b72-4e78-9e08-203aa65af9cf-config-out\") pod \"0e08c463-1b72-4e78-9e08-203aa65af9cf\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " Jul 09 11:18:02.411594 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.411163 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"0e08c463-1b72-4e78-9e08-203aa65af9cf\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " Jul 09 11:18:02.411594 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.411190 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-config\") pod \"0e08c463-1b72-4e78-9e08-203aa65af9cf\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " Jul 09 11:18:02.411594 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.411291 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e08c463-1b72-4e78-9e08-203aa65af9cf-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "0e08c463-1b72-4e78-9e08-203aa65af9cf" (UID: "0e08c463-1b72-4e78-9e08-203aa65af9cf"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 11:18:02.411594 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.411442 2550 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0e08c463-1b72-4e78-9e08-203aa65af9cf-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:18:02.411594 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.411446 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e08c463-1b72-4e78-9e08-203aa65af9cf-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "0e08c463-1b72-4e78-9e08-203aa65af9cf" (UID: "0e08c463-1b72-4e78-9e08-203aa65af9cf"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 11:18:02.411839 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.411695 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e08c463-1b72-4e78-9e08-203aa65af9cf-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "0e08c463-1b72-4e78-9e08-203aa65af9cf" (UID: "0e08c463-1b72-4e78-9e08-203aa65af9cf"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 11:18:02.412531 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.411999 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e08c463-1b72-4e78-9e08-203aa65af9cf-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "0e08c463-1b72-4e78-9e08-203aa65af9cf" (UID: "0e08c463-1b72-4e78-9e08-203aa65af9cf"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 11:18:02.412531 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.412386 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e08c463-1b72-4e78-9e08-203aa65af9cf-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "0e08c463-1b72-4e78-9e08-203aa65af9cf" (UID: "0e08c463-1b72-4e78-9e08-203aa65af9cf"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 11:18:02.412958 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.412921 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e08c463-1b72-4e78-9e08-203aa65af9cf-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "0e08c463-1b72-4e78-9e08-203aa65af9cf" (UID: "0e08c463-1b72-4e78-9e08-203aa65af9cf"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 11:18:02.414273 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.413857 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e08c463-1b72-4e78-9e08-203aa65af9cf-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "0e08c463-1b72-4e78-9e08-203aa65af9cf" (UID: "0e08c463-1b72-4e78-9e08-203aa65af9cf"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 11:18:02.414273 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.414238 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-config" (OuterVolumeSpecName: "config") pod "0e08c463-1b72-4e78-9e08-203aa65af9cf" (UID: "0e08c463-1b72-4e78-9e08-203aa65af9cf"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 11:18:02.414454 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.414320 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e08c463-1b72-4e78-9e08-203aa65af9cf-config-out" (OuterVolumeSpecName: "config-out") pod "0e08c463-1b72-4e78-9e08-203aa65af9cf" (UID: "0e08c463-1b72-4e78-9e08-203aa65af9cf"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 11:18:02.415093 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.414955 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "0e08c463-1b72-4e78-9e08-203aa65af9cf" (UID: "0e08c463-1b72-4e78-9e08-203aa65af9cf"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 11:18:02.415093 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.415074 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e08c463-1b72-4e78-9e08-203aa65af9cf-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "0e08c463-1b72-4e78-9e08-203aa65af9cf" (UID: "0e08c463-1b72-4e78-9e08-203aa65af9cf"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 11:18:02.415258 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.415187 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e08c463-1b72-4e78-9e08-203aa65af9cf-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "0e08c463-1b72-4e78-9e08-203aa65af9cf" (UID: "0e08c463-1b72-4e78-9e08-203aa65af9cf"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 11:18:02.415601 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.415578 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "0e08c463-1b72-4e78-9e08-203aa65af9cf" (UID: "0e08c463-1b72-4e78-9e08-203aa65af9cf"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 11:18:02.416553 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.416523 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "0e08c463-1b72-4e78-9e08-203aa65af9cf" (UID: "0e08c463-1b72-4e78-9e08-203aa65af9cf"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 11:18:02.512180 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.512150 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-web-config\") pod \"0e08c463-1b72-4e78-9e08-203aa65af9cf\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " Jul 09 11:18:02.512180 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.512189 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wbdz4\" (UniqueName: \"kubernetes.io/projected/0e08c463-1b72-4e78-9e08-203aa65af9cf-kube-api-access-wbdz4\") pod \"0e08c463-1b72-4e78-9e08-203aa65af9cf\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " Jul 09 11:18:02.512479 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.512254 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"0e08c463-1b72-4e78-9e08-203aa65af9cf\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " Jul 09 11:18:02.512479 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.512274 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-secret-kube-rbac-proxy\") pod \"0e08c463-1b72-4e78-9e08-203aa65af9cf\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " Jul 09 11:18:02.512479 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.512288 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-secret-grpc-tls\") pod \"0e08c463-1b72-4e78-9e08-203aa65af9cf\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " Jul 09 11:18:02.512479 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.512307 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-thanos-prometheus-http-client-file\") pod \"0e08c463-1b72-4e78-9e08-203aa65af9cf\" (UID: \"0e08c463-1b72-4e78-9e08-203aa65af9cf\") " Jul 09 11:18:02.512710 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.512503 2550 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0e08c463-1b72-4e78-9e08-203aa65af9cf-tls-assets\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:18:02.512710 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.512521 2550 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-secret-metrics-client-certs\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:18:02.512710 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.512533 2550 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-secret-prometheus-k8s-tls\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:18:02.512710 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.512547 2550 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e08c463-1b72-4e78-9e08-203aa65af9cf-configmap-metrics-client-ca\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:18:02.512710 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.512562 2550 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e08c463-1b72-4e78-9e08-203aa65af9cf-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:18:02.512710 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.512576 2550 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e08c463-1b72-4e78-9e08-203aa65af9cf-prometheus-trusted-ca-bundle\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:18:02.512710 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.512592 2550 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0e08c463-1b72-4e78-9e08-203aa65af9cf-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:18:02.512710 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.512606 2550 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0e08c463-1b72-4e78-9e08-203aa65af9cf-config-out\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:18:02.512710 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.512620 2550 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:18:02.512710 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.512632 2550 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-config\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:18:02.512710 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.512645 2550 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0e08c463-1b72-4e78-9e08-203aa65af9cf-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:18:02.512710 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.512658 2550 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0e08c463-1b72-4e78-9e08-203aa65af9cf-prometheus-k8s-db\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:18:02.512710 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.512673 2550 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e08c463-1b72-4e78-9e08-203aa65af9cf-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:18:02.514505 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.514478 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e08c463-1b72-4e78-9e08-203aa65af9cf-kube-api-access-wbdz4" (OuterVolumeSpecName: "kube-api-access-wbdz4") pod "0e08c463-1b72-4e78-9e08-203aa65af9cf" (UID: "0e08c463-1b72-4e78-9e08-203aa65af9cf"). InnerVolumeSpecName "kube-api-access-wbdz4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 11:18:02.514764 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.514744 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "0e08c463-1b72-4e78-9e08-203aa65af9cf" (UID: "0e08c463-1b72-4e78-9e08-203aa65af9cf"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 11:18:02.515051 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.515021 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "0e08c463-1b72-4e78-9e08-203aa65af9cf" (UID: "0e08c463-1b72-4e78-9e08-203aa65af9cf"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 11:18:02.515148 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.515107 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "0e08c463-1b72-4e78-9e08-203aa65af9cf" (UID: "0e08c463-1b72-4e78-9e08-203aa65af9cf"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 11:18:02.515148 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.515120 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "0e08c463-1b72-4e78-9e08-203aa65af9cf" (UID: "0e08c463-1b72-4e78-9e08-203aa65af9cf"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 11:18:02.524506 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.524475 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-web-config" (OuterVolumeSpecName: "web-config") pod "0e08c463-1b72-4e78-9e08-203aa65af9cf" (UID: "0e08c463-1b72-4e78-9e08-203aa65af9cf"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 11:18:02.582448 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.582415 2550 generic.go:356] "Generic (PLEG): container finished" podID="0e08c463-1b72-4e78-9e08-203aa65af9cf" containerID="cb302e76cfe94cd357825a527f567ebcf5bc26cf5e873995fbe715b7eb16f734" exitCode=0 Jul 09 11:18:02.582448 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.582446 2550 generic.go:356] "Generic (PLEG): container finished" podID="0e08c463-1b72-4e78-9e08-203aa65af9cf" containerID="13c54bebe1a0b513df7134d180cbc7079c04852ac840215a3f1387f38fa38645" exitCode=0 Jul 09 11:18:02.582632 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.582457 2550 generic.go:356] "Generic (PLEG): container finished" podID="0e08c463-1b72-4e78-9e08-203aa65af9cf" containerID="d013e13d73d977bff46b25451a8102750e10a4840130840d5ceb416f8e2e00a9" exitCode=0 Jul 09 11:18:02.582632 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.582466 2550 generic.go:356] "Generic (PLEG): container finished" podID="0e08c463-1b72-4e78-9e08-203aa65af9cf" containerID="43c42e707c0095f6682932fbf02b2cd0a03fdb2b095c5c09f6c86d1bbe653a8a" exitCode=0 Jul 09 11:18:02.582632 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.582474 2550 generic.go:356] "Generic (PLEG): container finished" podID="0e08c463-1b72-4e78-9e08-203aa65af9cf" containerID="a476b724478297f551c3741a66c5b2d758b82f2d1c2ddb80fd6f0d26a54a4ad6" exitCode=0 Jul 09 11:18:02.582632 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.582482 2550 generic.go:356] "Generic (PLEG): container finished" podID="0e08c463-1b72-4e78-9e08-203aa65af9cf" containerID="94b6d87719ae0d9ab237ea42bccaafc7d69756f692cf2a1cd4d0021a84831c82" exitCode=0 Jul 09 11:18:02.582632 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.582487 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e08c463-1b72-4e78-9e08-203aa65af9cf","Type":"ContainerDied","Data":"cb302e76cfe94cd357825a527f567ebcf5bc26cf5e873995fbe715b7eb16f734"} Jul 09 11:18:02.582632 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.582522 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.582632 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.582552 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e08c463-1b72-4e78-9e08-203aa65af9cf","Type":"ContainerDied","Data":"13c54bebe1a0b513df7134d180cbc7079c04852ac840215a3f1387f38fa38645"} Jul 09 11:18:02.582632 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.582582 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e08c463-1b72-4e78-9e08-203aa65af9cf","Type":"ContainerDied","Data":"d013e13d73d977bff46b25451a8102750e10a4840130840d5ceb416f8e2e00a9"} Jul 09 11:18:02.582632 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.582609 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e08c463-1b72-4e78-9e08-203aa65af9cf","Type":"ContainerDied","Data":"43c42e707c0095f6682932fbf02b2cd0a03fdb2b095c5c09f6c86d1bbe653a8a"} Jul 09 11:18:02.582632 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.582624 2550 scope.go:117] "RemoveContainer" containerID="cb302e76cfe94cd357825a527f567ebcf5bc26cf5e873995fbe715b7eb16f734" Jul 09 11:18:02.582632 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.582632 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e08c463-1b72-4e78-9e08-203aa65af9cf","Type":"ContainerDied","Data":"a476b724478297f551c3741a66c5b2d758b82f2d1c2ddb80fd6f0d26a54a4ad6"} Jul 09 11:18:02.583006 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.582657 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e08c463-1b72-4e78-9e08-203aa65af9cf","Type":"ContainerDied","Data":"94b6d87719ae0d9ab237ea42bccaafc7d69756f692cf2a1cd4d0021a84831c82"} Jul 09 11:18:02.583006 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.582682 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e08c463-1b72-4e78-9e08-203aa65af9cf","Type":"ContainerDied","Data":"5284e9cac5c258b2f2471b7078e65b078c6137a6aad7ec477a16b7fe566c6d15"} Jul 09 11:18:02.591245 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.591226 2550 scope.go:117] "RemoveContainer" containerID="13c54bebe1a0b513df7134d180cbc7079c04852ac840215a3f1387f38fa38645" Jul 09 11:18:02.598719 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.598703 2550 scope.go:117] "RemoveContainer" containerID="d013e13d73d977bff46b25451a8102750e10a4840130840d5ceb416f8e2e00a9" Jul 09 11:18:02.605028 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.605009 2550 scope.go:117] "RemoveContainer" containerID="43c42e707c0095f6682932fbf02b2cd0a03fdb2b095c5c09f6c86d1bbe653a8a" Jul 09 11:18:02.611160 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.611138 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 11:18:02.611897 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.611881 2550 scope.go:117] "RemoveContainer" containerID="a476b724478297f551c3741a66c5b2d758b82f2d1c2ddb80fd6f0d26a54a4ad6" Jul 09 11:18:02.613160 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.613143 2550 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:18:02.613160 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.613163 2550 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-secret-kube-rbac-proxy\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:18:02.613288 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.613178 2550 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-secret-grpc-tls\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:18:02.613288 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.613192 2550 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-thanos-prometheus-http-client-file\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:18:02.613288 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.613203 2550 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0e08c463-1b72-4e78-9e08-203aa65af9cf-web-config\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:18:02.613288 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.613217 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbdz4\" (UniqueName: \"kubernetes.io/projected/0e08c463-1b72-4e78-9e08-203aa65af9cf-kube-api-access-wbdz4\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:18:02.615835 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.615814 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 11:18:02.618822 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.618806 2550 scope.go:117] "RemoveContainer" containerID="94b6d87719ae0d9ab237ea42bccaafc7d69756f692cf2a1cd4d0021a84831c82" Jul 09 11:18:02.625608 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.625590 2550 scope.go:117] "RemoveContainer" containerID="8d7d636d20680ccf47960c76e9389f66f7eb18a5c0bf9a697ae267b1f99a921f" Jul 09 11:18:02.631935 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.631920 2550 scope.go:117] "RemoveContainer" containerID="cb302e76cfe94cd357825a527f567ebcf5bc26cf5e873995fbe715b7eb16f734" Jul 09 11:18:02.632191 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:18:02.632173 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb302e76cfe94cd357825a527f567ebcf5bc26cf5e873995fbe715b7eb16f734\": container with ID starting with cb302e76cfe94cd357825a527f567ebcf5bc26cf5e873995fbe715b7eb16f734 not found: ID does not exist" containerID="cb302e76cfe94cd357825a527f567ebcf5bc26cf5e873995fbe715b7eb16f734" Jul 09 11:18:02.632241 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.632201 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb302e76cfe94cd357825a527f567ebcf5bc26cf5e873995fbe715b7eb16f734"} err="failed to get container status \"cb302e76cfe94cd357825a527f567ebcf5bc26cf5e873995fbe715b7eb16f734\": rpc error: code = NotFound desc = could not find container \"cb302e76cfe94cd357825a527f567ebcf5bc26cf5e873995fbe715b7eb16f734\": container with ID starting with cb302e76cfe94cd357825a527f567ebcf5bc26cf5e873995fbe715b7eb16f734 not found: ID does not exist" Jul 09 11:18:02.632241 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.632220 2550 scope.go:117] "RemoveContainer" containerID="13c54bebe1a0b513df7134d180cbc7079c04852ac840215a3f1387f38fa38645" Jul 09 11:18:02.632493 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:18:02.632475 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13c54bebe1a0b513df7134d180cbc7079c04852ac840215a3f1387f38fa38645\": container with ID starting with 13c54bebe1a0b513df7134d180cbc7079c04852ac840215a3f1387f38fa38645 not found: ID does not exist" containerID="13c54bebe1a0b513df7134d180cbc7079c04852ac840215a3f1387f38fa38645" Jul 09 11:18:02.632543 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.632501 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13c54bebe1a0b513df7134d180cbc7079c04852ac840215a3f1387f38fa38645"} err="failed to get container status \"13c54bebe1a0b513df7134d180cbc7079c04852ac840215a3f1387f38fa38645\": rpc error: code = NotFound desc = could not find container \"13c54bebe1a0b513df7134d180cbc7079c04852ac840215a3f1387f38fa38645\": container with ID starting with 13c54bebe1a0b513df7134d180cbc7079c04852ac840215a3f1387f38fa38645 not found: ID does not exist" Jul 09 11:18:02.632543 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.632516 2550 scope.go:117] "RemoveContainer" containerID="d013e13d73d977bff46b25451a8102750e10a4840130840d5ceb416f8e2e00a9" Jul 09 11:18:02.632773 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:18:02.632755 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d013e13d73d977bff46b25451a8102750e10a4840130840d5ceb416f8e2e00a9\": container with ID starting with d013e13d73d977bff46b25451a8102750e10a4840130840d5ceb416f8e2e00a9 not found: ID does not exist" containerID="d013e13d73d977bff46b25451a8102750e10a4840130840d5ceb416f8e2e00a9" Jul 09 11:18:02.632836 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.632784 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d013e13d73d977bff46b25451a8102750e10a4840130840d5ceb416f8e2e00a9"} err="failed to get container status \"d013e13d73d977bff46b25451a8102750e10a4840130840d5ceb416f8e2e00a9\": rpc error: code = NotFound desc = could not find container \"d013e13d73d977bff46b25451a8102750e10a4840130840d5ceb416f8e2e00a9\": container with ID starting with d013e13d73d977bff46b25451a8102750e10a4840130840d5ceb416f8e2e00a9 not found: ID does not exist" Jul 09 11:18:02.632836 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.632804 2550 scope.go:117] "RemoveContainer" containerID="43c42e707c0095f6682932fbf02b2cd0a03fdb2b095c5c09f6c86d1bbe653a8a" Jul 09 11:18:02.633032 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:18:02.633015 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43c42e707c0095f6682932fbf02b2cd0a03fdb2b095c5c09f6c86d1bbe653a8a\": container with ID starting with 43c42e707c0095f6682932fbf02b2cd0a03fdb2b095c5c09f6c86d1bbe653a8a not found: ID does not exist" containerID="43c42e707c0095f6682932fbf02b2cd0a03fdb2b095c5c09f6c86d1bbe653a8a" Jul 09 11:18:02.633070 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.633035 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43c42e707c0095f6682932fbf02b2cd0a03fdb2b095c5c09f6c86d1bbe653a8a"} err="failed to get container status \"43c42e707c0095f6682932fbf02b2cd0a03fdb2b095c5c09f6c86d1bbe653a8a\": rpc error: code = NotFound desc = could not find container \"43c42e707c0095f6682932fbf02b2cd0a03fdb2b095c5c09f6c86d1bbe653a8a\": container with ID starting with 43c42e707c0095f6682932fbf02b2cd0a03fdb2b095c5c09f6c86d1bbe653a8a not found: ID does not exist" Jul 09 11:18:02.633070 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.633047 2550 scope.go:117] "RemoveContainer" containerID="a476b724478297f551c3741a66c5b2d758b82f2d1c2ddb80fd6f0d26a54a4ad6" Jul 09 11:18:02.633248 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:18:02.633231 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a476b724478297f551c3741a66c5b2d758b82f2d1c2ddb80fd6f0d26a54a4ad6\": container with ID starting with a476b724478297f551c3741a66c5b2d758b82f2d1c2ddb80fd6f0d26a54a4ad6 not found: ID does not exist" containerID="a476b724478297f551c3741a66c5b2d758b82f2d1c2ddb80fd6f0d26a54a4ad6" Jul 09 11:18:02.633312 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.633255 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a476b724478297f551c3741a66c5b2d758b82f2d1c2ddb80fd6f0d26a54a4ad6"} err="failed to get container status \"a476b724478297f551c3741a66c5b2d758b82f2d1c2ddb80fd6f0d26a54a4ad6\": rpc error: code = NotFound desc = could not find container \"a476b724478297f551c3741a66c5b2d758b82f2d1c2ddb80fd6f0d26a54a4ad6\": container with ID starting with a476b724478297f551c3741a66c5b2d758b82f2d1c2ddb80fd6f0d26a54a4ad6 not found: ID does not exist" Jul 09 11:18:02.633312 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.633272 2550 scope.go:117] "RemoveContainer" containerID="94b6d87719ae0d9ab237ea42bccaafc7d69756f692cf2a1cd4d0021a84831c82" Jul 09 11:18:02.633509 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:18:02.633494 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94b6d87719ae0d9ab237ea42bccaafc7d69756f692cf2a1cd4d0021a84831c82\": container with ID starting with 94b6d87719ae0d9ab237ea42bccaafc7d69756f692cf2a1cd4d0021a84831c82 not found: ID does not exist" containerID="94b6d87719ae0d9ab237ea42bccaafc7d69756f692cf2a1cd4d0021a84831c82" Jul 09 11:18:02.633547 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.633515 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94b6d87719ae0d9ab237ea42bccaafc7d69756f692cf2a1cd4d0021a84831c82"} err="failed to get container status \"94b6d87719ae0d9ab237ea42bccaafc7d69756f692cf2a1cd4d0021a84831c82\": rpc error: code = NotFound desc = could not find container \"94b6d87719ae0d9ab237ea42bccaafc7d69756f692cf2a1cd4d0021a84831c82\": container with ID starting with 94b6d87719ae0d9ab237ea42bccaafc7d69756f692cf2a1cd4d0021a84831c82 not found: ID does not exist" Jul 09 11:18:02.633547 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.633529 2550 scope.go:117] "RemoveContainer" containerID="8d7d636d20680ccf47960c76e9389f66f7eb18a5c0bf9a697ae267b1f99a921f" Jul 09 11:18:02.633719 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:18:02.633706 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d7d636d20680ccf47960c76e9389f66f7eb18a5c0bf9a697ae267b1f99a921f\": container with ID starting with 8d7d636d20680ccf47960c76e9389f66f7eb18a5c0bf9a697ae267b1f99a921f not found: ID does not exist" containerID="8d7d636d20680ccf47960c76e9389f66f7eb18a5c0bf9a697ae267b1f99a921f" Jul 09 11:18:02.633753 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.633722 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d7d636d20680ccf47960c76e9389f66f7eb18a5c0bf9a697ae267b1f99a921f"} err="failed to get container status \"8d7d636d20680ccf47960c76e9389f66f7eb18a5c0bf9a697ae267b1f99a921f\": rpc error: code = NotFound desc = could not find container \"8d7d636d20680ccf47960c76e9389f66f7eb18a5c0bf9a697ae267b1f99a921f\": container with ID starting with 8d7d636d20680ccf47960c76e9389f66f7eb18a5c0bf9a697ae267b1f99a921f not found: ID does not exist" Jul 09 11:18:02.633753 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.633736 2550 scope.go:117] "RemoveContainer" containerID="cb302e76cfe94cd357825a527f567ebcf5bc26cf5e873995fbe715b7eb16f734" Jul 09 11:18:02.633952 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.633935 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb302e76cfe94cd357825a527f567ebcf5bc26cf5e873995fbe715b7eb16f734"} err="failed to get container status \"cb302e76cfe94cd357825a527f567ebcf5bc26cf5e873995fbe715b7eb16f734\": rpc error: code = NotFound desc = could not find container \"cb302e76cfe94cd357825a527f567ebcf5bc26cf5e873995fbe715b7eb16f734\": container with ID starting with cb302e76cfe94cd357825a527f567ebcf5bc26cf5e873995fbe715b7eb16f734 not found: ID does not exist" Jul 09 11:18:02.633994 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.633954 2550 scope.go:117] "RemoveContainer" containerID="13c54bebe1a0b513df7134d180cbc7079c04852ac840215a3f1387f38fa38645" Jul 09 11:18:02.634147 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.634132 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13c54bebe1a0b513df7134d180cbc7079c04852ac840215a3f1387f38fa38645"} err="failed to get container status \"13c54bebe1a0b513df7134d180cbc7079c04852ac840215a3f1387f38fa38645\": rpc error: code = NotFound desc = could not find container \"13c54bebe1a0b513df7134d180cbc7079c04852ac840215a3f1387f38fa38645\": container with ID starting with 13c54bebe1a0b513df7134d180cbc7079c04852ac840215a3f1387f38fa38645 not found: ID does not exist" Jul 09 11:18:02.634182 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.634148 2550 scope.go:117] "RemoveContainer" containerID="d013e13d73d977bff46b25451a8102750e10a4840130840d5ceb416f8e2e00a9" Jul 09 11:18:02.634358 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.634324 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d013e13d73d977bff46b25451a8102750e10a4840130840d5ceb416f8e2e00a9"} err="failed to get container status \"d013e13d73d977bff46b25451a8102750e10a4840130840d5ceb416f8e2e00a9\": rpc error: code = NotFound desc = could not find container \"d013e13d73d977bff46b25451a8102750e10a4840130840d5ceb416f8e2e00a9\": container with ID starting with d013e13d73d977bff46b25451a8102750e10a4840130840d5ceb416f8e2e00a9 not found: ID does not exist" Jul 09 11:18:02.634358 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.634356 2550 scope.go:117] "RemoveContainer" containerID="43c42e707c0095f6682932fbf02b2cd0a03fdb2b095c5c09f6c86d1bbe653a8a" Jul 09 11:18:02.634566 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.634546 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43c42e707c0095f6682932fbf02b2cd0a03fdb2b095c5c09f6c86d1bbe653a8a"} err="failed to get container status \"43c42e707c0095f6682932fbf02b2cd0a03fdb2b095c5c09f6c86d1bbe653a8a\": rpc error: code = NotFound desc = could not find container \"43c42e707c0095f6682932fbf02b2cd0a03fdb2b095c5c09f6c86d1bbe653a8a\": container with ID starting with 43c42e707c0095f6682932fbf02b2cd0a03fdb2b095c5c09f6c86d1bbe653a8a not found: ID does not exist" Jul 09 11:18:02.634612 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.634566 2550 scope.go:117] "RemoveContainer" containerID="a476b724478297f551c3741a66c5b2d758b82f2d1c2ddb80fd6f0d26a54a4ad6" Jul 09 11:18:02.634794 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.634779 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a476b724478297f551c3741a66c5b2d758b82f2d1c2ddb80fd6f0d26a54a4ad6"} err="failed to get container status \"a476b724478297f551c3741a66c5b2d758b82f2d1c2ddb80fd6f0d26a54a4ad6\": rpc error: code = NotFound desc = could not find container \"a476b724478297f551c3741a66c5b2d758b82f2d1c2ddb80fd6f0d26a54a4ad6\": container with ID starting with a476b724478297f551c3741a66c5b2d758b82f2d1c2ddb80fd6f0d26a54a4ad6 not found: ID does not exist" Jul 09 11:18:02.634837 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.634795 2550 scope.go:117] "RemoveContainer" containerID="94b6d87719ae0d9ab237ea42bccaafc7d69756f692cf2a1cd4d0021a84831c82" Jul 09 11:18:02.634983 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.634967 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94b6d87719ae0d9ab237ea42bccaafc7d69756f692cf2a1cd4d0021a84831c82"} err="failed to get container status \"94b6d87719ae0d9ab237ea42bccaafc7d69756f692cf2a1cd4d0021a84831c82\": rpc error: code = NotFound desc = could not find container \"94b6d87719ae0d9ab237ea42bccaafc7d69756f692cf2a1cd4d0021a84831c82\": container with ID starting with 94b6d87719ae0d9ab237ea42bccaafc7d69756f692cf2a1cd4d0021a84831c82 not found: ID does not exist" Jul 09 11:18:02.635022 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.634984 2550 scope.go:117] "RemoveContainer" containerID="8d7d636d20680ccf47960c76e9389f66f7eb18a5c0bf9a697ae267b1f99a921f" Jul 09 11:18:02.635208 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.635193 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d7d636d20680ccf47960c76e9389f66f7eb18a5c0bf9a697ae267b1f99a921f"} err="failed to get container status \"8d7d636d20680ccf47960c76e9389f66f7eb18a5c0bf9a697ae267b1f99a921f\": rpc error: code = NotFound desc = could not find container \"8d7d636d20680ccf47960c76e9389f66f7eb18a5c0bf9a697ae267b1f99a921f\": container with ID starting with 8d7d636d20680ccf47960c76e9389f66f7eb18a5c0bf9a697ae267b1f99a921f not found: ID does not exist" Jul 09 11:18:02.635258 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.635209 2550 scope.go:117] "RemoveContainer" containerID="cb302e76cfe94cd357825a527f567ebcf5bc26cf5e873995fbe715b7eb16f734" Jul 09 11:18:02.635420 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.635403 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb302e76cfe94cd357825a527f567ebcf5bc26cf5e873995fbe715b7eb16f734"} err="failed to get container status \"cb302e76cfe94cd357825a527f567ebcf5bc26cf5e873995fbe715b7eb16f734\": rpc error: code = NotFound desc = could not find container \"cb302e76cfe94cd357825a527f567ebcf5bc26cf5e873995fbe715b7eb16f734\": container with ID starting with cb302e76cfe94cd357825a527f567ebcf5bc26cf5e873995fbe715b7eb16f734 not found: ID does not exist" Jul 09 11:18:02.635467 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.635420 2550 scope.go:117] "RemoveContainer" containerID="13c54bebe1a0b513df7134d180cbc7079c04852ac840215a3f1387f38fa38645" Jul 09 11:18:02.635633 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.635616 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13c54bebe1a0b513df7134d180cbc7079c04852ac840215a3f1387f38fa38645"} err="failed to get container status \"13c54bebe1a0b513df7134d180cbc7079c04852ac840215a3f1387f38fa38645\": rpc error: code = NotFound desc = could not find container \"13c54bebe1a0b513df7134d180cbc7079c04852ac840215a3f1387f38fa38645\": container with ID starting with 13c54bebe1a0b513df7134d180cbc7079c04852ac840215a3f1387f38fa38645 not found: ID does not exist" Jul 09 11:18:02.635680 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.635634 2550 scope.go:117] "RemoveContainer" containerID="d013e13d73d977bff46b25451a8102750e10a4840130840d5ceb416f8e2e00a9" Jul 09 11:18:02.635830 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.635815 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d013e13d73d977bff46b25451a8102750e10a4840130840d5ceb416f8e2e00a9"} err="failed to get container status \"d013e13d73d977bff46b25451a8102750e10a4840130840d5ceb416f8e2e00a9\": rpc error: code = NotFound desc = could not find container \"d013e13d73d977bff46b25451a8102750e10a4840130840d5ceb416f8e2e00a9\": container with ID starting with d013e13d73d977bff46b25451a8102750e10a4840130840d5ceb416f8e2e00a9 not found: ID does not exist" Jul 09 11:18:02.635866 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.635830 2550 scope.go:117] "RemoveContainer" containerID="43c42e707c0095f6682932fbf02b2cd0a03fdb2b095c5c09f6c86d1bbe653a8a" Jul 09 11:18:02.636006 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.635993 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43c42e707c0095f6682932fbf02b2cd0a03fdb2b095c5c09f6c86d1bbe653a8a"} err="failed to get container status \"43c42e707c0095f6682932fbf02b2cd0a03fdb2b095c5c09f6c86d1bbe653a8a\": rpc error: code = NotFound desc = could not find container \"43c42e707c0095f6682932fbf02b2cd0a03fdb2b095c5c09f6c86d1bbe653a8a\": container with ID starting with 43c42e707c0095f6682932fbf02b2cd0a03fdb2b095c5c09f6c86d1bbe653a8a not found: ID does not exist" Jul 09 11:18:02.636047 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.636006 2550 scope.go:117] "RemoveContainer" containerID="a476b724478297f551c3741a66c5b2d758b82f2d1c2ddb80fd6f0d26a54a4ad6" Jul 09 11:18:02.636179 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.636166 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a476b724478297f551c3741a66c5b2d758b82f2d1c2ddb80fd6f0d26a54a4ad6"} err="failed to get container status \"a476b724478297f551c3741a66c5b2d758b82f2d1c2ddb80fd6f0d26a54a4ad6\": rpc error: code = NotFound desc = could not find container \"a476b724478297f551c3741a66c5b2d758b82f2d1c2ddb80fd6f0d26a54a4ad6\": container with ID starting with a476b724478297f551c3741a66c5b2d758b82f2d1c2ddb80fd6f0d26a54a4ad6 not found: ID does not exist" Jul 09 11:18:02.636225 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.636178 2550 scope.go:117] "RemoveContainer" containerID="94b6d87719ae0d9ab237ea42bccaafc7d69756f692cf2a1cd4d0021a84831c82" Jul 09 11:18:02.636365 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.636349 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94b6d87719ae0d9ab237ea42bccaafc7d69756f692cf2a1cd4d0021a84831c82"} err="failed to get container status \"94b6d87719ae0d9ab237ea42bccaafc7d69756f692cf2a1cd4d0021a84831c82\": rpc error: code = NotFound desc = could not find container \"94b6d87719ae0d9ab237ea42bccaafc7d69756f692cf2a1cd4d0021a84831c82\": container with ID starting with 94b6d87719ae0d9ab237ea42bccaafc7d69756f692cf2a1cd4d0021a84831c82 not found: ID does not exist" Jul 09 11:18:02.636407 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.636364 2550 scope.go:117] "RemoveContainer" containerID="8d7d636d20680ccf47960c76e9389f66f7eb18a5c0bf9a697ae267b1f99a921f" Jul 09 11:18:02.636588 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.636559 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d7d636d20680ccf47960c76e9389f66f7eb18a5c0bf9a697ae267b1f99a921f"} err="failed to get container status \"8d7d636d20680ccf47960c76e9389f66f7eb18a5c0bf9a697ae267b1f99a921f\": rpc error: code = NotFound desc = could not find container \"8d7d636d20680ccf47960c76e9389f66f7eb18a5c0bf9a697ae267b1f99a921f\": container with ID starting with 8d7d636d20680ccf47960c76e9389f66f7eb18a5c0bf9a697ae267b1f99a921f not found: ID does not exist" Jul 09 11:18:02.636657 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.636600 2550 scope.go:117] "RemoveContainer" containerID="cb302e76cfe94cd357825a527f567ebcf5bc26cf5e873995fbe715b7eb16f734" Jul 09 11:18:02.636798 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.636783 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb302e76cfe94cd357825a527f567ebcf5bc26cf5e873995fbe715b7eb16f734"} err="failed to get container status \"cb302e76cfe94cd357825a527f567ebcf5bc26cf5e873995fbe715b7eb16f734\": rpc error: code = NotFound desc = could not find container \"cb302e76cfe94cd357825a527f567ebcf5bc26cf5e873995fbe715b7eb16f734\": container with ID starting with cb302e76cfe94cd357825a527f567ebcf5bc26cf5e873995fbe715b7eb16f734 not found: ID does not exist" Jul 09 11:18:02.636798 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.636797 2550 scope.go:117] "RemoveContainer" containerID="13c54bebe1a0b513df7134d180cbc7079c04852ac840215a3f1387f38fa38645" Jul 09 11:18:02.636950 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.636936 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13c54bebe1a0b513df7134d180cbc7079c04852ac840215a3f1387f38fa38645"} err="failed to get container status \"13c54bebe1a0b513df7134d180cbc7079c04852ac840215a3f1387f38fa38645\": rpc error: code = NotFound desc = could not find container \"13c54bebe1a0b513df7134d180cbc7079c04852ac840215a3f1387f38fa38645\": container with ID starting with 13c54bebe1a0b513df7134d180cbc7079c04852ac840215a3f1387f38fa38645 not found: ID does not exist" Jul 09 11:18:02.636950 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.636950 2550 scope.go:117] "RemoveContainer" containerID="d013e13d73d977bff46b25451a8102750e10a4840130840d5ceb416f8e2e00a9" Jul 09 11:18:02.637099 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.637085 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d013e13d73d977bff46b25451a8102750e10a4840130840d5ceb416f8e2e00a9"} err="failed to get container status \"d013e13d73d977bff46b25451a8102750e10a4840130840d5ceb416f8e2e00a9\": rpc error: code = NotFound desc = could not find container \"d013e13d73d977bff46b25451a8102750e10a4840130840d5ceb416f8e2e00a9\": container with ID starting with d013e13d73d977bff46b25451a8102750e10a4840130840d5ceb416f8e2e00a9 not found: ID does not exist" Jul 09 11:18:02.637138 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.637098 2550 scope.go:117] "RemoveContainer" containerID="43c42e707c0095f6682932fbf02b2cd0a03fdb2b095c5c09f6c86d1bbe653a8a" Jul 09 11:18:02.637257 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.637243 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43c42e707c0095f6682932fbf02b2cd0a03fdb2b095c5c09f6c86d1bbe653a8a"} err="failed to get container status \"43c42e707c0095f6682932fbf02b2cd0a03fdb2b095c5c09f6c86d1bbe653a8a\": rpc error: code = NotFound desc = could not find container \"43c42e707c0095f6682932fbf02b2cd0a03fdb2b095c5c09f6c86d1bbe653a8a\": container with ID starting with 43c42e707c0095f6682932fbf02b2cd0a03fdb2b095c5c09f6c86d1bbe653a8a not found: ID does not exist" Jul 09 11:18:02.637257 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.637256 2550 scope.go:117] "RemoveContainer" containerID="a476b724478297f551c3741a66c5b2d758b82f2d1c2ddb80fd6f0d26a54a4ad6" Jul 09 11:18:02.637493 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.637473 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a476b724478297f551c3741a66c5b2d758b82f2d1c2ddb80fd6f0d26a54a4ad6"} err="failed to get container status \"a476b724478297f551c3741a66c5b2d758b82f2d1c2ddb80fd6f0d26a54a4ad6\": rpc error: code = NotFound desc = could not find container \"a476b724478297f551c3741a66c5b2d758b82f2d1c2ddb80fd6f0d26a54a4ad6\": container with ID starting with a476b724478297f551c3741a66c5b2d758b82f2d1c2ddb80fd6f0d26a54a4ad6 not found: ID does not exist" Jul 09 11:18:02.637493 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.637493 2550 scope.go:117] "RemoveContainer" containerID="94b6d87719ae0d9ab237ea42bccaafc7d69756f692cf2a1cd4d0021a84831c82" Jul 09 11:18:02.637715 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.637696 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94b6d87719ae0d9ab237ea42bccaafc7d69756f692cf2a1cd4d0021a84831c82"} err="failed to get container status \"94b6d87719ae0d9ab237ea42bccaafc7d69756f692cf2a1cd4d0021a84831c82\": rpc error: code = NotFound desc = could not find container \"94b6d87719ae0d9ab237ea42bccaafc7d69756f692cf2a1cd4d0021a84831c82\": container with ID starting with 94b6d87719ae0d9ab237ea42bccaafc7d69756f692cf2a1cd4d0021a84831c82 not found: ID does not exist" Jul 09 11:18:02.637788 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.637717 2550 scope.go:117] "RemoveContainer" containerID="8d7d636d20680ccf47960c76e9389f66f7eb18a5c0bf9a697ae267b1f99a921f" Jul 09 11:18:02.641490 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.641458 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d7d636d20680ccf47960c76e9389f66f7eb18a5c0bf9a697ae267b1f99a921f"} err="failed to get container status \"8d7d636d20680ccf47960c76e9389f66f7eb18a5c0bf9a697ae267b1f99a921f\": rpc error: code = NotFound desc = could not find container \"8d7d636d20680ccf47960c76e9389f66f7eb18a5c0bf9a697ae267b1f99a921f\": container with ID starting with 8d7d636d20680ccf47960c76e9389f66f7eb18a5c0bf9a697ae267b1f99a921f not found: ID does not exist" Jul 09 11:18:02.641594 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.641490 2550 scope.go:117] "RemoveContainer" containerID="cb302e76cfe94cd357825a527f567ebcf5bc26cf5e873995fbe715b7eb16f734" Jul 09 11:18:02.641763 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.641740 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb302e76cfe94cd357825a527f567ebcf5bc26cf5e873995fbe715b7eb16f734"} err="failed to get container status \"cb302e76cfe94cd357825a527f567ebcf5bc26cf5e873995fbe715b7eb16f734\": rpc error: code = NotFound desc = could not find container \"cb302e76cfe94cd357825a527f567ebcf5bc26cf5e873995fbe715b7eb16f734\": container with ID starting with cb302e76cfe94cd357825a527f567ebcf5bc26cf5e873995fbe715b7eb16f734 not found: ID does not exist" Jul 09 11:18:02.641763 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.641763 2550 scope.go:117] "RemoveContainer" containerID="13c54bebe1a0b513df7134d180cbc7079c04852ac840215a3f1387f38fa38645" Jul 09 11:18:02.642046 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.642029 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13c54bebe1a0b513df7134d180cbc7079c04852ac840215a3f1387f38fa38645"} err="failed to get container status \"13c54bebe1a0b513df7134d180cbc7079c04852ac840215a3f1387f38fa38645\": rpc error: code = NotFound desc = could not find container \"13c54bebe1a0b513df7134d180cbc7079c04852ac840215a3f1387f38fa38645\": container with ID starting with 13c54bebe1a0b513df7134d180cbc7079c04852ac840215a3f1387f38fa38645 not found: ID does not exist" Jul 09 11:18:02.642103 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.642047 2550 scope.go:117] "RemoveContainer" containerID="d013e13d73d977bff46b25451a8102750e10a4840130840d5ceb416f8e2e00a9" Jul 09 11:18:02.642295 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.642278 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d013e13d73d977bff46b25451a8102750e10a4840130840d5ceb416f8e2e00a9"} err="failed to get container status \"d013e13d73d977bff46b25451a8102750e10a4840130840d5ceb416f8e2e00a9\": rpc error: code = NotFound desc = could not find container \"d013e13d73d977bff46b25451a8102750e10a4840130840d5ceb416f8e2e00a9\": container with ID starting with d013e13d73d977bff46b25451a8102750e10a4840130840d5ceb416f8e2e00a9 not found: ID does not exist" Jul 09 11:18:02.642369 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.642296 2550 scope.go:117] "RemoveContainer" containerID="43c42e707c0095f6682932fbf02b2cd0a03fdb2b095c5c09f6c86d1bbe653a8a" Jul 09 11:18:02.642586 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.642567 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43c42e707c0095f6682932fbf02b2cd0a03fdb2b095c5c09f6c86d1bbe653a8a"} err="failed to get container status \"43c42e707c0095f6682932fbf02b2cd0a03fdb2b095c5c09f6c86d1bbe653a8a\": rpc error: code = NotFound desc = could not find container \"43c42e707c0095f6682932fbf02b2cd0a03fdb2b095c5c09f6c86d1bbe653a8a\": container with ID starting with 43c42e707c0095f6682932fbf02b2cd0a03fdb2b095c5c09f6c86d1bbe653a8a not found: ID does not exist" Jul 09 11:18:02.642637 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.642587 2550 scope.go:117] "RemoveContainer" containerID="a476b724478297f551c3741a66c5b2d758b82f2d1c2ddb80fd6f0d26a54a4ad6" Jul 09 11:18:02.642806 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.642789 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a476b724478297f551c3741a66c5b2d758b82f2d1c2ddb80fd6f0d26a54a4ad6"} err="failed to get container status \"a476b724478297f551c3741a66c5b2d758b82f2d1c2ddb80fd6f0d26a54a4ad6\": rpc error: code = NotFound desc = could not find container \"a476b724478297f551c3741a66c5b2d758b82f2d1c2ddb80fd6f0d26a54a4ad6\": container with ID starting with a476b724478297f551c3741a66c5b2d758b82f2d1c2ddb80fd6f0d26a54a4ad6 not found: ID does not exist" Jul 09 11:18:02.642806 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.642805 2550 scope.go:117] "RemoveContainer" containerID="94b6d87719ae0d9ab237ea42bccaafc7d69756f692cf2a1cd4d0021a84831c82" Jul 09 11:18:02.643029 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.643009 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94b6d87719ae0d9ab237ea42bccaafc7d69756f692cf2a1cd4d0021a84831c82"} err="failed to get container status \"94b6d87719ae0d9ab237ea42bccaafc7d69756f692cf2a1cd4d0021a84831c82\": rpc error: code = NotFound desc = could not find container \"94b6d87719ae0d9ab237ea42bccaafc7d69756f692cf2a1cd4d0021a84831c82\": container with ID starting with 94b6d87719ae0d9ab237ea42bccaafc7d69756f692cf2a1cd4d0021a84831c82 not found: ID does not exist" Jul 09 11:18:02.643081 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.643031 2550 scope.go:117] "RemoveContainer" containerID="8d7d636d20680ccf47960c76e9389f66f7eb18a5c0bf9a697ae267b1f99a921f" Jul 09 11:18:02.643248 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.643228 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d7d636d20680ccf47960c76e9389f66f7eb18a5c0bf9a697ae267b1f99a921f"} err="failed to get container status \"8d7d636d20680ccf47960c76e9389f66f7eb18a5c0bf9a697ae267b1f99a921f\": rpc error: code = NotFound desc = could not find container \"8d7d636d20680ccf47960c76e9389f66f7eb18a5c0bf9a697ae267b1f99a921f\": container with ID starting with 8d7d636d20680ccf47960c76e9389f66f7eb18a5c0bf9a697ae267b1f99a921f not found: ID does not exist" Jul 09 11:18:02.643320 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.643251 2550 scope.go:117] "RemoveContainer" containerID="cb302e76cfe94cd357825a527f567ebcf5bc26cf5e873995fbe715b7eb16f734" Jul 09 11:18:02.643504 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.643487 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb302e76cfe94cd357825a527f567ebcf5bc26cf5e873995fbe715b7eb16f734"} err="failed to get container status \"cb302e76cfe94cd357825a527f567ebcf5bc26cf5e873995fbe715b7eb16f734\": rpc error: code = NotFound desc = could not find container \"cb302e76cfe94cd357825a527f567ebcf5bc26cf5e873995fbe715b7eb16f734\": container with ID starting with cb302e76cfe94cd357825a527f567ebcf5bc26cf5e873995fbe715b7eb16f734 not found: ID does not exist" Jul 09 11:18:02.643559 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.643504 2550 scope.go:117] "RemoveContainer" containerID="13c54bebe1a0b513df7134d180cbc7079c04852ac840215a3f1387f38fa38645" Jul 09 11:18:02.643703 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.643683 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13c54bebe1a0b513df7134d180cbc7079c04852ac840215a3f1387f38fa38645"} err="failed to get container status \"13c54bebe1a0b513df7134d180cbc7079c04852ac840215a3f1387f38fa38645\": rpc error: code = NotFound desc = could not find container \"13c54bebe1a0b513df7134d180cbc7079c04852ac840215a3f1387f38fa38645\": container with ID starting with 13c54bebe1a0b513df7134d180cbc7079c04852ac840215a3f1387f38fa38645 not found: ID does not exist" Jul 09 11:18:02.643746 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.643703 2550 scope.go:117] "RemoveContainer" containerID="d013e13d73d977bff46b25451a8102750e10a4840130840d5ceb416f8e2e00a9" Jul 09 11:18:02.643876 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.643860 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d013e13d73d977bff46b25451a8102750e10a4840130840d5ceb416f8e2e00a9"} err="failed to get container status \"d013e13d73d977bff46b25451a8102750e10a4840130840d5ceb416f8e2e00a9\": rpc error: code = NotFound desc = could not find container \"d013e13d73d977bff46b25451a8102750e10a4840130840d5ceb416f8e2e00a9\": container with ID starting with d013e13d73d977bff46b25451a8102750e10a4840130840d5ceb416f8e2e00a9 not found: ID does not exist" Jul 09 11:18:02.643918 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.643876 2550 scope.go:117] "RemoveContainer" containerID="43c42e707c0095f6682932fbf02b2cd0a03fdb2b095c5c09f6c86d1bbe653a8a" Jul 09 11:18:02.644087 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.644068 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43c42e707c0095f6682932fbf02b2cd0a03fdb2b095c5c09f6c86d1bbe653a8a"} err="failed to get container status \"43c42e707c0095f6682932fbf02b2cd0a03fdb2b095c5c09f6c86d1bbe653a8a\": rpc error: code = NotFound desc = could not find container \"43c42e707c0095f6682932fbf02b2cd0a03fdb2b095c5c09f6c86d1bbe653a8a\": container with ID starting with 43c42e707c0095f6682932fbf02b2cd0a03fdb2b095c5c09f6c86d1bbe653a8a not found: ID does not exist" Jul 09 11:18:02.644125 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.644088 2550 scope.go:117] "RemoveContainer" containerID="a476b724478297f551c3741a66c5b2d758b82f2d1c2ddb80fd6f0d26a54a4ad6" Jul 09 11:18:02.644283 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.644268 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a476b724478297f551c3741a66c5b2d758b82f2d1c2ddb80fd6f0d26a54a4ad6"} err="failed to get container status \"a476b724478297f551c3741a66c5b2d758b82f2d1c2ddb80fd6f0d26a54a4ad6\": rpc error: code = NotFound desc = could not find container \"a476b724478297f551c3741a66c5b2d758b82f2d1c2ddb80fd6f0d26a54a4ad6\": container with ID starting with a476b724478297f551c3741a66c5b2d758b82f2d1c2ddb80fd6f0d26a54a4ad6 not found: ID does not exist" Jul 09 11:18:02.644324 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.644283 2550 scope.go:117] "RemoveContainer" containerID="94b6d87719ae0d9ab237ea42bccaafc7d69756f692cf2a1cd4d0021a84831c82" Jul 09 11:18:02.644536 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.644520 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94b6d87719ae0d9ab237ea42bccaafc7d69756f692cf2a1cd4d0021a84831c82"} err="failed to get container status \"94b6d87719ae0d9ab237ea42bccaafc7d69756f692cf2a1cd4d0021a84831c82\": rpc error: code = NotFound desc = could not find container \"94b6d87719ae0d9ab237ea42bccaafc7d69756f692cf2a1cd4d0021a84831c82\": container with ID starting with 94b6d87719ae0d9ab237ea42bccaafc7d69756f692cf2a1cd4d0021a84831c82 not found: ID does not exist" Jul 09 11:18:02.644574 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.644537 2550 scope.go:117] "RemoveContainer" containerID="8d7d636d20680ccf47960c76e9389f66f7eb18a5c0bf9a697ae267b1f99a921f" Jul 09 11:18:02.644739 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.644719 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d7d636d20680ccf47960c76e9389f66f7eb18a5c0bf9a697ae267b1f99a921f"} err="failed to get container status \"8d7d636d20680ccf47960c76e9389f66f7eb18a5c0bf9a697ae267b1f99a921f\": rpc error: code = NotFound desc = could not find container \"8d7d636d20680ccf47960c76e9389f66f7eb18a5c0bf9a697ae267b1f99a921f\": container with ID starting with 8d7d636d20680ccf47960c76e9389f66f7eb18a5c0bf9a697ae267b1f99a921f not found: ID does not exist" Jul 09 11:18:02.649695 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.649674 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 11:18:02.649930 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.649917 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e08c463-1b72-4e78-9e08-203aa65af9cf" containerName="thanos-sidecar" Jul 09 11:18:02.649972 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.649932 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e08c463-1b72-4e78-9e08-203aa65af9cf" containerName="thanos-sidecar" Jul 09 11:18:02.649972 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.649941 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e08c463-1b72-4e78-9e08-203aa65af9cf" containerName="init-config-reloader" Jul 09 11:18:02.649972 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.649947 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e08c463-1b72-4e78-9e08-203aa65af9cf" containerName="init-config-reloader" Jul 09 11:18:02.649972 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.649956 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e08c463-1b72-4e78-9e08-203aa65af9cf" containerName="prometheus" Jul 09 11:18:02.649972 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.649961 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e08c463-1b72-4e78-9e08-203aa65af9cf" containerName="prometheus" Jul 09 11:18:02.649972 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.649968 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e08c463-1b72-4e78-9e08-203aa65af9cf" containerName="kube-rbac-proxy" Jul 09 11:18:02.649972 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.649975 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e08c463-1b72-4e78-9e08-203aa65af9cf" containerName="kube-rbac-proxy" Jul 09 11:18:02.650171 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.649982 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e08c463-1b72-4e78-9e08-203aa65af9cf" containerName="config-reloader" Jul 09 11:18:02.650171 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.649988 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e08c463-1b72-4e78-9e08-203aa65af9cf" containerName="config-reloader" Jul 09 11:18:02.650171 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.649996 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48f05f33-fe91-4449-a9a8-a03a95d45006" containerName="registry" Jul 09 11:18:02.650171 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.650001 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="48f05f33-fe91-4449-a9a8-a03a95d45006" containerName="registry" Jul 09 11:18:02.650171 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.650008 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e08c463-1b72-4e78-9e08-203aa65af9cf" containerName="kube-rbac-proxy-thanos" Jul 09 11:18:02.650171 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.650013 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e08c463-1b72-4e78-9e08-203aa65af9cf" containerName="kube-rbac-proxy-thanos" Jul 09 11:18:02.650171 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.650021 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e08c463-1b72-4e78-9e08-203aa65af9cf" containerName="kube-rbac-proxy-web" Jul 09 11:18:02.650171 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.650026 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e08c463-1b72-4e78-9e08-203aa65af9cf" containerName="kube-rbac-proxy-web" Jul 09 11:18:02.650171 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.650063 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e08c463-1b72-4e78-9e08-203aa65af9cf" containerName="config-reloader" Jul 09 11:18:02.650171 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.650070 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e08c463-1b72-4e78-9e08-203aa65af9cf" containerName="thanos-sidecar" Jul 09 11:18:02.650171 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.650077 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="48f05f33-fe91-4449-a9a8-a03a95d45006" containerName="registry" Jul 09 11:18:02.650171 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.650081 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e08c463-1b72-4e78-9e08-203aa65af9cf" containerName="kube-rbac-proxy-thanos" Jul 09 11:18:02.650171 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.650088 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e08c463-1b72-4e78-9e08-203aa65af9cf" containerName="prometheus" Jul 09 11:18:02.650171 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.650093 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e08c463-1b72-4e78-9e08-203aa65af9cf" containerName="kube-rbac-proxy-web" Jul 09 11:18:02.650171 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.650098 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e08c463-1b72-4e78-9e08-203aa65af9cf" containerName="kube-rbac-proxy" Jul 09 11:18:02.653884 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.653870 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.656975 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.656954 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 09 11:18:02.657075 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.657065 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 09 11:18:02.657133 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.657118 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7bllvd22d2kub\"" Jul 09 11:18:02.657401 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.657387 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 09 11:18:02.657522 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.657504 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 09 11:18:02.657743 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.657727 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 09 11:18:02.657806 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.657770 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 09 11:18:02.657851 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.657776 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-pgx9k\"" Jul 09 11:18:02.657893 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.657858 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 09 11:18:02.658240 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.658226 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 09 11:18:02.658532 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.658519 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 09 11:18:02.658658 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.658643 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 09 11:18:02.660433 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.660410 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 09 11:18:02.660841 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.660817 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 09 11:18:02.662542 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.662467 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 09 11:18:02.665124 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.665100 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 09 11:18:02.667839 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.667817 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 09 11:18:02.676258 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.676237 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 11:18:02.714298 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.714260 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/54f88ee4-f654-4b86-bce9-b17d7f76835b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.714505 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.714304 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/54f88ee4-f654-4b86-bce9-b17d7f76835b-web-config\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.714505 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.714361 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/54f88ee4-f654-4b86-bce9-b17d7f76835b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.714505 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.714386 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/54f88ee4-f654-4b86-bce9-b17d7f76835b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.714505 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.714406 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vx8cd\" (UniqueName: \"kubernetes.io/projected/54f88ee4-f654-4b86-bce9-b17d7f76835b-kube-api-access-vx8cd\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.714505 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.714427 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54f88ee4-f654-4b86-bce9-b17d7f76835b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.714505 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.714452 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/54f88ee4-f654-4b86-bce9-b17d7f76835b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.714759 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.714510 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/54f88ee4-f654-4b86-bce9-b17d7f76835b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.714759 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.714537 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/54f88ee4-f654-4b86-bce9-b17d7f76835b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.714759 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.714556 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/54f88ee4-f654-4b86-bce9-b17d7f76835b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.714759 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.714574 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/54f88ee4-f654-4b86-bce9-b17d7f76835b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.714759 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.714594 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/54f88ee4-f654-4b86-bce9-b17d7f76835b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.714759 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.714608 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/54f88ee4-f654-4b86-bce9-b17d7f76835b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.714759 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.714625 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/54f88ee4-f654-4b86-bce9-b17d7f76835b-config-out\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.714759 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.714656 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/54f88ee4-f654-4b86-bce9-b17d7f76835b-config\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.714759 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.714693 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54f88ee4-f654-4b86-bce9-b17d7f76835b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.714759 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.714753 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54f88ee4-f654-4b86-bce9-b17d7f76835b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.715107 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.714786 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/54f88ee4-f654-4b86-bce9-b17d7f76835b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.715107 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.714805 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/54f88ee4-f654-4b86-bce9-b17d7f76835b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.715107 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.714825 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/54f88ee4-f654-4b86-bce9-b17d7f76835b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.815628 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.815582 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/54f88ee4-f654-4b86-bce9-b17d7f76835b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.815628 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.815633 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/54f88ee4-f654-4b86-bce9-b17d7f76835b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.815874 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.815659 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/54f88ee4-f654-4b86-bce9-b17d7f76835b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.815874 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.815686 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/54f88ee4-f654-4b86-bce9-b17d7f76835b-config-out\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.815874 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.815715 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/54f88ee4-f654-4b86-bce9-b17d7f76835b-config\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.815874 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.815737 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54f88ee4-f654-4b86-bce9-b17d7f76835b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.815874 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.815767 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54f88ee4-f654-4b86-bce9-b17d7f76835b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.815874 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.815842 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/54f88ee4-f654-4b86-bce9-b17d7f76835b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.816204 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.815892 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/54f88ee4-f654-4b86-bce9-b17d7f76835b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.816269 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.816237 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/54f88ee4-f654-4b86-bce9-b17d7f76835b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.816324 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.816291 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/54f88ee4-f654-4b86-bce9-b17d7f76835b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.816402 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.816324 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/54f88ee4-f654-4b86-bce9-b17d7f76835b-web-config\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.816402 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.816379 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/54f88ee4-f654-4b86-bce9-b17d7f76835b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.816402 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.816380 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/54f88ee4-f654-4b86-bce9-b17d7f76835b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.816560 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.816409 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/54f88ee4-f654-4b86-bce9-b17d7f76835b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.816560 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.816434 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx8cd\" (UniqueName: \"kubernetes.io/projected/54f88ee4-f654-4b86-bce9-b17d7f76835b-kube-api-access-vx8cd\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.816560 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.816459 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54f88ee4-f654-4b86-bce9-b17d7f76835b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.816560 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.816489 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/54f88ee4-f654-4b86-bce9-b17d7f76835b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.816560 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.816524 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/54f88ee4-f654-4b86-bce9-b17d7f76835b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.816560 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.816549 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/54f88ee4-f654-4b86-bce9-b17d7f76835b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.816854 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.816581 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/54f88ee4-f654-4b86-bce9-b17d7f76835b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.816854 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.816590 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54f88ee4-f654-4b86-bce9-b17d7f76835b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.816854 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.816599 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54f88ee4-f654-4b86-bce9-b17d7f76835b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.817233 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.817209 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/54f88ee4-f654-4b86-bce9-b17d7f76835b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.818790 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.818763 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/54f88ee4-f654-4b86-bce9-b17d7f76835b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.818883 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.818764 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/54f88ee4-f654-4b86-bce9-b17d7f76835b-config\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.818883 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.818790 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/54f88ee4-f654-4b86-bce9-b17d7f76835b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.820119 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.819009 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/54f88ee4-f654-4b86-bce9-b17d7f76835b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.820119 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.819091 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/54f88ee4-f654-4b86-bce9-b17d7f76835b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.820119 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.819321 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/54f88ee4-f654-4b86-bce9-b17d7f76835b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.820119 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.819590 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/54f88ee4-f654-4b86-bce9-b17d7f76835b-web-config\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.820119 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.819752 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/54f88ee4-f654-4b86-bce9-b17d7f76835b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.820119 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.820024 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/54f88ee4-f654-4b86-bce9-b17d7f76835b-config-out\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.820513 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.820140 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54f88ee4-f654-4b86-bce9-b17d7f76835b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.820513 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.820165 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/54f88ee4-f654-4b86-bce9-b17d7f76835b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.821114 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.821093 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/54f88ee4-f654-4b86-bce9-b17d7f76835b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.821404 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.821387 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/54f88ee4-f654-4b86-bce9-b17d7f76835b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.821586 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.821568 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/54f88ee4-f654-4b86-bce9-b17d7f76835b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.822126 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.822106 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/54f88ee4-f654-4b86-bce9-b17d7f76835b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.828984 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.828964 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vx8cd\" (UniqueName: \"kubernetes.io/projected/54f88ee4-f654-4b86-bce9-b17d7f76835b-kube-api-access-vx8cd\") pod \"prometheus-k8s-0\" (UID: \"54f88ee4-f654-4b86-bce9-b17d7f76835b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:02.964553 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:02.964473 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:18:03.105854 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:03.105829 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 11:18:03.108148 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:18:03.108123 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54f88ee4_f654_4b86_bce9_b17d7f76835b.slice/crio-663fac543cee05d75b5bb33a7b054ec743be35f680226fcfb2edc3daa65051c7 WatchSource:0}: Error finding container 663fac543cee05d75b5bb33a7b054ec743be35f680226fcfb2edc3daa65051c7: Status 404 returned error can't find the container with id 663fac543cee05d75b5bb33a7b054ec743be35f680226fcfb2edc3daa65051c7 Jul 09 11:18:03.299486 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:03.299452 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e08c463-1b72-4e78-9e08-203aa65af9cf" path="/var/lib/kubelet/pods/0e08c463-1b72-4e78-9e08-203aa65af9cf/volumes" Jul 09 11:18:03.589050 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:03.589013 2550 generic.go:356] "Generic (PLEG): container finished" podID="54f88ee4-f654-4b86-bce9-b17d7f76835b" containerID="cf975fb7df189b4c6c0a17875c6410561f7ec26e51dbb9ba20ca6633c730d5f8" exitCode=0 Jul 09 11:18:03.589222 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:03.589103 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"54f88ee4-f654-4b86-bce9-b17d7f76835b","Type":"ContainerDied","Data":"cf975fb7df189b4c6c0a17875c6410561f7ec26e51dbb9ba20ca6633c730d5f8"} Jul 09 11:18:03.589222 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:03.589178 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"54f88ee4-f654-4b86-bce9-b17d7f76835b","Type":"ContainerStarted","Data":"663fac543cee05d75b5bb33a7b054ec743be35f680226fcfb2edc3daa65051c7"} Jul 09 11:18:04.599250 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:04.596961 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"54f88ee4-f654-4b86-bce9-b17d7f76835b","Type":"ContainerStarted","Data":"d702c6f35476ce4f2222220d1953090d1323c888bd1ce5eda079b36136dd670d"} Jul 09 11:18:04.599250 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:04.597107 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"54f88ee4-f654-4b86-bce9-b17d7f76835b","Type":"ContainerStarted","Data":"9250c0b075df50a7fd30df88fd1ef5352442411fe455b788261b188dc3298a28"} Jul 09 11:18:04.599250 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:04.597163 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"54f88ee4-f654-4b86-bce9-b17d7f76835b","Type":"ContainerStarted","Data":"2db92d36e6c1f53d8a7d202c64097210df7648075ccff80552ec9393b6f9ec36"} Jul 09 11:18:04.599250 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:04.597217 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"54f88ee4-f654-4b86-bce9-b17d7f76835b","Type":"ContainerStarted","Data":"31f58dfb30988849e997dbf403a682a581e3b76ec838e47fce214a396247e4c3"} Jul 09 11:18:04.599250 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:04.597273 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"54f88ee4-f654-4b86-bce9-b17d7f76835b","Type":"ContainerStarted","Data":"839196de874467a270b1850aba0e42d8f458a9d074636f8a609b9a79385f439b"} Jul 09 11:18:04.599250 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:04.597361 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"54f88ee4-f654-4b86-bce9-b17d7f76835b","Type":"ContainerStarted","Data":"9616d995b498cef4edc7b655739f268e1cd2cdabee9580df22c464695c37a777"} Jul 09 11:18:04.632853 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:04.632797 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.612291101 podStartE2EDuration="2.632778201s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.4871ms" isStatefulPod=false podCreationTimestamp="2026-07-09 11:18:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:18:04.631265681 +0000 UTC m=+389.833584833" watchObservedRunningTime="2026-07-09 11:18:04.632778201 +0000 UTC m=+389.835097341" Jul 09 11:18:07.965513 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:18:07.965476 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:19:02.965648 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:19:02.965565 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:19:02.980714 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:19:02.980691 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:19:03.830697 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:19:03.830670 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 11:19:59.266728 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:19:59.266692 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-z9nv7"] Jul 09 11:19:59.268664 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:19:59.268648 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-z9nv7" Jul 09 11:19:59.271804 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:19:59.271772 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 09 11:19:59.271804 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:19:59.271787 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-jn784\"" Jul 09 11:19:59.272939 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:19:59.272922 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 09 11:19:59.281787 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:19:59.281765 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-z9nv7"] Jul 09 11:19:59.361998 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:19:59.361967 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5c4a144a-965a-44e6-9172-323dc3e7f580-tmp\") pod \"openshift-lws-operator-fd8ccff4c-z9nv7\" (UID: \"5c4a144a-965a-44e6-9172-323dc3e7f580\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-z9nv7" Jul 09 11:19:59.362164 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:19:59.362009 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkxdn\" (UniqueName: \"kubernetes.io/projected/5c4a144a-965a-44e6-9172-323dc3e7f580-kube-api-access-zkxdn\") pod \"openshift-lws-operator-fd8ccff4c-z9nv7\" (UID: \"5c4a144a-965a-44e6-9172-323dc3e7f580\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-z9nv7" Jul 09 11:19:59.462668 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:19:59.462632 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5c4a144a-965a-44e6-9172-323dc3e7f580-tmp\") pod \"openshift-lws-operator-fd8ccff4c-z9nv7\" (UID: \"5c4a144a-965a-44e6-9172-323dc3e7f580\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-z9nv7" Jul 09 11:19:59.462668 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:19:59.462676 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkxdn\" (UniqueName: \"kubernetes.io/projected/5c4a144a-965a-44e6-9172-323dc3e7f580-kube-api-access-zkxdn\") pod \"openshift-lws-operator-fd8ccff4c-z9nv7\" (UID: \"5c4a144a-965a-44e6-9172-323dc3e7f580\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-z9nv7" Jul 09 11:19:59.463033 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:19:59.463012 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5c4a144a-965a-44e6-9172-323dc3e7f580-tmp\") pod \"openshift-lws-operator-fd8ccff4c-z9nv7\" (UID: \"5c4a144a-965a-44e6-9172-323dc3e7f580\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-z9nv7" Jul 09 11:19:59.473356 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:19:59.473304 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkxdn\" (UniqueName: \"kubernetes.io/projected/5c4a144a-965a-44e6-9172-323dc3e7f580-kube-api-access-zkxdn\") pod \"openshift-lws-operator-fd8ccff4c-z9nv7\" (UID: \"5c4a144a-965a-44e6-9172-323dc3e7f580\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-z9nv7" Jul 09 11:19:59.590555 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:19:59.590520 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-z9nv7" Jul 09 11:19:59.717888 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:19:59.717858 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-z9nv7"] Jul 09 11:19:59.721362 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:19:59.721316 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5c4a144a_965a_44e6_9172_323dc3e7f580.slice/crio-7cfaf913349c9ffb72ee32923551ce40c242044e7fa8ca9be7256b44ebac0cbc WatchSource:0}: Error finding container 7cfaf913349c9ffb72ee32923551ce40c242044e7fa8ca9be7256b44ebac0cbc: Status 404 returned error can't find the container with id 7cfaf913349c9ffb72ee32923551ce40c242044e7fa8ca9be7256b44ebac0cbc Jul 09 11:20:00.010146 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:00.010067 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-z9nv7" event={"ID":"5c4a144a-965a-44e6-9172-323dc3e7f580","Type":"ContainerStarted","Data":"7cfaf913349c9ffb72ee32923551ce40c242044e7fa8ca9be7256b44ebac0cbc"} Jul 09 11:20:03.023350 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:03.023298 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-z9nv7" event={"ID":"5c4a144a-965a-44e6-9172-323dc3e7f580","Type":"ContainerStarted","Data":"8a84ea74693d77d5878a0da52176b3409d2b5b137229a6a5e2df2664d91a0c3a"} Jul 09 11:20:03.040180 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:03.040133 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-z9nv7" podStartSLOduration=0.852515661 podStartE2EDuration="4.040115031s" totalImagesPullingTime="3.18759937s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 11:19:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:20:03.038972379 +0000 UTC m=+508.241291544" watchObservedRunningTime="2026-07-09 11:20:03.040115031 +0000 UTC m=+508.242434180" Jul 09 11:20:12.633231 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:12.633195 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-cbdc75774-jwm2l"] Jul 09 11:20:12.636096 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:12.636080 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-cbdc75774-jwm2l" Jul 09 11:20:12.639375 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:12.639353 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 09 11:20:12.640137 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:12.640120 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-khblb\"" Jul 09 11:20:12.640664 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:12.640646 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 09 11:20:12.640825 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:12.640814 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 09 11:20:12.659878 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:12.659847 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-cbdc75774-jwm2l"] Jul 09 11:20:12.669698 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:12.669667 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mr99w\" (UniqueName: \"kubernetes.io/projected/3567c8ce-832f-4fa0-a14b-ac2aa4276bfe-kube-api-access-mr99w\") pod \"lws-controller-manager-cbdc75774-jwm2l\" (UID: \"3567c8ce-832f-4fa0-a14b-ac2aa4276bfe\") " pod="openshift-lws-operator/lws-controller-manager-cbdc75774-jwm2l" Jul 09 11:20:12.669855 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:12.669721 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3567c8ce-832f-4fa0-a14b-ac2aa4276bfe-manager-config\") pod \"lws-controller-manager-cbdc75774-jwm2l\" (UID: \"3567c8ce-832f-4fa0-a14b-ac2aa4276bfe\") " pod="openshift-lws-operator/lws-controller-manager-cbdc75774-jwm2l" Jul 09 11:20:12.669855 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:12.669792 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3567c8ce-832f-4fa0-a14b-ac2aa4276bfe-metrics-cert\") pod \"lws-controller-manager-cbdc75774-jwm2l\" (UID: \"3567c8ce-832f-4fa0-a14b-ac2aa4276bfe\") " pod="openshift-lws-operator/lws-controller-manager-cbdc75774-jwm2l" Jul 09 11:20:12.669855 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:12.669835 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3567c8ce-832f-4fa0-a14b-ac2aa4276bfe-cert\") pod \"lws-controller-manager-cbdc75774-jwm2l\" (UID: \"3567c8ce-832f-4fa0-a14b-ac2aa4276bfe\") " pod="openshift-lws-operator/lws-controller-manager-cbdc75774-jwm2l" Jul 09 11:20:12.771205 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:12.771165 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3567c8ce-832f-4fa0-a14b-ac2aa4276bfe-metrics-cert\") pod \"lws-controller-manager-cbdc75774-jwm2l\" (UID: \"3567c8ce-832f-4fa0-a14b-ac2aa4276bfe\") " pod="openshift-lws-operator/lws-controller-manager-cbdc75774-jwm2l" Jul 09 11:20:12.771205 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:12.771202 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3567c8ce-832f-4fa0-a14b-ac2aa4276bfe-cert\") pod \"lws-controller-manager-cbdc75774-jwm2l\" (UID: \"3567c8ce-832f-4fa0-a14b-ac2aa4276bfe\") " pod="openshift-lws-operator/lws-controller-manager-cbdc75774-jwm2l" Jul 09 11:20:12.771466 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:12.771242 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mr99w\" (UniqueName: \"kubernetes.io/projected/3567c8ce-832f-4fa0-a14b-ac2aa4276bfe-kube-api-access-mr99w\") pod \"lws-controller-manager-cbdc75774-jwm2l\" (UID: \"3567c8ce-832f-4fa0-a14b-ac2aa4276bfe\") " pod="openshift-lws-operator/lws-controller-manager-cbdc75774-jwm2l" Jul 09 11:20:12.771466 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:12.771289 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3567c8ce-832f-4fa0-a14b-ac2aa4276bfe-manager-config\") pod \"lws-controller-manager-cbdc75774-jwm2l\" (UID: \"3567c8ce-832f-4fa0-a14b-ac2aa4276bfe\") " pod="openshift-lws-operator/lws-controller-manager-cbdc75774-jwm2l" Jul 09 11:20:12.771986 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:12.771963 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3567c8ce-832f-4fa0-a14b-ac2aa4276bfe-manager-config\") pod \"lws-controller-manager-cbdc75774-jwm2l\" (UID: \"3567c8ce-832f-4fa0-a14b-ac2aa4276bfe\") " pod="openshift-lws-operator/lws-controller-manager-cbdc75774-jwm2l" Jul 09 11:20:12.773633 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:12.773605 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3567c8ce-832f-4fa0-a14b-ac2aa4276bfe-metrics-cert\") pod \"lws-controller-manager-cbdc75774-jwm2l\" (UID: \"3567c8ce-832f-4fa0-a14b-ac2aa4276bfe\") " pod="openshift-lws-operator/lws-controller-manager-cbdc75774-jwm2l" Jul 09 11:20:12.773858 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:12.773834 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3567c8ce-832f-4fa0-a14b-ac2aa4276bfe-cert\") pod \"lws-controller-manager-cbdc75774-jwm2l\" (UID: \"3567c8ce-832f-4fa0-a14b-ac2aa4276bfe\") " pod="openshift-lws-operator/lws-controller-manager-cbdc75774-jwm2l" Jul 09 11:20:12.781808 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:12.781780 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mr99w\" (UniqueName: \"kubernetes.io/projected/3567c8ce-832f-4fa0-a14b-ac2aa4276bfe-kube-api-access-mr99w\") pod \"lws-controller-manager-cbdc75774-jwm2l\" (UID: \"3567c8ce-832f-4fa0-a14b-ac2aa4276bfe\") " pod="openshift-lws-operator/lws-controller-manager-cbdc75774-jwm2l" Jul 09 11:20:12.945216 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:12.945130 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-cbdc75774-jwm2l" Jul 09 11:20:13.082608 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:13.082577 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-cbdc75774-jwm2l"] Jul 09 11:20:13.085810 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:20:13.085783 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3567c8ce_832f_4fa0_a14b_ac2aa4276bfe.slice/crio-cd6de1efc0caf8f9710b2a88f2c0c7ac8ed03ea65f43120262f6fe8fe2e49133 WatchSource:0}: Error finding container cd6de1efc0caf8f9710b2a88f2c0c7ac8ed03ea65f43120262f6fe8fe2e49133: Status 404 returned error can't find the container with id cd6de1efc0caf8f9710b2a88f2c0c7ac8ed03ea65f43120262f6fe8fe2e49133 Jul 09 11:20:14.068228 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:14.068186 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-cbdc75774-jwm2l" event={"ID":"3567c8ce-832f-4fa0-a14b-ac2aa4276bfe","Type":"ContainerStarted","Data":"cd6de1efc0caf8f9710b2a88f2c0c7ac8ed03ea65f43120262f6fe8fe2e49133"} Jul 09 11:20:15.072985 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:15.072948 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-cbdc75774-jwm2l" event={"ID":"3567c8ce-832f-4fa0-a14b-ac2aa4276bfe","Type":"ContainerStarted","Data":"ed78ef3ba043941eb26797840a524f73dd40a83d4f15b1fd09ebc363e1a28f1b"} Jul 09 11:20:15.073393 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:15.073057 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-cbdc75774-jwm2l" Jul 09 11:20:15.089638 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:15.089589 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-cbdc75774-jwm2l" podStartSLOduration=1.668125197 podStartE2EDuration="3.089569831s" totalImagesPullingTime="1.421444634s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 11:20:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:20:15.088688145 +0000 UTC m=+520.291007295" watchObservedRunningTime="2026-07-09 11:20:15.089569831 +0000 UTC m=+520.291888980" Jul 09 11:20:18.840690 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:18.840659 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65b4d55d4c-k9d8r"] Jul 09 11:20:18.842997 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:18.842975 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-65b4d55d4c-k9d8r" Jul 09 11:20:18.850452 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:18.850427 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 09 11:20:18.850576 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:18.850490 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 09 11:20:18.850642 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:18.850632 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 09 11:20:18.850908 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:18.850895 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-5vbbz\"" Jul 09 11:20:18.858950 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:18.858933 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 09 11:20:18.869577 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:18.869551 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65b4d55d4c-k9d8r"] Jul 09 11:20:18.925122 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:18.925088 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5950a9fd-7f13-4ff3-b4eb-770d730c94f8-manifests\") pod \"opendatahub-operator-controller-manager-65b4d55d4c-k9d8r\" (UID: \"5950a9fd-7f13-4ff3-b4eb-770d730c94f8\") " pod="opendatahub/opendatahub-operator-controller-manager-65b4d55d4c-k9d8r" Jul 09 11:20:18.925122 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:18.925122 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5950a9fd-7f13-4ff3-b4eb-770d730c94f8-webhook-cert\") pod \"opendatahub-operator-controller-manager-65b4d55d4c-k9d8r\" (UID: \"5950a9fd-7f13-4ff3-b4eb-770d730c94f8\") " pod="opendatahub/opendatahub-operator-controller-manager-65b4d55d4c-k9d8r" Jul 09 11:20:18.925323 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:18.925161 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5950a9fd-7f13-4ff3-b4eb-770d730c94f8-tmp\") pod \"opendatahub-operator-controller-manager-65b4d55d4c-k9d8r\" (UID: \"5950a9fd-7f13-4ff3-b4eb-770d730c94f8\") " pod="opendatahub/opendatahub-operator-controller-manager-65b4d55d4c-k9d8r" Jul 09 11:20:18.925323 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:18.925176 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5950a9fd-7f13-4ff3-b4eb-770d730c94f8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65b4d55d4c-k9d8r\" (UID: \"5950a9fd-7f13-4ff3-b4eb-770d730c94f8\") " pod="opendatahub/opendatahub-operator-controller-manager-65b4d55d4c-k9d8r" Jul 09 11:20:18.925323 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:18.925214 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9cnq\" (UniqueName: \"kubernetes.io/projected/5950a9fd-7f13-4ff3-b4eb-770d730c94f8-kube-api-access-g9cnq\") pod \"opendatahub-operator-controller-manager-65b4d55d4c-k9d8r\" (UID: \"5950a9fd-7f13-4ff3-b4eb-770d730c94f8\") " pod="opendatahub/opendatahub-operator-controller-manager-65b4d55d4c-k9d8r" Jul 09 11:20:19.026461 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:19.026425 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5950a9fd-7f13-4ff3-b4eb-770d730c94f8-manifests\") pod \"opendatahub-operator-controller-manager-65b4d55d4c-k9d8r\" (UID: \"5950a9fd-7f13-4ff3-b4eb-770d730c94f8\") " pod="opendatahub/opendatahub-operator-controller-manager-65b4d55d4c-k9d8r" Jul 09 11:20:19.026461 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:19.026462 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5950a9fd-7f13-4ff3-b4eb-770d730c94f8-webhook-cert\") pod \"opendatahub-operator-controller-manager-65b4d55d4c-k9d8r\" (UID: \"5950a9fd-7f13-4ff3-b4eb-770d730c94f8\") " pod="opendatahub/opendatahub-operator-controller-manager-65b4d55d4c-k9d8r" Jul 09 11:20:19.026700 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:19.026496 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5950a9fd-7f13-4ff3-b4eb-770d730c94f8-tmp\") pod \"opendatahub-operator-controller-manager-65b4d55d4c-k9d8r\" (UID: \"5950a9fd-7f13-4ff3-b4eb-770d730c94f8\") " pod="opendatahub/opendatahub-operator-controller-manager-65b4d55d4c-k9d8r" Jul 09 11:20:19.026700 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:19.026515 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5950a9fd-7f13-4ff3-b4eb-770d730c94f8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65b4d55d4c-k9d8r\" (UID: \"5950a9fd-7f13-4ff3-b4eb-770d730c94f8\") " pod="opendatahub/opendatahub-operator-controller-manager-65b4d55d4c-k9d8r" Jul 09 11:20:19.026700 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:19.026530 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9cnq\" (UniqueName: \"kubernetes.io/projected/5950a9fd-7f13-4ff3-b4eb-770d730c94f8-kube-api-access-g9cnq\") pod \"opendatahub-operator-controller-manager-65b4d55d4c-k9d8r\" (UID: \"5950a9fd-7f13-4ff3-b4eb-770d730c94f8\") " pod="opendatahub/opendatahub-operator-controller-manager-65b4d55d4c-k9d8r" Jul 09 11:20:19.026915 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:19.026890 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5950a9fd-7f13-4ff3-b4eb-770d730c94f8-manifests\") pod \"opendatahub-operator-controller-manager-65b4d55d4c-k9d8r\" (UID: \"5950a9fd-7f13-4ff3-b4eb-770d730c94f8\") " pod="opendatahub/opendatahub-operator-controller-manager-65b4d55d4c-k9d8r" Jul 09 11:20:19.028862 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:19.028841 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5950a9fd-7f13-4ff3-b4eb-770d730c94f8-tmp\") pod \"opendatahub-operator-controller-manager-65b4d55d4c-k9d8r\" (UID: \"5950a9fd-7f13-4ff3-b4eb-770d730c94f8\") " pod="opendatahub/opendatahub-operator-controller-manager-65b4d55d4c-k9d8r" Jul 09 11:20:19.028989 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:19.028972 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5950a9fd-7f13-4ff3-b4eb-770d730c94f8-webhook-cert\") pod \"opendatahub-operator-controller-manager-65b4d55d4c-k9d8r\" (UID: \"5950a9fd-7f13-4ff3-b4eb-770d730c94f8\") " pod="opendatahub/opendatahub-operator-controller-manager-65b4d55d4c-k9d8r" Jul 09 11:20:19.029106 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:19.029087 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5950a9fd-7f13-4ff3-b4eb-770d730c94f8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65b4d55d4c-k9d8r\" (UID: \"5950a9fd-7f13-4ff3-b4eb-770d730c94f8\") " pod="opendatahub/opendatahub-operator-controller-manager-65b4d55d4c-k9d8r" Jul 09 11:20:19.038804 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:19.038775 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9cnq\" (UniqueName: \"kubernetes.io/projected/5950a9fd-7f13-4ff3-b4eb-770d730c94f8-kube-api-access-g9cnq\") pod \"opendatahub-operator-controller-manager-65b4d55d4c-k9d8r\" (UID: \"5950a9fd-7f13-4ff3-b4eb-770d730c94f8\") " pod="opendatahub/opendatahub-operator-controller-manager-65b4d55d4c-k9d8r" Jul 09 11:20:19.153007 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:19.152904 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-65b4d55d4c-k9d8r" Jul 09 11:20:19.292179 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:19.292051 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65b4d55d4c-k9d8r"] Jul 09 11:20:19.295082 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:20:19.295054 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5950a9fd_7f13_4ff3_b4eb_770d730c94f8.slice/crio-ab52850507fe08b06c65698ede9a25916a6c188e04b95439ec5df472fab28437 WatchSource:0}: Error finding container ab52850507fe08b06c65698ede9a25916a6c188e04b95439ec5df472fab28437: Status 404 returned error can't find the container with id ab52850507fe08b06c65698ede9a25916a6c188e04b95439ec5df472fab28437 Jul 09 11:20:20.095247 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:20.095204 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65b4d55d4c-k9d8r" event={"ID":"5950a9fd-7f13-4ff3-b4eb-770d730c94f8","Type":"ContainerStarted","Data":"ab52850507fe08b06c65698ede9a25916a6c188e04b95439ec5df472fab28437"} Jul 09 11:20:23.111168 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:23.111110 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65b4d55d4c-k9d8r" event={"ID":"5950a9fd-7f13-4ff3-b4eb-770d730c94f8","Type":"ContainerStarted","Data":"cc708b9b557c53aef8a1178a2ce6886b45579cec6f2026f3a6a2869eb0fdad44"} Jul 09 11:20:24.117420 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:24.117383 2550 generic.go:356] "Generic (PLEG): container finished" podID="5950a9fd-7f13-4ff3-b4eb-770d730c94f8" containerID="cc708b9b557c53aef8a1178a2ce6886b45579cec6f2026f3a6a2869eb0fdad44" exitCode=0 Jul 09 11:20:24.117838 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:24.117469 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65b4d55d4c-k9d8r" event={"ID":"5950a9fd-7f13-4ff3-b4eb-770d730c94f8","Type":"ContainerDied","Data":"cc708b9b557c53aef8a1178a2ce6886b45579cec6f2026f3a6a2869eb0fdad44"} Jul 09 11:20:25.123753 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:25.123720 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65b4d55d4c-k9d8r" event={"ID":"5950a9fd-7f13-4ff3-b4eb-770d730c94f8","Type":"ContainerStarted","Data":"e3af2d43814060fba6af104f1337553d389bf7471c1d892db3fa35d3e981b1ee"} Jul 09 11:20:25.124175 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:25.123794 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-65b4d55d4c-k9d8r" Jul 09 11:20:25.147978 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:25.147926 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-65b4d55d4c-k9d8r" podStartSLOduration=2.7744137330000003 podStartE2EDuration="7.147908075s" totalImagesPullingTime="3.285182393s" totalInitContainerRuntime="1.088311949s" isStatefulPod=false podCreationTimestamp="2026-07-09 11:20:18 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:20:25.147043017 +0000 UTC m=+530.349362179" watchObservedRunningTime="2026-07-09 11:20:25.147908075 +0000 UTC m=+530.350227223" Jul 09 11:20:26.078325 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:26.078286 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-cbdc75774-jwm2l" Jul 09 11:20:36.131098 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:36.131068 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-65b4d55d4c-k9d8r" Jul 09 11:20:43.695676 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:43.695638 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6579588b8b-5wdsw"] Jul 09 11:20:43.709904 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:43.709873 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6579588b8b-5wdsw"] Jul 09 11:20:43.710066 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:43.710005 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6579588b8b-5wdsw" Jul 09 11:20:43.712744 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:43.712720 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 09 11:20:43.713777 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:43.713758 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 09 11:20:43.713903 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:43.713817 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 09 11:20:43.713903 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:43.713817 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 09 11:20:43.714022 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:43.713901 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-hnmpt\"" Jul 09 11:20:43.838723 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:43.838688 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b394d164-b21a-4e26-a9ae-4a46e60ffa05-tls-certs\") pod \"kube-auth-proxy-6579588b8b-5wdsw\" (UID: \"b394d164-b21a-4e26-a9ae-4a46e60ffa05\") " pod="openshift-ingress/kube-auth-proxy-6579588b8b-5wdsw" Jul 09 11:20:43.838902 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:43.838740 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b394d164-b21a-4e26-a9ae-4a46e60ffa05-tmp\") pod \"kube-auth-proxy-6579588b8b-5wdsw\" (UID: \"b394d164-b21a-4e26-a9ae-4a46e60ffa05\") " pod="openshift-ingress/kube-auth-proxy-6579588b8b-5wdsw" Jul 09 11:20:43.838902 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:43.838809 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnhmq\" (UniqueName: \"kubernetes.io/projected/b394d164-b21a-4e26-a9ae-4a46e60ffa05-kube-api-access-jnhmq\") pod \"kube-auth-proxy-6579588b8b-5wdsw\" (UID: \"b394d164-b21a-4e26-a9ae-4a46e60ffa05\") " pod="openshift-ingress/kube-auth-proxy-6579588b8b-5wdsw" Jul 09 11:20:43.939255 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:43.939225 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b394d164-b21a-4e26-a9ae-4a46e60ffa05-tls-certs\") pod \"kube-auth-proxy-6579588b8b-5wdsw\" (UID: \"b394d164-b21a-4e26-a9ae-4a46e60ffa05\") " pod="openshift-ingress/kube-auth-proxy-6579588b8b-5wdsw" Jul 09 11:20:43.939460 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:43.939290 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b394d164-b21a-4e26-a9ae-4a46e60ffa05-tmp\") pod \"kube-auth-proxy-6579588b8b-5wdsw\" (UID: \"b394d164-b21a-4e26-a9ae-4a46e60ffa05\") " pod="openshift-ingress/kube-auth-proxy-6579588b8b-5wdsw" Jul 09 11:20:43.939460 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:43.939313 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnhmq\" (UniqueName: \"kubernetes.io/projected/b394d164-b21a-4e26-a9ae-4a46e60ffa05-kube-api-access-jnhmq\") pod \"kube-auth-proxy-6579588b8b-5wdsw\" (UID: \"b394d164-b21a-4e26-a9ae-4a46e60ffa05\") " pod="openshift-ingress/kube-auth-proxy-6579588b8b-5wdsw" Jul 09 11:20:43.941462 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:43.941437 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b394d164-b21a-4e26-a9ae-4a46e60ffa05-tmp\") pod \"kube-auth-proxy-6579588b8b-5wdsw\" (UID: \"b394d164-b21a-4e26-a9ae-4a46e60ffa05\") " pod="openshift-ingress/kube-auth-proxy-6579588b8b-5wdsw" Jul 09 11:20:43.941693 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:43.941677 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b394d164-b21a-4e26-a9ae-4a46e60ffa05-tls-certs\") pod \"kube-auth-proxy-6579588b8b-5wdsw\" (UID: \"b394d164-b21a-4e26-a9ae-4a46e60ffa05\") " pod="openshift-ingress/kube-auth-proxy-6579588b8b-5wdsw" Jul 09 11:20:43.947906 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:43.947843 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnhmq\" (UniqueName: \"kubernetes.io/projected/b394d164-b21a-4e26-a9ae-4a46e60ffa05-kube-api-access-jnhmq\") pod \"kube-auth-proxy-6579588b8b-5wdsw\" (UID: \"b394d164-b21a-4e26-a9ae-4a46e60ffa05\") " pod="openshift-ingress/kube-auth-proxy-6579588b8b-5wdsw" Jul 09 11:20:44.020554 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:44.020517 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6579588b8b-5wdsw" Jul 09 11:20:44.152109 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:44.151950 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6579588b8b-5wdsw"] Jul 09 11:20:44.154722 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:20:44.154695 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb394d164_b21a_4e26_a9ae_4a46e60ffa05.slice/crio-5c8f7537907a31424659934eae796d5c167a3d81a203c5742883294b847de82e WatchSource:0}: Error finding container 5c8f7537907a31424659934eae796d5c167a3d81a203c5742883294b847de82e: Status 404 returned error can't find the container with id 5c8f7537907a31424659934eae796d5c167a3d81a203c5742883294b847de82e Jul 09 11:20:44.209822 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:44.209736 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6579588b8b-5wdsw" event={"ID":"b394d164-b21a-4e26-a9ae-4a46e60ffa05","Type":"ContainerStarted","Data":"5c8f7537907a31424659934eae796d5c167a3d81a203c5742883294b847de82e"} Jul 09 11:20:47.224223 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:47.224133 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6579588b8b-5wdsw" event={"ID":"b394d164-b21a-4e26-a9ae-4a46e60ffa05","Type":"ContainerStarted","Data":"9eec1116f190b511ac0f0e865747426bcc671f0a7e34cbbb0aaef4f2f833d251"} Jul 09 11:20:47.290825 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:20:47.290767 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6579588b8b-5wdsw" podStartSLOduration=1.5733331769999999 podStartE2EDuration="4.290752247s" totalImagesPullingTime="2.71741907s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 11:20:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:20:47.289531803 +0000 UTC m=+552.491850954" watchObservedRunningTime="2026-07-09 11:20:47.290752247 +0000 UTC m=+552.493071397" Jul 09 11:21:35.188323 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:21:35.188296 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-98ghf_fccccfad-9a68-4bd9-a44f-af22e4339085/console-operator/2.log" Jul 09 11:21:35.188864 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:21:35.188481 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-98ghf_fccccfad-9a68-4bd9-a44f-af22e4339085/console-operator/2.log" Jul 09 11:22:19.003159 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:22:19.003122 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-jjr77"] Jul 09 11:22:19.005167 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:22:19.005148 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jjr77" Jul 09 11:22:19.008503 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:22:19.008470 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 09 11:22:19.008687 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:22:19.008663 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 09 11:22:19.008955 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:22:19.008937 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 09 11:22:19.009592 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:22:19.009575 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-r8k97\"" Jul 09 11:22:19.032447 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:22:19.032420 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-jjr77"] Jul 09 11:22:19.137574 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:22:19.137531 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5l8c4\" (UniqueName: \"kubernetes.io/projected/d5aae076-2bd6-40a4-b379-c3e7f1365d27-kube-api-access-5l8c4\") pod \"dns-operator-controller-manager-864dd78c4c-jjr77\" (UID: \"d5aae076-2bd6-40a4-b379-c3e7f1365d27\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jjr77" Jul 09 11:22:19.238205 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:22:19.238154 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5l8c4\" (UniqueName: \"kubernetes.io/projected/d5aae076-2bd6-40a4-b379-c3e7f1365d27-kube-api-access-5l8c4\") pod \"dns-operator-controller-manager-864dd78c4c-jjr77\" (UID: \"d5aae076-2bd6-40a4-b379-c3e7f1365d27\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jjr77" Jul 09 11:22:19.253789 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:22:19.253717 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5l8c4\" (UniqueName: \"kubernetes.io/projected/d5aae076-2bd6-40a4-b379-c3e7f1365d27-kube-api-access-5l8c4\") pod \"dns-operator-controller-manager-864dd78c4c-jjr77\" (UID: \"d5aae076-2bd6-40a4-b379-c3e7f1365d27\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jjr77" Jul 09 11:22:19.315033 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:22:19.315003 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jjr77" Jul 09 11:22:19.488634 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:22:19.488601 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-jjr77"] Jul 09 11:22:19.492241 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:22:19.492210 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5aae076_2bd6_40a4_b379_c3e7f1365d27.slice/crio-eec671e20be5349aea387d8740148a3f33a9ab6b8c780f3a986d31fba7d298ae WatchSource:0}: Error finding container eec671e20be5349aea387d8740148a3f33a9ab6b8c780f3a986d31fba7d298ae: Status 404 returned error can't find the container with id eec671e20be5349aea387d8740148a3f33a9ab6b8c780f3a986d31fba7d298ae Jul 09 11:22:19.494742 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:22:19.494722 2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 11:22:19.613152 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:22:19.613120 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jjr77" event={"ID":"d5aae076-2bd6-40a4-b379-c3e7f1365d27","Type":"ContainerStarted","Data":"eec671e20be5349aea387d8740148a3f33a9ab6b8c780f3a986d31fba7d298ae"} Jul 09 11:22:21.595051 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:22:21.594982 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-xzgnw"] Jul 09 11:22:21.605900 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:22:21.605664 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xzgnw" Jul 09 11:22:21.609008 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:22:21.608819 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-t7dgp\"" Jul 09 11:22:21.609008 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:22:21.608847 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-xzgnw"] Jul 09 11:22:21.609008 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:22:21.609005 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 09 11:22:21.609371 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:22:21.609351 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 09 11:22:21.659815 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:22:21.659682 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9cb6ed43-eb23-467c-82ef-18e489d53960-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-xzgnw\" (UID: \"9cb6ed43-eb23-467c-82ef-18e489d53960\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xzgnw" Jul 09 11:22:21.659815 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:22:21.659775 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whv75\" (UniqueName: \"kubernetes.io/projected/9cb6ed43-eb23-467c-82ef-18e489d53960-kube-api-access-whv75\") pod \"kuadrant-console-plugin-66b5774985-xzgnw\" (UID: \"9cb6ed43-eb23-467c-82ef-18e489d53960\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xzgnw" Jul 09 11:22:21.660073 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:22:21.659875 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9cb6ed43-eb23-467c-82ef-18e489d53960-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-xzgnw\" (UID: \"9cb6ed43-eb23-467c-82ef-18e489d53960\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xzgnw" Jul 09 11:22:21.761013 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:22:21.760968 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whv75\" (UniqueName: \"kubernetes.io/projected/9cb6ed43-eb23-467c-82ef-18e489d53960-kube-api-access-whv75\") pod \"kuadrant-console-plugin-66b5774985-xzgnw\" (UID: \"9cb6ed43-eb23-467c-82ef-18e489d53960\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xzgnw" Jul 09 11:22:21.761199 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:22:21.761033 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9cb6ed43-eb23-467c-82ef-18e489d53960-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-xzgnw\" (UID: \"9cb6ed43-eb23-467c-82ef-18e489d53960\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xzgnw" Jul 09 11:22:21.761199 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:22:21.761090 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9cb6ed43-eb23-467c-82ef-18e489d53960-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-xzgnw\" (UID: \"9cb6ed43-eb23-467c-82ef-18e489d53960\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xzgnw" Jul 09 11:22:21.761320 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:22:21.761230 2550 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 09 11:22:21.761320 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:22:21.761291 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9cb6ed43-eb23-467c-82ef-18e489d53960-plugin-serving-cert podName:9cb6ed43-eb23-467c-82ef-18e489d53960 nodeName:}" failed. No retries permitted until 2026-07-09 11:22:22.261269532 +0000 UTC m=+647.463588658 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/9cb6ed43-eb23-467c-82ef-18e489d53960-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-xzgnw" (UID: "9cb6ed43-eb23-467c-82ef-18e489d53960") : secret "plugin-serving-cert" not found Jul 09 11:22:21.761831 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:22:21.761812 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9cb6ed43-eb23-467c-82ef-18e489d53960-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-xzgnw\" (UID: \"9cb6ed43-eb23-467c-82ef-18e489d53960\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xzgnw" Jul 09 11:22:21.770908 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:22:21.770884 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-whv75\" (UniqueName: \"kubernetes.io/projected/9cb6ed43-eb23-467c-82ef-18e489d53960-kube-api-access-whv75\") pod \"kuadrant-console-plugin-66b5774985-xzgnw\" (UID: \"9cb6ed43-eb23-467c-82ef-18e489d53960\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xzgnw" Jul 09 11:22:22.267053 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:22:22.266954 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9cb6ed43-eb23-467c-82ef-18e489d53960-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-xzgnw\" (UID: \"9cb6ed43-eb23-467c-82ef-18e489d53960\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xzgnw" Jul 09 11:22:22.269374 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:22:22.269352 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9cb6ed43-eb23-467c-82ef-18e489d53960-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-xzgnw\" (UID: \"9cb6ed43-eb23-467c-82ef-18e489d53960\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xzgnw" Jul 09 11:22:22.518861 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:22:22.518776 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xzgnw" Jul 09 11:22:22.630411 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:22:22.630374 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jjr77" event={"ID":"d5aae076-2bd6-40a4-b379-c3e7f1365d27","Type":"ContainerStarted","Data":"351fda6bce67506c7c3901091557b3468cae514a36b64a108b5e0cc275dd8cdb"} Jul 09 11:22:22.630868 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:22:22.630516 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jjr77" Jul 09 11:22:22.665132 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:22:22.664822 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-xzgnw"] Jul 09 11:22:22.668913 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:22:22.668879 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9cb6ed43_eb23_467c_82ef_18e489d53960.slice/crio-b3a95f428eca872c445a0ba8a49331e88d4a4b21c63e442e3cc5d44a7ac443a6 WatchSource:0}: Error finding container b3a95f428eca872c445a0ba8a49331e88d4a4b21c63e442e3cc5d44a7ac443a6: Status 404 returned error can't find the container with id b3a95f428eca872c445a0ba8a49331e88d4a4b21c63e442e3cc5d44a7ac443a6 Jul 09 11:22:22.670729 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:22:22.670525 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jjr77" podStartSLOduration=2.280602547 podStartE2EDuration="4.670503594s" totalImagesPullingTime="2.389901047s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 11:22:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:22:22.668819884 +0000 UTC m=+647.871139034" watchObservedRunningTime="2026-07-09 11:22:22.670503594 +0000 UTC m=+647.872822745" Jul 09 11:22:23.637977 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:22:23.637937 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xzgnw" event={"ID":"9cb6ed43-eb23-467c-82ef-18e489d53960","Type":"ContainerStarted","Data":"b3a95f428eca872c445a0ba8a49331e88d4a4b21c63e442e3cc5d44a7ac443a6"} Jul 09 11:22:33.640501 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:22:33.640464 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jjr77" Jul 09 11:22:47.757064 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:22:47.757024 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xzgnw" event={"ID":"9cb6ed43-eb23-467c-82ef-18e489d53960","Type":"ContainerStarted","Data":"cdc8a89d60baf1aa946fa32c2c26c0d238db68c06c23ab82f38197767767a349"} Jul 09 11:22:47.776760 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:22:47.776688 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xzgnw" podStartSLOduration=2.458733873 podStartE2EDuration="26.776669314s" totalImagesPullingTime="24.317935441s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 11:22:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:22:47.776496108 +0000 UTC m=+672.978815278" watchObservedRunningTime="2026-07-09 11:22:47.776669314 +0000 UTC m=+672.978988468" Jul 09 11:23:05.557498 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:05.557463 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-hl9kv"] Jul 09 11:23:05.559576 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:05.559550 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-hl9kv" Jul 09 11:23:05.562096 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:05.562073 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-6nmlc\"" Jul 09 11:23:05.569910 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:05.569880 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-hl9kv"] Jul 09 11:23:05.678516 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:05.678485 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnzjf\" (UniqueName: \"kubernetes.io/projected/77d4d25c-791d-4135-8cc9-3dfd3e24ff0a-kube-api-access-nnzjf\") pod \"authorino-58cd955675-hl9kv\" (UID: \"77d4d25c-791d-4135-8cc9-3dfd3e24ff0a\") " pod="kuadrant-system/authorino-58cd955675-hl9kv" Jul 09 11:23:05.779071 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:05.779031 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnzjf\" (UniqueName: \"kubernetes.io/projected/77d4d25c-791d-4135-8cc9-3dfd3e24ff0a-kube-api-access-nnzjf\") pod \"authorino-58cd955675-hl9kv\" (UID: \"77d4d25c-791d-4135-8cc9-3dfd3e24ff0a\") " pod="kuadrant-system/authorino-58cd955675-hl9kv" Jul 09 11:23:05.787898 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:05.787870 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnzjf\" (UniqueName: \"kubernetes.io/projected/77d4d25c-791d-4135-8cc9-3dfd3e24ff0a-kube-api-access-nnzjf\") pod \"authorino-58cd955675-hl9kv\" (UID: \"77d4d25c-791d-4135-8cc9-3dfd3e24ff0a\") " pod="kuadrant-system/authorino-58cd955675-hl9kv" Jul 09 11:23:05.869630 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:05.869545 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-hl9kv" Jul 09 11:23:05.999387 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:05.999351 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-hl9kv"] Jul 09 11:23:06.002539 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:23:06.002509 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod77d4d25c_791d_4135_8cc9_3dfd3e24ff0a.slice/crio-87ad93409495db5aed083bd2dc24791f816f19cb8a31e7a8133943a67a6044ed WatchSource:0}: Error finding container 87ad93409495db5aed083bd2dc24791f816f19cb8a31e7a8133943a67a6044ed: Status 404 returned error can't find the container with id 87ad93409495db5aed083bd2dc24791f816f19cb8a31e7a8133943a67a6044ed Jul 09 11:23:06.845434 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:06.845361 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-hl9kv" event={"ID":"77d4d25c-791d-4135-8cc9-3dfd3e24ff0a","Type":"ContainerStarted","Data":"87ad93409495db5aed083bd2dc24791f816f19cb8a31e7a8133943a67a6044ed"} Jul 09 11:23:09.861225 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:09.861185 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-hl9kv" event={"ID":"77d4d25c-791d-4135-8cc9-3dfd3e24ff0a","Type":"ContainerStarted","Data":"ca74c2f79cd268aefdee1c4949d1450fe09488a1b1324b92bc2d08e0f15de58a"} Jul 09 11:23:09.876555 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:09.876509 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-hl9kv" podStartSLOduration=1.824394957 podStartE2EDuration="4.876493945s" totalImagesPullingTime="3.052098988s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 11:23:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:23:09.875794408 +0000 UTC m=+695.078113558" watchObservedRunningTime="2026-07-09 11:23:09.876493945 +0000 UTC m=+695.078813093" Jul 09 11:23:10.787166 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:10.787129 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-hl9kv"] Jul 09 11:23:11.869388 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:11.869346 2550 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-hl9kv" podUID="77d4d25c-791d-4135-8cc9-3dfd3e24ff0a" containerName="authorino" containerID="cri-o://ca74c2f79cd268aefdee1c4949d1450fe09488a1b1324b92bc2d08e0f15de58a" gracePeriod=30 Jul 09 11:23:12.114663 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:12.114639 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-hl9kv" Jul 09 11:23:12.239255 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:12.239158 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nnzjf\" (UniqueName: \"kubernetes.io/projected/77d4d25c-791d-4135-8cc9-3dfd3e24ff0a-kube-api-access-nnzjf\") pod \"77d4d25c-791d-4135-8cc9-3dfd3e24ff0a\" (UID: \"77d4d25c-791d-4135-8cc9-3dfd3e24ff0a\") " Jul 09 11:23:12.241290 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:12.241259 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77d4d25c-791d-4135-8cc9-3dfd3e24ff0a-kube-api-access-nnzjf" (OuterVolumeSpecName: "kube-api-access-nnzjf") pod "77d4d25c-791d-4135-8cc9-3dfd3e24ff0a" (UID: "77d4d25c-791d-4135-8cc9-3dfd3e24ff0a"). InnerVolumeSpecName "kube-api-access-nnzjf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 11:23:12.340782 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:12.340747 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nnzjf\" (UniqueName: \"kubernetes.io/projected/77d4d25c-791d-4135-8cc9-3dfd3e24ff0a-kube-api-access-nnzjf\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:23:12.875050 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:12.875018 2550 generic.go:356] "Generic (PLEG): container finished" podID="77d4d25c-791d-4135-8cc9-3dfd3e24ff0a" containerID="ca74c2f79cd268aefdee1c4949d1450fe09488a1b1324b92bc2d08e0f15de58a" exitCode=0 Jul 09 11:23:12.875510 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:12.875061 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-hl9kv" Jul 09 11:23:12.875510 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:12.875070 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-hl9kv" event={"ID":"77d4d25c-791d-4135-8cc9-3dfd3e24ff0a","Type":"ContainerDied","Data":"ca74c2f79cd268aefdee1c4949d1450fe09488a1b1324b92bc2d08e0f15de58a"} Jul 09 11:23:12.875510 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:12.875100 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-hl9kv" event={"ID":"77d4d25c-791d-4135-8cc9-3dfd3e24ff0a","Type":"ContainerDied","Data":"87ad93409495db5aed083bd2dc24791f816f19cb8a31e7a8133943a67a6044ed"} Jul 09 11:23:12.875510 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:12.875115 2550 scope.go:117] "RemoveContainer" containerID="ca74c2f79cd268aefdee1c4949d1450fe09488a1b1324b92bc2d08e0f15de58a" Jul 09 11:23:12.883457 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:12.883437 2550 scope.go:117] "RemoveContainer" containerID="ca74c2f79cd268aefdee1c4949d1450fe09488a1b1324b92bc2d08e0f15de58a" Jul 09 11:23:12.883726 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:23:12.883706 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca74c2f79cd268aefdee1c4949d1450fe09488a1b1324b92bc2d08e0f15de58a\": container with ID starting with ca74c2f79cd268aefdee1c4949d1450fe09488a1b1324b92bc2d08e0f15de58a not found: ID does not exist" containerID="ca74c2f79cd268aefdee1c4949d1450fe09488a1b1324b92bc2d08e0f15de58a" Jul 09 11:23:12.883772 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:12.883735 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca74c2f79cd268aefdee1c4949d1450fe09488a1b1324b92bc2d08e0f15de58a"} err="failed to get container status \"ca74c2f79cd268aefdee1c4949d1450fe09488a1b1324b92bc2d08e0f15de58a\": rpc error: code = NotFound desc = could not find container \"ca74c2f79cd268aefdee1c4949d1450fe09488a1b1324b92bc2d08e0f15de58a\": container with ID starting with ca74c2f79cd268aefdee1c4949d1450fe09488a1b1324b92bc2d08e0f15de58a not found: ID does not exist" Jul 09 11:23:12.895575 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:12.895539 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-hl9kv"] Jul 09 11:23:12.899184 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:12.899163 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-hl9kv"] Jul 09 11:23:13.299899 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:13.299872 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77d4d25c-791d-4135-8cc9-3dfd3e24ff0a" path="/var/lib/kubelet/pods/77d4d25c-791d-4135-8cc9-3dfd3e24ff0a/volumes" Jul 09 11:23:39.234498 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:39.234466 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lg2bq"] Jul 09 11:23:39.234956 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:39.234806 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77d4d25c-791d-4135-8cc9-3dfd3e24ff0a" containerName="authorino" Jul 09 11:23:39.234956 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:39.234818 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="77d4d25c-791d-4135-8cc9-3dfd3e24ff0a" containerName="authorino" Jul 09 11:23:39.234956 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:39.234897 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="77d4d25c-791d-4135-8cc9-3dfd3e24ff0a" containerName="authorino" Jul 09 11:23:39.240618 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:39.240601 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-lg2bq" Jul 09 11:23:39.243190 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:39.243171 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-6nmlc\"" Jul 09 11:23:39.247205 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:39.247182 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lg2bq"] Jul 09 11:23:39.271536 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:39.271503 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbzqb\" (UniqueName: \"kubernetes.io/projected/389a520a-98bd-416a-a913-082f0a544cab-kube-api-access-wbzqb\") pod \"authorino-5d4b67c446-lg2bq\" (UID: \"389a520a-98bd-416a-a913-082f0a544cab\") " pod="kuadrant-system/authorino-5d4b67c446-lg2bq" Jul 09 11:23:39.372782 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:39.372745 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbzqb\" (UniqueName: \"kubernetes.io/projected/389a520a-98bd-416a-a913-082f0a544cab-kube-api-access-wbzqb\") pod \"authorino-5d4b67c446-lg2bq\" (UID: \"389a520a-98bd-416a-a913-082f0a544cab\") " pod="kuadrant-system/authorino-5d4b67c446-lg2bq" Jul 09 11:23:39.382384 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:39.382358 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbzqb\" (UniqueName: \"kubernetes.io/projected/389a520a-98bd-416a-a913-082f0a544cab-kube-api-access-wbzqb\") pod \"authorino-5d4b67c446-lg2bq\" (UID: \"389a520a-98bd-416a-a913-082f0a544cab\") " pod="kuadrant-system/authorino-5d4b67c446-lg2bq" Jul 09 11:23:39.506538 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:39.506465 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lg2bq"] Jul 09 11:23:39.506716 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:39.506704 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-lg2bq" Jul 09 11:23:39.543968 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:39.543934 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56c97584f4-ndgqs"] Jul 09 11:23:39.547549 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:39.547528 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c97584f4-ndgqs" Jul 09 11:23:39.550901 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:39.550877 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 09 11:23:39.558801 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:39.558774 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c97584f4-ndgqs"] Jul 09 11:23:39.574468 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:39.574409 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d5837aec-c5f8-42fc-aeed-633265d01574-tls-cert\") pod \"authorino-56c97584f4-ndgqs\" (UID: \"d5837aec-c5f8-42fc-aeed-633265d01574\") " pod="kuadrant-system/authorino-56c97584f4-ndgqs" Jul 09 11:23:39.574721 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:39.574491 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swtml\" (UniqueName: \"kubernetes.io/projected/d5837aec-c5f8-42fc-aeed-633265d01574-kube-api-access-swtml\") pod \"authorino-56c97584f4-ndgqs\" (UID: \"d5837aec-c5f8-42fc-aeed-633265d01574\") " pod="kuadrant-system/authorino-56c97584f4-ndgqs" Jul 09 11:23:39.653181 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:39.653157 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lg2bq"] Jul 09 11:23:39.655523 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:23:39.655494 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod389a520a_98bd_416a_a913_082f0a544cab.slice/crio-47d52f9f50c65e7d1ba7813c7bd9434dd4c838f5d7954c177464facbcc3eb690 WatchSource:0}: Error finding container 47d52f9f50c65e7d1ba7813c7bd9434dd4c838f5d7954c177464facbcc3eb690: Status 404 returned error can't find the container with id 47d52f9f50c65e7d1ba7813c7bd9434dd4c838f5d7954c177464facbcc3eb690 Jul 09 11:23:39.675277 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:39.675248 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d5837aec-c5f8-42fc-aeed-633265d01574-tls-cert\") pod \"authorino-56c97584f4-ndgqs\" (UID: \"d5837aec-c5f8-42fc-aeed-633265d01574\") " pod="kuadrant-system/authorino-56c97584f4-ndgqs" Jul 09 11:23:39.675422 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:39.675289 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swtml\" (UniqueName: \"kubernetes.io/projected/d5837aec-c5f8-42fc-aeed-633265d01574-kube-api-access-swtml\") pod \"authorino-56c97584f4-ndgqs\" (UID: \"d5837aec-c5f8-42fc-aeed-633265d01574\") " pod="kuadrant-system/authorino-56c97584f4-ndgqs" Jul 09 11:23:39.677715 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:39.677689 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d5837aec-c5f8-42fc-aeed-633265d01574-tls-cert\") pod \"authorino-56c97584f4-ndgqs\" (UID: \"d5837aec-c5f8-42fc-aeed-633265d01574\") " pod="kuadrant-system/authorino-56c97584f4-ndgqs" Jul 09 11:23:39.690290 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:39.690265 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-swtml\" (UniqueName: \"kubernetes.io/projected/d5837aec-c5f8-42fc-aeed-633265d01574-kube-api-access-swtml\") pod \"authorino-56c97584f4-ndgqs\" (UID: \"d5837aec-c5f8-42fc-aeed-633265d01574\") " pod="kuadrant-system/authorino-56c97584f4-ndgqs" Jul 09 11:23:39.859651 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:39.859610 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c97584f4-ndgqs" Jul 09 11:23:39.997428 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:39.997351 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-lg2bq" event={"ID":"389a520a-98bd-416a-a913-082f0a544cab","Type":"ContainerStarted","Data":"47d52f9f50c65e7d1ba7813c7bd9434dd4c838f5d7954c177464facbcc3eb690"} Jul 09 11:23:40.017948 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:40.017920 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c97584f4-ndgqs"] Jul 09 11:23:40.020221 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:23:40.020192 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd5837aec_c5f8_42fc_aeed_633265d01574.slice/crio-9aeeb474c3881d9211e2be79c2912fec0329d51fafcf899774e5fb1f31dcd186 WatchSource:0}: Error finding container 9aeeb474c3881d9211e2be79c2912fec0329d51fafcf899774e5fb1f31dcd186: Status 404 returned error can't find the container with id 9aeeb474c3881d9211e2be79c2912fec0329d51fafcf899774e5fb1f31dcd186 Jul 09 11:23:41.002966 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:41.002923 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c97584f4-ndgqs" event={"ID":"d5837aec-c5f8-42fc-aeed-633265d01574","Type":"ContainerStarted","Data":"0c5e2f70f5ac2d48df2b0f59bcc704502e5165fde91749231261b7e66fe3c49a"} Jul 09 11:23:41.003527 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:41.002983 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c97584f4-ndgqs" event={"ID":"d5837aec-c5f8-42fc-aeed-633265d01574","Type":"ContainerStarted","Data":"9aeeb474c3881d9211e2be79c2912fec0329d51fafcf899774e5fb1f31dcd186"} Jul 09 11:23:41.004312 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:41.004290 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-lg2bq" event={"ID":"389a520a-98bd-416a-a913-082f0a544cab","Type":"ContainerStarted","Data":"7534e23e1ec5af010697051f7a2b4f9b28586eb42a9044b761dac8ef12ccd249"} Jul 09 11:23:41.004416 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:41.004313 2550 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-lg2bq" podUID="389a520a-98bd-416a-a913-082f0a544cab" containerName="authorino" containerID="cri-o://7534e23e1ec5af010697051f7a2b4f9b28586eb42a9044b761dac8ef12ccd249" gracePeriod=30 Jul 09 11:23:41.030403 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:41.030353 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-56c97584f4-ndgqs" podStartSLOduration=1.634198977 podStartE2EDuration="2.030315465s" totalImagesPullingTime="396.116488ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 11:23:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:23:41.030115983 +0000 UTC m=+726.232435133" watchObservedRunningTime="2026-07-09 11:23:41.030315465 +0000 UTC m=+726.232634614" Jul 09 11:23:41.236855 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:41.236832 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-lg2bq" Jul 09 11:23:41.289235 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:41.289206 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wbzqb\" (UniqueName: \"kubernetes.io/projected/389a520a-98bd-416a-a913-082f0a544cab-kube-api-access-wbzqb\") pod \"389a520a-98bd-416a-a913-082f0a544cab\" (UID: \"389a520a-98bd-416a-a913-082f0a544cab\") " Jul 09 11:23:41.291308 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:41.291286 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/389a520a-98bd-416a-a913-082f0a544cab-kube-api-access-wbzqb" (OuterVolumeSpecName: "kube-api-access-wbzqb") pod "389a520a-98bd-416a-a913-082f0a544cab" (UID: "389a520a-98bd-416a-a913-082f0a544cab"). InnerVolumeSpecName "kube-api-access-wbzqb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 11:23:41.390455 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:41.390427 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbzqb\" (UniqueName: \"kubernetes.io/projected/389a520a-98bd-416a-a913-082f0a544cab-kube-api-access-wbzqb\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:23:42.010781 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:42.010743 2550 generic.go:356] "Generic (PLEG): container finished" podID="389a520a-98bd-416a-a913-082f0a544cab" containerID="7534e23e1ec5af010697051f7a2b4f9b28586eb42a9044b761dac8ef12ccd249" exitCode=0 Jul 09 11:23:42.011246 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:42.010814 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-lg2bq" Jul 09 11:23:42.011246 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:42.010833 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-lg2bq" event={"ID":"389a520a-98bd-416a-a913-082f0a544cab","Type":"ContainerDied","Data":"7534e23e1ec5af010697051f7a2b4f9b28586eb42a9044b761dac8ef12ccd249"} Jul 09 11:23:42.011246 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:42.010890 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-lg2bq" event={"ID":"389a520a-98bd-416a-a913-082f0a544cab","Type":"ContainerDied","Data":"47d52f9f50c65e7d1ba7813c7bd9434dd4c838f5d7954c177464facbcc3eb690"} Jul 09 11:23:42.011246 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:42.010913 2550 scope.go:117] "RemoveContainer" containerID="7534e23e1ec5af010697051f7a2b4f9b28586eb42a9044b761dac8ef12ccd249" Jul 09 11:23:42.019355 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:42.019319 2550 scope.go:117] "RemoveContainer" containerID="7534e23e1ec5af010697051f7a2b4f9b28586eb42a9044b761dac8ef12ccd249" Jul 09 11:23:42.019629 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:23:42.019611 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7534e23e1ec5af010697051f7a2b4f9b28586eb42a9044b761dac8ef12ccd249\": container with ID starting with 7534e23e1ec5af010697051f7a2b4f9b28586eb42a9044b761dac8ef12ccd249 not found: ID does not exist" containerID="7534e23e1ec5af010697051f7a2b4f9b28586eb42a9044b761dac8ef12ccd249" Jul 09 11:23:42.019685 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:42.019636 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7534e23e1ec5af010697051f7a2b4f9b28586eb42a9044b761dac8ef12ccd249"} err="failed to get container status \"7534e23e1ec5af010697051f7a2b4f9b28586eb42a9044b761dac8ef12ccd249\": rpc error: code = NotFound desc = could not find container \"7534e23e1ec5af010697051f7a2b4f9b28586eb42a9044b761dac8ef12ccd249\": container with ID starting with 7534e23e1ec5af010697051f7a2b4f9b28586eb42a9044b761dac8ef12ccd249 not found: ID does not exist" Jul 09 11:23:42.058603 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:42.058567 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lg2bq"] Jul 09 11:23:42.069799 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:42.069773 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lg2bq"] Jul 09 11:23:43.299503 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:43.299475 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="389a520a-98bd-416a-a913-082f0a544cab" path="/var/lib/kubelet/pods/389a520a-98bd-416a-a913-082f0a544cab/volumes" Jul 09 11:23:46.189873 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:46.189845 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-59bbb887b4-g429m"] Jul 09 11:23:46.190264 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:46.190176 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="389a520a-98bd-416a-a913-082f0a544cab" containerName="authorino" Jul 09 11:23:46.190264 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:46.190188 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="389a520a-98bd-416a-a913-082f0a544cab" containerName="authorino" Jul 09 11:23:46.190264 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:46.190242 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="389a520a-98bd-416a-a913-082f0a544cab" containerName="authorino" Jul 09 11:23:46.194611 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:46.194592 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59bbb887b4-g429m" Jul 09 11:23:46.202911 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:46.202891 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 09 11:23:46.202911 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:46.202904 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-d9swf\"" Jul 09 11:23:46.203091 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:46.202929 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 09 11:23:46.219803 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:46.219779 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-59bbb887b4-g429m"] Jul 09 11:23:46.335174 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:46.335129 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/24a208e8-5e70-4283-96c8-3817161a5d8c-webhook-cert\") pod \"maas-controller-59bbb887b4-g429m\" (UID: \"24a208e8-5e70-4283-96c8-3817161a5d8c\") " pod="opendatahub/maas-controller-59bbb887b4-g429m" Jul 09 11:23:46.335398 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:46.335191 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szthk\" (UniqueName: \"kubernetes.io/projected/24a208e8-5e70-4283-96c8-3817161a5d8c-kube-api-access-szthk\") pod \"maas-controller-59bbb887b4-g429m\" (UID: \"24a208e8-5e70-4283-96c8-3817161a5d8c\") " pod="opendatahub/maas-controller-59bbb887b4-g429m" Jul 09 11:23:46.436179 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:46.436125 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/24a208e8-5e70-4283-96c8-3817161a5d8c-webhook-cert\") pod \"maas-controller-59bbb887b4-g429m\" (UID: \"24a208e8-5e70-4283-96c8-3817161a5d8c\") " pod="opendatahub/maas-controller-59bbb887b4-g429m" Jul 09 11:23:46.436179 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:46.436180 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szthk\" (UniqueName: \"kubernetes.io/projected/24a208e8-5e70-4283-96c8-3817161a5d8c-kube-api-access-szthk\") pod \"maas-controller-59bbb887b4-g429m\" (UID: \"24a208e8-5e70-4283-96c8-3817161a5d8c\") " pod="opendatahub/maas-controller-59bbb887b4-g429m" Jul 09 11:23:46.438664 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:46.438639 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/24a208e8-5e70-4283-96c8-3817161a5d8c-webhook-cert\") pod \"maas-controller-59bbb887b4-g429m\" (UID: \"24a208e8-5e70-4283-96c8-3817161a5d8c\") " pod="opendatahub/maas-controller-59bbb887b4-g429m" Jul 09 11:23:46.447665 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:46.447610 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-szthk\" (UniqueName: \"kubernetes.io/projected/24a208e8-5e70-4283-96c8-3817161a5d8c-kube-api-access-szthk\") pod \"maas-controller-59bbb887b4-g429m\" (UID: \"24a208e8-5e70-4283-96c8-3817161a5d8c\") " pod="opendatahub/maas-controller-59bbb887b4-g429m" Jul 09 11:23:46.504616 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:46.504585 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59bbb887b4-g429m" Jul 09 11:23:46.638391 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:46.638364 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-59bbb887b4-g429m"] Jul 09 11:23:46.640725 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:23:46.640696 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24a208e8_5e70_4283_96c8_3817161a5d8c.slice/crio-360e5903d32fe3a44306e5ef9dc321ecf5b0518ad1792a24e53c0f7d63d64004 WatchSource:0}: Error finding container 360e5903d32fe3a44306e5ef9dc321ecf5b0518ad1792a24e53c0f7d63d64004: Status 404 returned error can't find the container with id 360e5903d32fe3a44306e5ef9dc321ecf5b0518ad1792a24e53c0f7d63d64004 Jul 09 11:23:47.035177 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:47.035141 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59bbb887b4-g429m" event={"ID":"24a208e8-5e70-4283-96c8-3817161a5d8c","Type":"ContainerStarted","Data":"360e5903d32fe3a44306e5ef9dc321ecf5b0518ad1792a24e53c0f7d63d64004"} Jul 09 11:23:50.054170 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:50.054135 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59bbb887b4-g429m" event={"ID":"24a208e8-5e70-4283-96c8-3817161a5d8c","Type":"ContainerStarted","Data":"63db84d4f5249a794d137393bba5e1c5f160e02444b63675c6718e22ee5e13d5"} Jul 09 11:23:50.054554 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:50.054259 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-59bbb887b4-g429m" Jul 09 11:23:50.082902 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:23:50.082853 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-59bbb887b4-g429m" podStartSLOduration=1.627741281 podStartE2EDuration="4.082835884s" totalImagesPullingTime="2.455094603s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 11:23:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:23:50.080790151 +0000 UTC m=+735.283109301" watchObservedRunningTime="2026-07-09 11:23:50.082835884 +0000 UTC m=+735.285155032" Jul 09 11:24:01.065006 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:01.064972 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-59bbb887b4-g429m" Jul 09 11:24:22.007785 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:22.007751 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-gsdht"] Jul 09 11:24:22.014767 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:22.014739 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-gsdht" Jul 09 11:24:22.017306 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:22.017283 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-5vj8g\"" Jul 09 11:24:22.017459 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:22.017286 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 09 11:24:22.020637 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:22.020612 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-gsdht"] Jul 09 11:24:22.044726 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:22.044696 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f482m\" (UniqueName: \"kubernetes.io/projected/14411769-06f0-4fec-9e81-915e42460c39-kube-api-access-f482m\") pod \"payload-pre-processing-7964457696-gsdht\" (UID: \"14411769-06f0-4fec-9e81-915e42460c39\") " pod="openshift-ingress/payload-pre-processing-7964457696-gsdht" Jul 09 11:24:22.044862 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:22.044741 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14411769-06f0-4fec-9e81-915e42460c39-plugins-config-volume\") pod \"payload-pre-processing-7964457696-gsdht\" (UID: \"14411769-06f0-4fec-9e81-915e42460c39\") " pod="openshift-ingress/payload-pre-processing-7964457696-gsdht" Jul 09 11:24:22.104751 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:22.104719 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-g6whk"] Jul 09 11:24:22.108454 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:22.108433 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-g6whk" Jul 09 11:24:22.119772 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:22.119749 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-g6whk"] Jul 09 11:24:22.145488 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:22.145461 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0243f69-6970-4783-98c2-590c6884422a-plugins-config-volume\") pod \"payload-processing-58f86c7798-g6whk\" (UID: \"e0243f69-6970-4783-98c2-590c6884422a\") " pod="openshift-ingress/payload-processing-58f86c7798-g6whk" Jul 09 11:24:22.145679 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:22.145506 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f482m\" (UniqueName: \"kubernetes.io/projected/14411769-06f0-4fec-9e81-915e42460c39-kube-api-access-f482m\") pod \"payload-pre-processing-7964457696-gsdht\" (UID: \"14411769-06f0-4fec-9e81-915e42460c39\") " pod="openshift-ingress/payload-pre-processing-7964457696-gsdht" Jul 09 11:24:22.145679 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:22.145643 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rw5mt\" (UniqueName: \"kubernetes.io/projected/e0243f69-6970-4783-98c2-590c6884422a-kube-api-access-rw5mt\") pod \"payload-processing-58f86c7798-g6whk\" (UID: \"e0243f69-6970-4783-98c2-590c6884422a\") " pod="openshift-ingress/payload-processing-58f86c7798-g6whk" Jul 09 11:24:22.145679 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:22.145696 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14411769-06f0-4fec-9e81-915e42460c39-plugins-config-volume\") pod \"payload-pre-processing-7964457696-gsdht\" (UID: \"14411769-06f0-4fec-9e81-915e42460c39\") " pod="openshift-ingress/payload-pre-processing-7964457696-gsdht" Jul 09 11:24:22.146463 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:22.146444 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14411769-06f0-4fec-9e81-915e42460c39-plugins-config-volume\") pod \"payload-pre-processing-7964457696-gsdht\" (UID: \"14411769-06f0-4fec-9e81-915e42460c39\") " pod="openshift-ingress/payload-pre-processing-7964457696-gsdht" Jul 09 11:24:22.154258 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:22.154236 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f482m\" (UniqueName: \"kubernetes.io/projected/14411769-06f0-4fec-9e81-915e42460c39-kube-api-access-f482m\") pod \"payload-pre-processing-7964457696-gsdht\" (UID: \"14411769-06f0-4fec-9e81-915e42460c39\") " pod="openshift-ingress/payload-pre-processing-7964457696-gsdht" Jul 09 11:24:22.246680 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:22.246646 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0243f69-6970-4783-98c2-590c6884422a-plugins-config-volume\") pod \"payload-processing-58f86c7798-g6whk\" (UID: \"e0243f69-6970-4783-98c2-590c6884422a\") " pod="openshift-ingress/payload-processing-58f86c7798-g6whk" Jul 09 11:24:22.246939 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:22.246710 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rw5mt\" (UniqueName: \"kubernetes.io/projected/e0243f69-6970-4783-98c2-590c6884422a-kube-api-access-rw5mt\") pod \"payload-processing-58f86c7798-g6whk\" (UID: \"e0243f69-6970-4783-98c2-590c6884422a\") " pod="openshift-ingress/payload-processing-58f86c7798-g6whk" Jul 09 11:24:22.247525 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:22.247503 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0243f69-6970-4783-98c2-590c6884422a-plugins-config-volume\") pod \"payload-processing-58f86c7798-g6whk\" (UID: \"e0243f69-6970-4783-98c2-590c6884422a\") " pod="openshift-ingress/payload-processing-58f86c7798-g6whk" Jul 09 11:24:22.255543 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:22.255513 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rw5mt\" (UniqueName: \"kubernetes.io/projected/e0243f69-6970-4783-98c2-590c6884422a-kube-api-access-rw5mt\") pod \"payload-processing-58f86c7798-g6whk\" (UID: \"e0243f69-6970-4783-98c2-590c6884422a\") " pod="openshift-ingress/payload-processing-58f86c7798-g6whk" Jul 09 11:24:22.326322 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:22.326291 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-gsdht" Jul 09 11:24:22.420812 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:22.420769 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-g6whk" Jul 09 11:24:22.462591 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:22.462560 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-gsdht"] Jul 09 11:24:22.465832 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:24:22.465805 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14411769_06f0_4fec_9e81_915e42460c39.slice/crio-2960dffda21554ee3af5326801e03a3b1e4d9e2505d43958be89e183647b5275 WatchSource:0}: Error finding container 2960dffda21554ee3af5326801e03a3b1e4d9e2505d43958be89e183647b5275: Status 404 returned error can't find the container with id 2960dffda21554ee3af5326801e03a3b1e4d9e2505d43958be89e183647b5275 Jul 09 11:24:22.557450 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:22.557417 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-g6whk"] Jul 09 11:24:22.561308 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:24:22.561279 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0243f69_6970_4783_98c2_590c6884422a.slice/crio-61ea2e2d6e53084da1b81a1aa0bf46b58dd3e9fca4f2b79a58d2a1cfa58af517 WatchSource:0}: Error finding container 61ea2e2d6e53084da1b81a1aa0bf46b58dd3e9fca4f2b79a58d2a1cfa58af517: Status 404 returned error can't find the container with id 61ea2e2d6e53084da1b81a1aa0bf46b58dd3e9fca4f2b79a58d2a1cfa58af517 Jul 09 11:24:23.204401 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:23.204362 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-g6whk" event={"ID":"e0243f69-6970-4783-98c2-590c6884422a","Type":"ContainerStarted","Data":"61ea2e2d6e53084da1b81a1aa0bf46b58dd3e9fca4f2b79a58d2a1cfa58af517"} Jul 09 11:24:23.206414 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:23.206383 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-gsdht" event={"ID":"14411769-06f0-4fec-9e81-915e42460c39","Type":"ContainerStarted","Data":"2960dffda21554ee3af5326801e03a3b1e4d9e2505d43958be89e183647b5275"} Jul 09 11:24:24.212899 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:24.212797 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-gsdht" event={"ID":"14411769-06f0-4fec-9e81-915e42460c39","Type":"ContainerStarted","Data":"29e6a21cb2c6b47f8b40d2016b72f0498f2a09bbac2b41e75ffe4e3006b19b87"} Jul 09 11:24:24.212899 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:24.212870 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-gsdht" Jul 09 11:24:24.214399 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:24.214375 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-g6whk" event={"ID":"e0243f69-6970-4783-98c2-590c6884422a","Type":"ContainerStarted","Data":"87b6d32e5216baa3b9d873a836a58de2d12dbf57e7ca2f25ae1b4b71e858c7c6"} Jul 09 11:24:24.214520 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:24.214500 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-g6whk" Jul 09 11:24:24.249676 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:24.249624 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-gsdht" podStartSLOduration=1.8361998339999999 podStartE2EDuration="3.249601967s" totalImagesPullingTime="1.413402133s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 11:24:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:24:24.247816962 +0000 UTC m=+769.450136115" watchObservedRunningTime="2026-07-09 11:24:24.249601967 +0000 UTC m=+769.451921118" Jul 09 11:24:24.278607 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:24.278552 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-g6whk" podStartSLOduration=0.958188564 podStartE2EDuration="2.278532394s" totalImagesPullingTime="1.32034383s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 11:24:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:24:24.278347178 +0000 UTC m=+769.480666324" watchObservedRunningTime="2026-07-09 11:24:24.278532394 +0000 UTC m=+769.480851589" Jul 09 11:24:35.221525 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:35.221499 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-g6whk" Jul 09 11:24:35.221907 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:35.221861 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-gsdht" Jul 09 11:24:41.946125 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:41.946090 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-bbvgz"] Jul 09 11:24:41.950893 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:41.950873 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bbvgz" Jul 09 11:24:41.953509 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:41.953482 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 09 11:24:41.953635 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:41.953610 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-t79pf\"" Jul 09 11:24:41.954669 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:41.954649 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 09 11:24:41.954770 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:41.954655 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 09 11:24:41.970460 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:41.970426 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-bbvgz"] Jul 09 11:24:42.030704 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:42.030667 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee68bc2a-5b49-4282-8b75-6a47ec4231c6-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bbvgz\" (UID: \"ee68bc2a-5b49-4282-8b75-6a47ec4231c6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bbvgz" Jul 09 11:24:42.030704 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:42.030707 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smfhm\" (UniqueName: \"kubernetes.io/projected/ee68bc2a-5b49-4282-8b75-6a47ec4231c6-kube-api-access-smfhm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bbvgz\" (UID: \"ee68bc2a-5b49-4282-8b75-6a47ec4231c6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bbvgz" Jul 09 11:24:42.030920 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:42.030731 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee68bc2a-5b49-4282-8b75-6a47ec4231c6-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bbvgz\" (UID: \"ee68bc2a-5b49-4282-8b75-6a47ec4231c6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bbvgz" Jul 09 11:24:42.030920 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:42.030778 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee68bc2a-5b49-4282-8b75-6a47ec4231c6-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bbvgz\" (UID: \"ee68bc2a-5b49-4282-8b75-6a47ec4231c6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bbvgz" Jul 09 11:24:42.030920 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:42.030797 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee68bc2a-5b49-4282-8b75-6a47ec4231c6-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bbvgz\" (UID: \"ee68bc2a-5b49-4282-8b75-6a47ec4231c6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bbvgz" Jul 09 11:24:42.030920 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:42.030821 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee68bc2a-5b49-4282-8b75-6a47ec4231c6-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bbvgz\" (UID: \"ee68bc2a-5b49-4282-8b75-6a47ec4231c6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bbvgz" Jul 09 11:24:42.131505 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:42.131473 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee68bc2a-5b49-4282-8b75-6a47ec4231c6-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bbvgz\" (UID: \"ee68bc2a-5b49-4282-8b75-6a47ec4231c6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bbvgz" Jul 09 11:24:42.131505 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:42.131514 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smfhm\" (UniqueName: \"kubernetes.io/projected/ee68bc2a-5b49-4282-8b75-6a47ec4231c6-kube-api-access-smfhm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bbvgz\" (UID: \"ee68bc2a-5b49-4282-8b75-6a47ec4231c6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bbvgz" Jul 09 11:24:42.131753 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:42.131545 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee68bc2a-5b49-4282-8b75-6a47ec4231c6-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bbvgz\" (UID: \"ee68bc2a-5b49-4282-8b75-6a47ec4231c6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bbvgz" Jul 09 11:24:42.131753 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:42.131577 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee68bc2a-5b49-4282-8b75-6a47ec4231c6-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bbvgz\" (UID: \"ee68bc2a-5b49-4282-8b75-6a47ec4231c6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bbvgz" Jul 09 11:24:42.131753 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:42.131606 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee68bc2a-5b49-4282-8b75-6a47ec4231c6-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bbvgz\" (UID: \"ee68bc2a-5b49-4282-8b75-6a47ec4231c6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bbvgz" Jul 09 11:24:42.131753 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:42.131646 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee68bc2a-5b49-4282-8b75-6a47ec4231c6-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bbvgz\" (UID: \"ee68bc2a-5b49-4282-8b75-6a47ec4231c6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bbvgz" Jul 09 11:24:42.131951 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:42.131931 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee68bc2a-5b49-4282-8b75-6a47ec4231c6-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bbvgz\" (UID: \"ee68bc2a-5b49-4282-8b75-6a47ec4231c6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bbvgz" Jul 09 11:24:42.132009 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:42.131961 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee68bc2a-5b49-4282-8b75-6a47ec4231c6-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bbvgz\" (UID: \"ee68bc2a-5b49-4282-8b75-6a47ec4231c6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bbvgz" Jul 09 11:24:42.132009 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:42.132000 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee68bc2a-5b49-4282-8b75-6a47ec4231c6-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bbvgz\" (UID: \"ee68bc2a-5b49-4282-8b75-6a47ec4231c6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bbvgz" Jul 09 11:24:42.133968 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:42.133944 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee68bc2a-5b49-4282-8b75-6a47ec4231c6-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bbvgz\" (UID: \"ee68bc2a-5b49-4282-8b75-6a47ec4231c6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bbvgz" Jul 09 11:24:42.134143 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:42.134127 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee68bc2a-5b49-4282-8b75-6a47ec4231c6-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bbvgz\" (UID: \"ee68bc2a-5b49-4282-8b75-6a47ec4231c6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bbvgz" Jul 09 11:24:42.142892 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:42.142862 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-smfhm\" (UniqueName: \"kubernetes.io/projected/ee68bc2a-5b49-4282-8b75-6a47ec4231c6-kube-api-access-smfhm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bbvgz\" (UID: \"ee68bc2a-5b49-4282-8b75-6a47ec4231c6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bbvgz" Jul 09 11:24:42.261992 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:42.261892 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bbvgz" Jul 09 11:24:42.394428 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:42.394400 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-bbvgz"] Jul 09 11:24:42.397012 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:24:42.396986 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee68bc2a_5b49_4282_8b75_6a47ec4231c6.slice/crio-c9f5dc27f7ea546c3b26b8d8378589b79bffc1013c3d0bf38d2eb5c03dc34ed6 WatchSource:0}: Error finding container c9f5dc27f7ea546c3b26b8d8378589b79bffc1013c3d0bf38d2eb5c03dc34ed6: Status 404 returned error can't find the container with id c9f5dc27f7ea546c3b26b8d8378589b79bffc1013c3d0bf38d2eb5c03dc34ed6 Jul 09 11:24:43.303820 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:43.303786 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bbvgz" event={"ID":"ee68bc2a-5b49-4282-8b75-6a47ec4231c6","Type":"ContainerStarted","Data":"c9f5dc27f7ea546c3b26b8d8378589b79bffc1013c3d0bf38d2eb5c03dc34ed6"} Jul 09 11:24:44.310098 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:44.310063 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bbvgz" event={"ID":"ee68bc2a-5b49-4282-8b75-6a47ec4231c6","Type":"ContainerStarted","Data":"6c23a74934368fc5d4c3dc82a4f66929d61861a83fe9c8526085cc82f9af9224"} Jul 09 11:24:44.327033 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:44.326986 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bbvgz" podStartSLOduration=2.166814451 podStartE2EDuration="3.326968025s" totalImagesPullingTime="1.160153574s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 11:24:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:24:44.325936098 +0000 UTC m=+789.528255242" watchObservedRunningTime="2026-07-09 11:24:44.326968025 +0000 UTC m=+789.529287175" Jul 09 11:24:52.262406 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:52.262367 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bbvgz" Jul 09 11:24:52.262930 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:52.262428 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bbvgz" Jul 09 11:24:52.275311 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:52.275284 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bbvgz" Jul 09 11:24:52.358854 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:24:52.358825 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bbvgz" Jul 09 11:25:13.087604 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:13.087529 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-k2bm7"] Jul 09 11:25:13.089783 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:13.089767 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2bm7" Jul 09 11:25:13.092790 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:13.092770 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 09 11:25:13.108183 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:13.108152 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-k2bm7"] Jul 09 11:25:13.207632 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:13.207602 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwhh7\" (UniqueName: \"kubernetes.io/projected/2125e350-8d19-4618-baa3-dbe9fbc19ba3-kube-api-access-kwhh7\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k2bm7\" (UID: \"2125e350-8d19-4618-baa3-dbe9fbc19ba3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2bm7" Jul 09 11:25:13.207799 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:13.207647 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2125e350-8d19-4618-baa3-dbe9fbc19ba3-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k2bm7\" (UID: \"2125e350-8d19-4618-baa3-dbe9fbc19ba3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2bm7" Jul 09 11:25:13.207799 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:13.207675 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2125e350-8d19-4618-baa3-dbe9fbc19ba3-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k2bm7\" (UID: \"2125e350-8d19-4618-baa3-dbe9fbc19ba3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2bm7" Jul 09 11:25:13.207799 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:13.207699 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2125e350-8d19-4618-baa3-dbe9fbc19ba3-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k2bm7\" (UID: \"2125e350-8d19-4618-baa3-dbe9fbc19ba3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2bm7" Jul 09 11:25:13.207799 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:13.207728 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2125e350-8d19-4618-baa3-dbe9fbc19ba3-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k2bm7\" (UID: \"2125e350-8d19-4618-baa3-dbe9fbc19ba3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2bm7" Jul 09 11:25:13.207799 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:13.207768 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2125e350-8d19-4618-baa3-dbe9fbc19ba3-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k2bm7\" (UID: \"2125e350-8d19-4618-baa3-dbe9fbc19ba3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2bm7" Jul 09 11:25:13.309232 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:13.309192 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2125e350-8d19-4618-baa3-dbe9fbc19ba3-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k2bm7\" (UID: \"2125e350-8d19-4618-baa3-dbe9fbc19ba3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2bm7" Jul 09 11:25:13.309468 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:13.309242 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2125e350-8d19-4618-baa3-dbe9fbc19ba3-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k2bm7\" (UID: \"2125e350-8d19-4618-baa3-dbe9fbc19ba3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2bm7" Jul 09 11:25:13.309468 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:13.309287 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2125e350-8d19-4618-baa3-dbe9fbc19ba3-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k2bm7\" (UID: \"2125e350-8d19-4618-baa3-dbe9fbc19ba3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2bm7" Jul 09 11:25:13.309468 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:13.309309 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2125e350-8d19-4618-baa3-dbe9fbc19ba3-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k2bm7\" (UID: \"2125e350-8d19-4618-baa3-dbe9fbc19ba3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2bm7" Jul 09 11:25:13.309468 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:13.309425 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwhh7\" (UniqueName: \"kubernetes.io/projected/2125e350-8d19-4618-baa3-dbe9fbc19ba3-kube-api-access-kwhh7\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k2bm7\" (UID: \"2125e350-8d19-4618-baa3-dbe9fbc19ba3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2bm7" Jul 09 11:25:13.309695 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:13.309478 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2125e350-8d19-4618-baa3-dbe9fbc19ba3-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k2bm7\" (UID: \"2125e350-8d19-4618-baa3-dbe9fbc19ba3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2bm7" Jul 09 11:25:13.309695 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:13.309640 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2125e350-8d19-4618-baa3-dbe9fbc19ba3-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k2bm7\" (UID: \"2125e350-8d19-4618-baa3-dbe9fbc19ba3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2bm7" Jul 09 11:25:13.309802 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:13.309693 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2125e350-8d19-4618-baa3-dbe9fbc19ba3-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k2bm7\" (UID: \"2125e350-8d19-4618-baa3-dbe9fbc19ba3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2bm7" Jul 09 11:25:13.309802 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:13.309768 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2125e350-8d19-4618-baa3-dbe9fbc19ba3-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k2bm7\" (UID: \"2125e350-8d19-4618-baa3-dbe9fbc19ba3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2bm7" Jul 09 11:25:13.311659 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:13.311634 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2125e350-8d19-4618-baa3-dbe9fbc19ba3-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k2bm7\" (UID: \"2125e350-8d19-4618-baa3-dbe9fbc19ba3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2bm7" Jul 09 11:25:13.311866 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:13.311849 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2125e350-8d19-4618-baa3-dbe9fbc19ba3-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k2bm7\" (UID: \"2125e350-8d19-4618-baa3-dbe9fbc19ba3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2bm7" Jul 09 11:25:13.318899 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:13.318873 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwhh7\" (UniqueName: \"kubernetes.io/projected/2125e350-8d19-4618-baa3-dbe9fbc19ba3-kube-api-access-kwhh7\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k2bm7\" (UID: \"2125e350-8d19-4618-baa3-dbe9fbc19ba3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2bm7" Jul 09 11:25:13.400448 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:13.400355 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2bm7" Jul 09 11:25:13.530824 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:13.530801 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-k2bm7"] Jul 09 11:25:13.533532 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:25:13.533503 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2125e350_8d19_4618_baa3_dbe9fbc19ba3.slice/crio-2675f811f72c995dc9c905dd163f637535f4def6a2ff26a31bc022563c79f68e WatchSource:0}: Error finding container 2675f811f72c995dc9c905dd163f637535f4def6a2ff26a31bc022563c79f68e: Status 404 returned error can't find the container with id 2675f811f72c995dc9c905dd163f637535f4def6a2ff26a31bc022563c79f68e Jul 09 11:25:14.444585 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:14.444545 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2bm7" event={"ID":"2125e350-8d19-4618-baa3-dbe9fbc19ba3","Type":"ContainerStarted","Data":"a82a6b4d8eb3dceadea3bfc2b101d7d4ebccab39e3f78b9a147539e2aad491a6"} Jul 09 11:25:14.445038 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:14.444610 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2bm7" event={"ID":"2125e350-8d19-4618-baa3-dbe9fbc19ba3","Type":"ContainerStarted","Data":"2675f811f72c995dc9c905dd163f637535f4def6a2ff26a31bc022563c79f68e"} Jul 09 11:25:14.462796 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:14.462742 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2bm7" podStartSLOduration=1.260691493 podStartE2EDuration="1.462723386s" totalImagesPullingTime="202.031893ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 11:25:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:25:14.462356832 +0000 UTC m=+819.664676005" watchObservedRunningTime="2026-07-09 11:25:14.462723386 +0000 UTC m=+819.665042536" Jul 09 11:25:23.401118 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:23.401075 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2bm7" Jul 09 11:25:23.401705 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:23.401163 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2bm7" Jul 09 11:25:23.413974 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:23.413946 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2bm7" Jul 09 11:25:23.499221 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:23.499195 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2bm7" Jul 09 11:25:32.532259 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:32.532228 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p"] Jul 09 11:25:32.535819 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:32.535800 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p" Jul 09 11:25:32.538347 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:32.538310 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 09 11:25:32.547340 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:32.547299 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p"] Jul 09 11:25:32.688247 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:32.688211 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f6df83b-e283-40e4-aa83-e2cf7a8c1f2f-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p\" (UID: \"7f6df83b-e283-40e4-aa83-e2cf7a8c1f2f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p" Jul 09 11:25:32.688435 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:32.688263 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f6df83b-e283-40e4-aa83-e2cf7a8c1f2f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p\" (UID: \"7f6df83b-e283-40e4-aa83-e2cf7a8c1f2f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p" Jul 09 11:25:32.688435 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:32.688280 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f6df83b-e283-40e4-aa83-e2cf7a8c1f2f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p\" (UID: \"7f6df83b-e283-40e4-aa83-e2cf7a8c1f2f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p" Jul 09 11:25:32.688435 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:32.688375 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7s4l\" (UniqueName: \"kubernetes.io/projected/7f6df83b-e283-40e4-aa83-e2cf7a8c1f2f-kube-api-access-n7s4l\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p\" (UID: \"7f6df83b-e283-40e4-aa83-e2cf7a8c1f2f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p" Jul 09 11:25:32.688435 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:32.688424 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f6df83b-e283-40e4-aa83-e2cf7a8c1f2f-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p\" (UID: \"7f6df83b-e283-40e4-aa83-e2cf7a8c1f2f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p" Jul 09 11:25:32.688576 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:32.688481 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f6df83b-e283-40e4-aa83-e2cf7a8c1f2f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p\" (UID: \"7f6df83b-e283-40e4-aa83-e2cf7a8c1f2f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p" Jul 09 11:25:32.789787 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:32.789756 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7s4l\" (UniqueName: \"kubernetes.io/projected/7f6df83b-e283-40e4-aa83-e2cf7a8c1f2f-kube-api-access-n7s4l\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p\" (UID: \"7f6df83b-e283-40e4-aa83-e2cf7a8c1f2f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p" Jul 09 11:25:32.789955 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:32.789817 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f6df83b-e283-40e4-aa83-e2cf7a8c1f2f-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p\" (UID: \"7f6df83b-e283-40e4-aa83-e2cf7a8c1f2f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p" Jul 09 11:25:32.789955 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:32.789874 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f6df83b-e283-40e4-aa83-e2cf7a8c1f2f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p\" (UID: \"7f6df83b-e283-40e4-aa83-e2cf7a8c1f2f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p" Jul 09 11:25:32.789955 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:32.789913 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f6df83b-e283-40e4-aa83-e2cf7a8c1f2f-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p\" (UID: \"7f6df83b-e283-40e4-aa83-e2cf7a8c1f2f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p" Jul 09 11:25:32.790119 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:32.789955 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f6df83b-e283-40e4-aa83-e2cf7a8c1f2f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p\" (UID: \"7f6df83b-e283-40e4-aa83-e2cf7a8c1f2f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p" Jul 09 11:25:32.790119 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:32.789977 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f6df83b-e283-40e4-aa83-e2cf7a8c1f2f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p\" (UID: \"7f6df83b-e283-40e4-aa83-e2cf7a8c1f2f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p" Jul 09 11:25:32.790302 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:32.790275 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f6df83b-e283-40e4-aa83-e2cf7a8c1f2f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p\" (UID: \"7f6df83b-e283-40e4-aa83-e2cf7a8c1f2f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p" Jul 09 11:25:32.790409 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:32.790356 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f6df83b-e283-40e4-aa83-e2cf7a8c1f2f-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p\" (UID: \"7f6df83b-e283-40e4-aa83-e2cf7a8c1f2f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p" Jul 09 11:25:32.790452 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:32.790431 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f6df83b-e283-40e4-aa83-e2cf7a8c1f2f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p\" (UID: \"7f6df83b-e283-40e4-aa83-e2cf7a8c1f2f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p" Jul 09 11:25:32.792080 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:32.792058 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f6df83b-e283-40e4-aa83-e2cf7a8c1f2f-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p\" (UID: \"7f6df83b-e283-40e4-aa83-e2cf7a8c1f2f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p" Jul 09 11:25:32.792479 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:32.792460 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f6df83b-e283-40e4-aa83-e2cf7a8c1f2f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p\" (UID: \"7f6df83b-e283-40e4-aa83-e2cf7a8c1f2f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p" Jul 09 11:25:32.797892 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:32.797873 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7s4l\" (UniqueName: \"kubernetes.io/projected/7f6df83b-e283-40e4-aa83-e2cf7a8c1f2f-kube-api-access-n7s4l\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p\" (UID: \"7f6df83b-e283-40e4-aa83-e2cf7a8c1f2f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p" Jul 09 11:25:32.846926 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:32.846891 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p" Jul 09 11:25:32.980631 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:32.980594 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p"] Jul 09 11:25:32.982994 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:25:32.982965 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7f6df83b_e283_40e4_aa83_e2cf7a8c1f2f.slice/crio-0b4c58a0f578462f09faced0410b274baad7043e27d1c68cf3b55c0b7eb0edc3 WatchSource:0}: Error finding container 0b4c58a0f578462f09faced0410b274baad7043e27d1c68cf3b55c0b7eb0edc3: Status 404 returned error can't find the container with id 0b4c58a0f578462f09faced0410b274baad7043e27d1c68cf3b55c0b7eb0edc3 Jul 09 11:25:33.532874 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:33.532794 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p" event={"ID":"7f6df83b-e283-40e4-aa83-e2cf7a8c1f2f","Type":"ContainerStarted","Data":"e50033261efd3356a46d69dfc1da46b2e8c40f060dcc3b3971c90a669beebff1"} Jul 09 11:25:33.532874 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:33.532835 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p" event={"ID":"7f6df83b-e283-40e4-aa83-e2cf7a8c1f2f","Type":"ContainerStarted","Data":"0b4c58a0f578462f09faced0410b274baad7043e27d1c68cf3b55c0b7eb0edc3"} Jul 09 11:25:33.553461 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:33.553410 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p" podStartSLOduration=1.351203708 podStartE2EDuration="1.55339353s" totalImagesPullingTime="202.189822ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 11:25:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:25:33.551853237 +0000 UTC m=+838.754172404" watchObservedRunningTime="2026-07-09 11:25:33.55339353 +0000 UTC m=+838.755712680" Jul 09 11:25:42.847908 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:42.847872 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p" Jul 09 11:25:42.848414 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:42.848136 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p" Jul 09 11:25:42.860637 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:42.860613 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p" Jul 09 11:25:43.595963 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:25:43.595933 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p" Jul 09 11:26:32.968037 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:26:32.967955 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c97584f4-ndgqs"] Jul 09 11:26:32.968528 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:26:32.968171 2550 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-56c97584f4-ndgqs" podUID="d5837aec-c5f8-42fc-aeed-633265d01574" containerName="authorino" containerID="cri-o://0c5e2f70f5ac2d48df2b0f59bcc704502e5165fde91749231261b7e66fe3c49a" gracePeriod=30 Jul 09 11:26:33.209571 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:26:33.209544 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c97584f4-ndgqs" Jul 09 11:26:33.241981 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:26:33.241910 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-swtml\" (UniqueName: \"kubernetes.io/projected/d5837aec-c5f8-42fc-aeed-633265d01574-kube-api-access-swtml\") pod \"d5837aec-c5f8-42fc-aeed-633265d01574\" (UID: \"d5837aec-c5f8-42fc-aeed-633265d01574\") " Jul 09 11:26:33.242165 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:26:33.242018 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d5837aec-c5f8-42fc-aeed-633265d01574-tls-cert\") pod \"d5837aec-c5f8-42fc-aeed-633265d01574\" (UID: \"d5837aec-c5f8-42fc-aeed-633265d01574\") " Jul 09 11:26:33.243903 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:26:33.243872 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5837aec-c5f8-42fc-aeed-633265d01574-kube-api-access-swtml" (OuterVolumeSpecName: "kube-api-access-swtml") pod "d5837aec-c5f8-42fc-aeed-633265d01574" (UID: "d5837aec-c5f8-42fc-aeed-633265d01574"). InnerVolumeSpecName "kube-api-access-swtml". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 11:26:33.251479 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:26:33.251460 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5837aec-c5f8-42fc-aeed-633265d01574-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "d5837aec-c5f8-42fc-aeed-633265d01574" (UID: "d5837aec-c5f8-42fc-aeed-633265d01574"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 11:26:33.343052 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:26:33.343016 2550 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d5837aec-c5f8-42fc-aeed-633265d01574-tls-cert\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:26:33.343052 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:26:33.343044 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-swtml\" (UniqueName: \"kubernetes.io/projected/d5837aec-c5f8-42fc-aeed-633265d01574-kube-api-access-swtml\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:26:33.818889 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:26:33.818857 2550 generic.go:356] "Generic (PLEG): container finished" podID="d5837aec-c5f8-42fc-aeed-633265d01574" containerID="0c5e2f70f5ac2d48df2b0f59bcc704502e5165fde91749231261b7e66fe3c49a" exitCode=0 Jul 09 11:26:33.819071 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:26:33.818918 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c97584f4-ndgqs" Jul 09 11:26:33.819071 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:26:33.818932 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c97584f4-ndgqs" event={"ID":"d5837aec-c5f8-42fc-aeed-633265d01574","Type":"ContainerDied","Data":"0c5e2f70f5ac2d48df2b0f59bcc704502e5165fde91749231261b7e66fe3c49a"} Jul 09 11:26:33.819071 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:26:33.818965 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c97584f4-ndgqs" event={"ID":"d5837aec-c5f8-42fc-aeed-633265d01574","Type":"ContainerDied","Data":"9aeeb474c3881d9211e2be79c2912fec0329d51fafcf899774e5fb1f31dcd186"} Jul 09 11:26:33.819071 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:26:33.818983 2550 scope.go:117] "RemoveContainer" containerID="0c5e2f70f5ac2d48df2b0f59bcc704502e5165fde91749231261b7e66fe3c49a" Jul 09 11:26:33.827728 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:26:33.827708 2550 scope.go:117] "RemoveContainer" containerID="0c5e2f70f5ac2d48df2b0f59bcc704502e5165fde91749231261b7e66fe3c49a" Jul 09 11:26:33.827992 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:26:33.827972 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c5e2f70f5ac2d48df2b0f59bcc704502e5165fde91749231261b7e66fe3c49a\": container with ID starting with 0c5e2f70f5ac2d48df2b0f59bcc704502e5165fde91749231261b7e66fe3c49a not found: ID does not exist" containerID="0c5e2f70f5ac2d48df2b0f59bcc704502e5165fde91749231261b7e66fe3c49a" Jul 09 11:26:33.828061 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:26:33.828003 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c5e2f70f5ac2d48df2b0f59bcc704502e5165fde91749231261b7e66fe3c49a"} err="failed to get container status \"0c5e2f70f5ac2d48df2b0f59bcc704502e5165fde91749231261b7e66fe3c49a\": rpc error: code = NotFound desc = could not find container \"0c5e2f70f5ac2d48df2b0f59bcc704502e5165fde91749231261b7e66fe3c49a\": container with ID starting with 0c5e2f70f5ac2d48df2b0f59bcc704502e5165fde91749231261b7e66fe3c49a not found: ID does not exist" Jul 09 11:26:33.837779 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:26:33.837756 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c97584f4-ndgqs"] Jul 09 11:26:33.844094 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:26:33.844071 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-56c97584f4-ndgqs"] Jul 09 11:26:35.219552 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:26:35.219525 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-98ghf_fccccfad-9a68-4bd9-a44f-af22e4339085/console-operator/2.log" Jul 09 11:26:35.220443 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:26:35.220418 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-98ghf_fccccfad-9a68-4bd9-a44f-af22e4339085/console-operator/2.log" Jul 09 11:26:35.300703 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:26:35.300673 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5837aec-c5f8-42fc-aeed-633265d01574" path="/var/lib/kubelet/pods/d5837aec-c5f8-42fc-aeed-633265d01574/volumes" Jul 09 11:27:43.737982 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:27:43.737949 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-59bbb887b4-g429m"] Jul 09 11:27:43.738500 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:27:43.738165 2550 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-59bbb887b4-g429m" podUID="24a208e8-5e70-4283-96c8-3817161a5d8c" containerName="manager" containerID="cri-o://63db84d4f5249a794d137393bba5e1c5f160e02444b63675c6718e22ee5e13d5" gracePeriod=30 Jul 09 11:27:43.979297 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:27:43.979273 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59bbb887b4-g429m" Jul 09 11:27:44.082055 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:27:44.082018 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-szthk\" (UniqueName: \"kubernetes.io/projected/24a208e8-5e70-4283-96c8-3817161a5d8c-kube-api-access-szthk\") pod \"24a208e8-5e70-4283-96c8-3817161a5d8c\" (UID: \"24a208e8-5e70-4283-96c8-3817161a5d8c\") " Jul 09 11:27:44.082252 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:27:44.082097 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/24a208e8-5e70-4283-96c8-3817161a5d8c-webhook-cert\") pod \"24a208e8-5e70-4283-96c8-3817161a5d8c\" (UID: \"24a208e8-5e70-4283-96c8-3817161a5d8c\") " Jul 09 11:27:44.084221 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:27:44.084185 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24a208e8-5e70-4283-96c8-3817161a5d8c-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "24a208e8-5e70-4283-96c8-3817161a5d8c" (UID: "24a208e8-5e70-4283-96c8-3817161a5d8c"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 11:27:44.084356 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:27:44.084249 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24a208e8-5e70-4283-96c8-3817161a5d8c-kube-api-access-szthk" (OuterVolumeSpecName: "kube-api-access-szthk") pod "24a208e8-5e70-4283-96c8-3817161a5d8c" (UID: "24a208e8-5e70-4283-96c8-3817161a5d8c"). InnerVolumeSpecName "kube-api-access-szthk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 11:27:44.148261 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:27:44.148224 2550 generic.go:356] "Generic (PLEG): container finished" podID="24a208e8-5e70-4283-96c8-3817161a5d8c" containerID="63db84d4f5249a794d137393bba5e1c5f160e02444b63675c6718e22ee5e13d5" exitCode=0 Jul 09 11:27:44.148472 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:27:44.148276 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59bbb887b4-g429m" Jul 09 11:27:44.148472 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:27:44.148313 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59bbb887b4-g429m" event={"ID":"24a208e8-5e70-4283-96c8-3817161a5d8c","Type":"ContainerDied","Data":"63db84d4f5249a794d137393bba5e1c5f160e02444b63675c6718e22ee5e13d5"} Jul 09 11:27:44.148472 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:27:44.148393 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59bbb887b4-g429m" event={"ID":"24a208e8-5e70-4283-96c8-3817161a5d8c","Type":"ContainerDied","Data":"360e5903d32fe3a44306e5ef9dc321ecf5b0518ad1792a24e53c0f7d63d64004"} Jul 09 11:27:44.148472 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:27:44.148417 2550 scope.go:117] "RemoveContainer" containerID="63db84d4f5249a794d137393bba5e1c5f160e02444b63675c6718e22ee5e13d5" Jul 09 11:27:44.160008 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:27:44.159991 2550 scope.go:117] "RemoveContainer" containerID="63db84d4f5249a794d137393bba5e1c5f160e02444b63675c6718e22ee5e13d5" Jul 09 11:27:44.160237 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:27:44.160221 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63db84d4f5249a794d137393bba5e1c5f160e02444b63675c6718e22ee5e13d5\": container with ID starting with 63db84d4f5249a794d137393bba5e1c5f160e02444b63675c6718e22ee5e13d5 not found: ID does not exist" containerID="63db84d4f5249a794d137393bba5e1c5f160e02444b63675c6718e22ee5e13d5" Jul 09 11:27:44.160286 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:27:44.160245 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63db84d4f5249a794d137393bba5e1c5f160e02444b63675c6718e22ee5e13d5"} err="failed to get container status \"63db84d4f5249a794d137393bba5e1c5f160e02444b63675c6718e22ee5e13d5\": rpc error: code = NotFound desc = could not find container \"63db84d4f5249a794d137393bba5e1c5f160e02444b63675c6718e22ee5e13d5\": container with ID starting with 63db84d4f5249a794d137393bba5e1c5f160e02444b63675c6718e22ee5e13d5 not found: ID does not exist" Jul 09 11:27:44.174615 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:27:44.174592 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-59bbb887b4-g429m"] Jul 09 11:27:44.179417 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:27:44.179394 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-59bbb887b4-g429m"] Jul 09 11:27:44.183235 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:27:44.183219 2550 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/24a208e8-5e70-4283-96c8-3817161a5d8c-webhook-cert\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:27:44.183293 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:27:44.183237 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szthk\" (UniqueName: \"kubernetes.io/projected/24a208e8-5e70-4283-96c8-3817161a5d8c-kube-api-access-szthk\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:27:45.299390 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:27:45.299360 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24a208e8-5e70-4283-96c8-3817161a5d8c" path="/var/lib/kubelet/pods/24a208e8-5e70-4283-96c8-3817161a5d8c/volumes" Jul 09 11:30:00.145527 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:30:00.145492 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726610-zn9rr"] Jul 09 11:30:00.145970 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:30:00.145832 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5837aec-c5f8-42fc-aeed-633265d01574" containerName="authorino" Jul 09 11:30:00.145970 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:30:00.145843 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5837aec-c5f8-42fc-aeed-633265d01574" containerName="authorino" Jul 09 11:30:00.145970 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:30:00.145861 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24a208e8-5e70-4283-96c8-3817161a5d8c" containerName="manager" Jul 09 11:30:00.145970 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:30:00.145867 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="24a208e8-5e70-4283-96c8-3817161a5d8c" containerName="manager" Jul 09 11:30:00.145970 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:30:00.145926 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="24a208e8-5e70-4283-96c8-3817161a5d8c" containerName="manager" Jul 09 11:30:00.145970 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:30:00.145935 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5837aec-c5f8-42fc-aeed-633265d01574" containerName="authorino" Jul 09 11:30:00.148856 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:30:00.148839 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726610-zn9rr" Jul 09 11:30:00.151313 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:30:00.151273 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-q69zr\"" Jul 09 11:30:00.151463 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:30:00.151341 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 09 11:30:00.151463 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:30:00.151394 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 09 11:30:00.162697 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:30:00.162669 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726610-zn9rr"] Jul 09 11:30:00.305519 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:30:00.305484 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cjlq\" (UniqueName: \"kubernetes.io/projected/27cd695f-b129-4d35-bdbf-9dc06ffede6c-kube-api-access-7cjlq\") pod \"maas-api-key-cleanup-29726610-zn9rr\" (UID: \"27cd695f-b129-4d35-bdbf-9dc06ffede6c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726610-zn9rr" Jul 09 11:30:00.406949 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:30:00.406853 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cjlq\" (UniqueName: \"kubernetes.io/projected/27cd695f-b129-4d35-bdbf-9dc06ffede6c-kube-api-access-7cjlq\") pod \"maas-api-key-cleanup-29726610-zn9rr\" (UID: \"27cd695f-b129-4d35-bdbf-9dc06ffede6c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726610-zn9rr" Jul 09 11:30:00.415189 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:30:00.415157 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cjlq\" (UniqueName: \"kubernetes.io/projected/27cd695f-b129-4d35-bdbf-9dc06ffede6c-kube-api-access-7cjlq\") pod \"maas-api-key-cleanup-29726610-zn9rr\" (UID: \"27cd695f-b129-4d35-bdbf-9dc06ffede6c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726610-zn9rr" Jul 09 11:30:00.459080 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:30:00.459048 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726610-zn9rr" Jul 09 11:30:00.590819 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:30:00.590792 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726610-zn9rr"] Jul 09 11:30:00.592988 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:30:00.592961 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27cd695f_b129_4d35_bdbf_9dc06ffede6c.slice/crio-06a770648c3cc68baebd3e8cabb8aba187f2fca9b2d01e17172ee9ae5bf93669 WatchSource:0}: Error finding container 06a770648c3cc68baebd3e8cabb8aba187f2fca9b2d01e17172ee9ae5bf93669: Status 404 returned error can't find the container with id 06a770648c3cc68baebd3e8cabb8aba187f2fca9b2d01e17172ee9ae5bf93669 Jul 09 11:30:00.595072 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:30:00.595056 2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 11:30:00.758704 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:30:00.758621 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726610-zn9rr" event={"ID":"27cd695f-b129-4d35-bdbf-9dc06ffede6c","Type":"ContainerStarted","Data":"06a770648c3cc68baebd3e8cabb8aba187f2fca9b2d01e17172ee9ae5bf93669"} Jul 09 11:30:04.786086 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:30:04.786045 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726610-zn9rr" event={"ID":"27cd695f-b129-4d35-bdbf-9dc06ffede6c","Type":"ContainerStarted","Data":"3fc5e65e9ed33c8c6e80eca807d60648d92609c9becbd61e9f28d39905b02d9a"} Jul 09 11:30:04.802902 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:30:04.802852 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726610-zn9rr" podStartSLOduration=1.49264443 podStartE2EDuration="4.802832805s" totalImagesPullingTime="3.310188375s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 11:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:30:04.801902298 +0000 UTC m=+1110.004221448" watchObservedRunningTime="2026-07-09 11:30:04.802832805 +0000 UTC m=+1110.005152156" Jul 09 11:31:00.011817 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:31:00.011739 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726610-zn9rr"] Jul 09 11:31:00.012283 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:31:00.011970 2550 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726610-zn9rr" podUID="27cd695f-b129-4d35-bdbf-9dc06ffede6c" containerName="cleanup" containerID="cri-o://3fc5e65e9ed33c8c6e80eca807d60648d92609c9becbd61e9f28d39905b02d9a" gracePeriod=30 Jul 09 11:31:30.173533 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:31:30.173501 2550 generic.go:356] "Generic (PLEG): container finished" podID="27cd695f-b129-4d35-bdbf-9dc06ffede6c" containerID="3fc5e65e9ed33c8c6e80eca807d60648d92609c9becbd61e9f28d39905b02d9a" exitCode=137 Jul 09 11:31:30.173970 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:31:30.173555 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726610-zn9rr" event={"ID":"27cd695f-b129-4d35-bdbf-9dc06ffede6c","Type":"ContainerDied","Data":"3fc5e65e9ed33c8c6e80eca807d60648d92609c9becbd61e9f28d39905b02d9a"} Jul 09 11:31:30.173970 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:31:30.173610 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726610-zn9rr" event={"ID":"27cd695f-b129-4d35-bdbf-9dc06ffede6c","Type":"ContainerDied","Data":"06a770648c3cc68baebd3e8cabb8aba187f2fca9b2d01e17172ee9ae5bf93669"} Jul 09 11:31:30.173970 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:31:30.173630 2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="06a770648c3cc68baebd3e8cabb8aba187f2fca9b2d01e17172ee9ae5bf93669" Jul 09 11:31:30.177975 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:31:30.177954 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726610-zn9rr" Jul 09 11:31:30.320994 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:31:30.320964 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7cjlq\" (UniqueName: \"kubernetes.io/projected/27cd695f-b129-4d35-bdbf-9dc06ffede6c-kube-api-access-7cjlq\") pod \"27cd695f-b129-4d35-bdbf-9dc06ffede6c\" (UID: \"27cd695f-b129-4d35-bdbf-9dc06ffede6c\") " Jul 09 11:31:30.323244 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:31:30.323207 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27cd695f-b129-4d35-bdbf-9dc06ffede6c-kube-api-access-7cjlq" (OuterVolumeSpecName: "kube-api-access-7cjlq") pod "27cd695f-b129-4d35-bdbf-9dc06ffede6c" (UID: "27cd695f-b129-4d35-bdbf-9dc06ffede6c"). InnerVolumeSpecName "kube-api-access-7cjlq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 11:31:30.422201 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:31:30.422160 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7cjlq\" (UniqueName: \"kubernetes.io/projected/27cd695f-b129-4d35-bdbf-9dc06ffede6c-kube-api-access-7cjlq\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:31:31.178344 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:31:31.178311 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726610-zn9rr" Jul 09 11:31:31.198124 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:31:31.198096 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726610-zn9rr"] Jul 09 11:31:31.201865 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:31:31.201842 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726610-zn9rr"] Jul 09 11:31:31.300250 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:31:31.300217 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27cd695f-b129-4d35-bdbf-9dc06ffede6c" path="/var/lib/kubelet/pods/27cd695f-b129-4d35-bdbf-9dc06ffede6c/volumes" Jul 09 11:31:35.244771 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:31:35.244723 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-98ghf_fccccfad-9a68-4bd9-a44f-af22e4339085/console-operator/2.log" Jul 09 11:31:35.247454 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:31:35.247428 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-98ghf_fccccfad-9a68-4bd9-a44f-af22e4339085/console-operator/2.log" Jul 09 11:36:35.273432 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:36:35.273397 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-98ghf_fccccfad-9a68-4bd9-a44f-af22e4339085/console-operator/2.log" Jul 09 11:36:35.276777 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:36:35.276755 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-98ghf_fccccfad-9a68-4bd9-a44f-af22e4339085/console-operator/2.log" Jul 09 11:36:35.310840 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:36:35.310813 2550 scope.go:117] "RemoveContainer" containerID="3fc5e65e9ed33c8c6e80eca807d60648d92609c9becbd61e9f28d39905b02d9a" Jul 09 11:41:35.303718 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:41:35.303693 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-98ghf_fccccfad-9a68-4bd9-a44f-af22e4339085/console-operator/2.log" Jul 09 11:41:35.307998 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:41:35.307961 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-98ghf_fccccfad-9a68-4bd9-a44f-af22e4339085/console-operator/2.log" Jul 09 11:42:58.533229 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:42:58.533152 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-b5eec9ef/e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj"] Jul 09 11:42:58.533721 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:42:58.533543 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27cd695f-b129-4d35-bdbf-9dc06ffede6c" containerName="cleanup" Jul 09 11:42:58.533721 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:42:58.533558 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="27cd695f-b129-4d35-bdbf-9dc06ffede6c" containerName="cleanup" Jul 09 11:42:58.533721 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:42:58.533637 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="27cd695f-b129-4d35-bdbf-9dc06ffede6c" containerName="cleanup" Jul 09 11:42:58.536621 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:42:58.536603 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-b5eec9ef/e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj" Jul 09 11:42:58.540289 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:42:58.540268 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-b5eec9ef\"/\"default-dockercfg-pc8fr\"" Jul 09 11:42:58.540289 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:42:58.540289 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-b5eec9ef\"/\"kube-root-ca.crt\"" Jul 09 11:42:58.540492 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:42:58.540289 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-b5eec9ef\"/\"e2e-lifecycle-model-b5eec9ef-kserve-self-signed-certs\"" Jul 09 11:42:58.540492 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:42:58.540270 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-b5eec9ef\"/\"openshift-service-ca.crt\"" Jul 09 11:42:58.546677 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:42:58.546652 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-b5eec9ef/e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj"] Jul 09 11:42:58.684598 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:42:58.684563 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57gmq\" (UniqueName: \"kubernetes.io/projected/33aecf04-c1fc-4ea1-9d95-e4343dfce110-kube-api-access-57gmq\") pod \"e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj\" (UID: \"33aecf04-c1fc-4ea1-9d95-e4343dfce110\") " pod="ai-tenant-e2e-mt-b5eec9ef/e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj" Jul 09 11:42:58.684598 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:42:58.684606 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33aecf04-c1fc-4ea1-9d95-e4343dfce110-tls-certs\") pod \"e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj\" (UID: \"33aecf04-c1fc-4ea1-9d95-e4343dfce110\") " pod="ai-tenant-e2e-mt-b5eec9ef/e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj" Jul 09 11:42:58.684828 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:42:58.684640 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33aecf04-c1fc-4ea1-9d95-e4343dfce110-dshm\") pod \"e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj\" (UID: \"33aecf04-c1fc-4ea1-9d95-e4343dfce110\") " pod="ai-tenant-e2e-mt-b5eec9ef/e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj" Jul 09 11:42:58.684828 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:42:58.684672 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33aecf04-c1fc-4ea1-9d95-e4343dfce110-tmp-dir\") pod \"e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj\" (UID: \"33aecf04-c1fc-4ea1-9d95-e4343dfce110\") " pod="ai-tenant-e2e-mt-b5eec9ef/e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj" Jul 09 11:42:58.684828 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:42:58.684722 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33aecf04-c1fc-4ea1-9d95-e4343dfce110-home\") pod \"e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj\" (UID: \"33aecf04-c1fc-4ea1-9d95-e4343dfce110\") " pod="ai-tenant-e2e-mt-b5eec9ef/e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj" Jul 09 11:42:58.684828 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:42:58.684746 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33aecf04-c1fc-4ea1-9d95-e4343dfce110-model-cache\") pod \"e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj\" (UID: \"33aecf04-c1fc-4ea1-9d95-e4343dfce110\") " pod="ai-tenant-e2e-mt-b5eec9ef/e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj" Jul 09 11:42:58.786093 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:42:58.786069 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33aecf04-c1fc-4ea1-9d95-e4343dfce110-model-cache\") pod \"e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj\" (UID: \"33aecf04-c1fc-4ea1-9d95-e4343dfce110\") " pod="ai-tenant-e2e-mt-b5eec9ef/e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj" Jul 09 11:42:58.786257 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:42:58.786117 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57gmq\" (UniqueName: \"kubernetes.io/projected/33aecf04-c1fc-4ea1-9d95-e4343dfce110-kube-api-access-57gmq\") pod \"e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj\" (UID: \"33aecf04-c1fc-4ea1-9d95-e4343dfce110\") " pod="ai-tenant-e2e-mt-b5eec9ef/e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj" Jul 09 11:42:58.786257 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:42:58.786142 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33aecf04-c1fc-4ea1-9d95-e4343dfce110-tls-certs\") pod \"e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj\" (UID: \"33aecf04-c1fc-4ea1-9d95-e4343dfce110\") " pod="ai-tenant-e2e-mt-b5eec9ef/e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj" Jul 09 11:42:58.786257 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:42:58.786175 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33aecf04-c1fc-4ea1-9d95-e4343dfce110-dshm\") pod \"e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj\" (UID: \"33aecf04-c1fc-4ea1-9d95-e4343dfce110\") " pod="ai-tenant-e2e-mt-b5eec9ef/e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj" Jul 09 11:42:58.786257 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:42:58.786200 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33aecf04-c1fc-4ea1-9d95-e4343dfce110-tmp-dir\") pod \"e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj\" (UID: \"33aecf04-c1fc-4ea1-9d95-e4343dfce110\") " pod="ai-tenant-e2e-mt-b5eec9ef/e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj" Jul 09 11:42:58.786257 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:42:58.786244 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33aecf04-c1fc-4ea1-9d95-e4343dfce110-home\") pod \"e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj\" (UID: \"33aecf04-c1fc-4ea1-9d95-e4343dfce110\") " pod="ai-tenant-e2e-mt-b5eec9ef/e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj" Jul 09 11:42:58.786653 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:42:58.786633 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33aecf04-c1fc-4ea1-9d95-e4343dfce110-home\") pod \"e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj\" (UID: \"33aecf04-c1fc-4ea1-9d95-e4343dfce110\") " pod="ai-tenant-e2e-mt-b5eec9ef/e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj" Jul 09 11:42:58.786737 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:42:58.786679 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33aecf04-c1fc-4ea1-9d95-e4343dfce110-model-cache\") pod \"e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj\" (UID: \"33aecf04-c1fc-4ea1-9d95-e4343dfce110\") " pod="ai-tenant-e2e-mt-b5eec9ef/e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj" Jul 09 11:42:58.786813 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:42:58.786774 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33aecf04-c1fc-4ea1-9d95-e4343dfce110-tmp-dir\") pod \"e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj\" (UID: \"33aecf04-c1fc-4ea1-9d95-e4343dfce110\") " pod="ai-tenant-e2e-mt-b5eec9ef/e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj" Jul 09 11:42:58.788435 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:42:58.788414 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33aecf04-c1fc-4ea1-9d95-e4343dfce110-dshm\") pod \"e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj\" (UID: \"33aecf04-c1fc-4ea1-9d95-e4343dfce110\") " pod="ai-tenant-e2e-mt-b5eec9ef/e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj" Jul 09 11:42:58.788680 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:42:58.788662 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33aecf04-c1fc-4ea1-9d95-e4343dfce110-tls-certs\") pod \"e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj\" (UID: \"33aecf04-c1fc-4ea1-9d95-e4343dfce110\") " pod="ai-tenant-e2e-mt-b5eec9ef/e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj" Jul 09 11:42:58.794398 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:42:58.794372 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-57gmq\" (UniqueName: \"kubernetes.io/projected/33aecf04-c1fc-4ea1-9d95-e4343dfce110-kube-api-access-57gmq\") pod \"e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj\" (UID: \"33aecf04-c1fc-4ea1-9d95-e4343dfce110\") " pod="ai-tenant-e2e-mt-b5eec9ef/e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj" Jul 09 11:42:58.847671 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:42:58.847637 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-b5eec9ef/e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj" Jul 09 11:42:58.979465 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:42:58.979418 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-b5eec9ef/e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj"] Jul 09 11:42:58.981703 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:42:58.981678 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod33aecf04_c1fc_4ea1_9d95_e4343dfce110.slice/crio-fee82a69e76c7b073a5270e0d6ab4152af52886f91a5b4ba2b8439839f6f5715 WatchSource:0}: Error finding container fee82a69e76c7b073a5270e0d6ab4152af52886f91a5b4ba2b8439839f6f5715: Status 404 returned error can't find the container with id fee82a69e76c7b073a5270e0d6ab4152af52886f91a5b4ba2b8439839f6f5715 Jul 09 11:42:59.284011 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:42:59.283968 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-b5eec9ef/e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj" event={"ID":"33aecf04-c1fc-4ea1-9d95-e4343dfce110","Type":"ContainerStarted","Data":"a6686d5eb3f2caac65ab636227dd92b3e17339e580707b6b4c99c6cee5e4305f"} Jul 09 11:42:59.284189 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:42:59.284021 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-b5eec9ef/e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj" event={"ID":"33aecf04-c1fc-4ea1-9d95-e4343dfce110","Type":"ContainerStarted","Data":"fee82a69e76c7b073a5270e0d6ab4152af52886f91a5b4ba2b8439839f6f5715"} Jul 09 11:42:59.304291 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:42:59.304185 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-b5eec9ef/e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj" podStartSLOduration=1.304169332 podStartE2EDuration="1.304169332s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 11:42:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:42:59.302020887 +0000 UTC m=+1884.504340036" watchObservedRunningTime="2026-07-09 11:42:59.304169332 +0000 UTC m=+1884.506488481" Jul 09 11:43:08.848500 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:43:08.848468 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-b5eec9ef/e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj" Jul 09 11:43:08.849063 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:43:08.848533 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-b5eec9ef/e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj" Jul 09 11:43:08.860847 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:43:08.860824 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-b5eec9ef/e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj" Jul 09 11:43:09.338675 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:43:09.338646 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-b5eec9ef/e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj" Jul 09 11:43:44.840887 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:43:44.840850 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-b5eec9ef/e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj"] Jul 09 11:43:59.861236 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:43:59.861173 2550 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-b5eec9ef/e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj" podUID="33aecf04-c1fc-4ea1-9d95-e4343dfce110" containerName="main" containerID="cri-o://a6686d5eb3f2caac65ab636227dd92b3e17339e580707b6b4c99c6cee5e4305f" gracePeriod=45 Jul 09 11:44:00.116631 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:44:00.116580 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-b5eec9ef/e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj" Jul 09 11:44:00.140198 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:44:00.140162 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33aecf04-c1fc-4ea1-9d95-e4343dfce110-home\") pod \"33aecf04-c1fc-4ea1-9d95-e4343dfce110\" (UID: \"33aecf04-c1fc-4ea1-9d95-e4343dfce110\") " Jul 09 11:44:00.140376 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:44:00.140236 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33aecf04-c1fc-4ea1-9d95-e4343dfce110-tls-certs\") pod \"33aecf04-c1fc-4ea1-9d95-e4343dfce110\" (UID: \"33aecf04-c1fc-4ea1-9d95-e4343dfce110\") " Jul 09 11:44:00.140376 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:44:00.140261 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33aecf04-c1fc-4ea1-9d95-e4343dfce110-tmp-dir\") pod \"33aecf04-c1fc-4ea1-9d95-e4343dfce110\" (UID: \"33aecf04-c1fc-4ea1-9d95-e4343dfce110\") " Jul 09 11:44:00.140376 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:44:00.140285 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33aecf04-c1fc-4ea1-9d95-e4343dfce110-dshm\") pod \"33aecf04-c1fc-4ea1-9d95-e4343dfce110\" (UID: \"33aecf04-c1fc-4ea1-9d95-e4343dfce110\") " Jul 09 11:44:00.140376 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:44:00.140314 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-57gmq\" (UniqueName: \"kubernetes.io/projected/33aecf04-c1fc-4ea1-9d95-e4343dfce110-kube-api-access-57gmq\") pod \"33aecf04-c1fc-4ea1-9d95-e4343dfce110\" (UID: \"33aecf04-c1fc-4ea1-9d95-e4343dfce110\") " Jul 09 11:44:00.140675 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:44:00.140394 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33aecf04-c1fc-4ea1-9d95-e4343dfce110-model-cache\") pod \"33aecf04-c1fc-4ea1-9d95-e4343dfce110\" (UID: \"33aecf04-c1fc-4ea1-9d95-e4343dfce110\") " Jul 09 11:44:00.140675 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:44:00.140480 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33aecf04-c1fc-4ea1-9d95-e4343dfce110-home" (OuterVolumeSpecName: "home") pod "33aecf04-c1fc-4ea1-9d95-e4343dfce110" (UID: "33aecf04-c1fc-4ea1-9d95-e4343dfce110"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 11:44:00.140782 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:44:00.140748 2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33aecf04-c1fc-4ea1-9d95-e4343dfce110-home\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:44:00.140782 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:44:00.140761 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33aecf04-c1fc-4ea1-9d95-e4343dfce110-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "33aecf04-c1fc-4ea1-9d95-e4343dfce110" (UID: "33aecf04-c1fc-4ea1-9d95-e4343dfce110"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 11:44:00.141134 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:44:00.141112 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33aecf04-c1fc-4ea1-9d95-e4343dfce110-model-cache" (OuterVolumeSpecName: "model-cache") pod "33aecf04-c1fc-4ea1-9d95-e4343dfce110" (UID: "33aecf04-c1fc-4ea1-9d95-e4343dfce110"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 11:44:00.143216 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:44:00.143179 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33aecf04-c1fc-4ea1-9d95-e4343dfce110-dshm" (OuterVolumeSpecName: "dshm") pod "33aecf04-c1fc-4ea1-9d95-e4343dfce110" (UID: "33aecf04-c1fc-4ea1-9d95-e4343dfce110"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 11:44:00.143353 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:44:00.143270 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33aecf04-c1fc-4ea1-9d95-e4343dfce110-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "33aecf04-c1fc-4ea1-9d95-e4343dfce110" (UID: "33aecf04-c1fc-4ea1-9d95-e4343dfce110"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 11:44:00.143353 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:44:00.143313 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33aecf04-c1fc-4ea1-9d95-e4343dfce110-kube-api-access-57gmq" (OuterVolumeSpecName: "kube-api-access-57gmq") pod "33aecf04-c1fc-4ea1-9d95-e4343dfce110" (UID: "33aecf04-c1fc-4ea1-9d95-e4343dfce110"). InnerVolumeSpecName "kube-api-access-57gmq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 11:44:00.241161 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:44:00.241127 2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33aecf04-c1fc-4ea1-9d95-e4343dfce110-tls-certs\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:44:00.241161 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:44:00.241172 2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33aecf04-c1fc-4ea1-9d95-e4343dfce110-tmp-dir\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:44:00.241392 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:44:00.241181 2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33aecf04-c1fc-4ea1-9d95-e4343dfce110-dshm\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:44:00.241392 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:44:00.241190 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-57gmq\" (UniqueName: \"kubernetes.io/projected/33aecf04-c1fc-4ea1-9d95-e4343dfce110-kube-api-access-57gmq\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:44:00.241392 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:44:00.241200 2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33aecf04-c1fc-4ea1-9d95-e4343dfce110-model-cache\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:44:00.560229 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:44:00.560198 2550 generic.go:356] "Generic (PLEG): container finished" podID="33aecf04-c1fc-4ea1-9d95-e4343dfce110" containerID="a6686d5eb3f2caac65ab636227dd92b3e17339e580707b6b4c99c6cee5e4305f" exitCode=0 Jul 09 11:44:00.560426 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:44:00.560240 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-b5eec9ef/e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj" Jul 09 11:44:00.560426 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:44:00.560284 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-b5eec9ef/e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj" event={"ID":"33aecf04-c1fc-4ea1-9d95-e4343dfce110","Type":"ContainerDied","Data":"a6686d5eb3f2caac65ab636227dd92b3e17339e580707b6b4c99c6cee5e4305f"} Jul 09 11:44:00.560426 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:44:00.560321 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-b5eec9ef/e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj" event={"ID":"33aecf04-c1fc-4ea1-9d95-e4343dfce110","Type":"ContainerDied","Data":"fee82a69e76c7b073a5270e0d6ab4152af52886f91a5b4ba2b8439839f6f5715"} Jul 09 11:44:00.560426 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:44:00.560356 2550 scope.go:117] "RemoveContainer" containerID="a6686d5eb3f2caac65ab636227dd92b3e17339e580707b6b4c99c6cee5e4305f" Jul 09 11:44:00.569802 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:44:00.569783 2550 scope.go:117] "RemoveContainer" containerID="a6686d5eb3f2caac65ab636227dd92b3e17339e580707b6b4c99c6cee5e4305f" Jul 09 11:44:00.570082 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:44:00.570063 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6686d5eb3f2caac65ab636227dd92b3e17339e580707b6b4c99c6cee5e4305f\": container with ID starting with a6686d5eb3f2caac65ab636227dd92b3e17339e580707b6b4c99c6cee5e4305f not found: ID does not exist" containerID="a6686d5eb3f2caac65ab636227dd92b3e17339e580707b6b4c99c6cee5e4305f" Jul 09 11:44:00.570121 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:44:00.570092 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6686d5eb3f2caac65ab636227dd92b3e17339e580707b6b4c99c6cee5e4305f"} err="failed to get container status \"a6686d5eb3f2caac65ab636227dd92b3e17339e580707b6b4c99c6cee5e4305f\": rpc error: code = NotFound desc = could not find container \"a6686d5eb3f2caac65ab636227dd92b3e17339e580707b6b4c99c6cee5e4305f\": container with ID starting with a6686d5eb3f2caac65ab636227dd92b3e17339e580707b6b4c99c6cee5e4305f not found: ID does not exist" Jul 09 11:44:00.581657 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:44:00.581633 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-b5eec9ef/e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj"] Jul 09 11:44:00.585144 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:44:00.585121 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-b5eec9ef/e2e-lifecycle-model-b5eec9ef-kserve-855cccb779-pnxbj"] Jul 09 11:44:01.300565 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:44:01.300536 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33aecf04-c1fc-4ea1-9d95-e4343dfce110" path="/var/lib/kubelet/pods/33aecf04-c1fc-4ea1-9d95-e4343dfce110/volumes" Jul 09 11:45:00.145545 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:00.145507 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726625-4rf6w"] Jul 09 11:45:00.146025 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:00.145867 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33aecf04-c1fc-4ea1-9d95-e4343dfce110" containerName="main" Jul 09 11:45:00.146025 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:00.145878 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="33aecf04-c1fc-4ea1-9d95-e4343dfce110" containerName="main" Jul 09 11:45:00.146025 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:00.145940 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="33aecf04-c1fc-4ea1-9d95-e4343dfce110" containerName="main" Jul 09 11:45:00.149249 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:00.149234 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726625-4rf6w" Jul 09 11:45:00.152092 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:00.152065 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 09 11:45:00.152225 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:00.152109 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 09 11:45:00.152863 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:00.152847 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-q69zr\"" Jul 09 11:45:00.159863 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:00.159832 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726625-4rf6w"] Jul 09 11:45:00.289183 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:00.289149 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktnp7\" (UniqueName: \"kubernetes.io/projected/cd88de32-82e5-4b18-8296-2e58997d2519-kube-api-access-ktnp7\") pod \"maas-api-key-cleanup-29726625-4rf6w\" (UID: \"cd88de32-82e5-4b18-8296-2e58997d2519\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726625-4rf6w" Jul 09 11:45:00.390610 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:00.390570 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktnp7\" (UniqueName: \"kubernetes.io/projected/cd88de32-82e5-4b18-8296-2e58997d2519-kube-api-access-ktnp7\") pod \"maas-api-key-cleanup-29726625-4rf6w\" (UID: \"cd88de32-82e5-4b18-8296-2e58997d2519\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726625-4rf6w" Jul 09 11:45:00.401466 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:00.401396 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktnp7\" (UniqueName: \"kubernetes.io/projected/cd88de32-82e5-4b18-8296-2e58997d2519-kube-api-access-ktnp7\") pod \"maas-api-key-cleanup-29726625-4rf6w\" (UID: \"cd88de32-82e5-4b18-8296-2e58997d2519\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726625-4rf6w" Jul 09 11:45:00.460316 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:00.460279 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726625-4rf6w" Jul 09 11:45:00.590810 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:00.590784 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726625-4rf6w"] Jul 09 11:45:00.593635 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:45:00.593609 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd88de32_82e5_4b18_8296_2e58997d2519.slice/crio-7b122177a817dc16789f5204e41ea44a3a33ccdcae723413718ed390a56c42a1 WatchSource:0}: Error finding container 7b122177a817dc16789f5204e41ea44a3a33ccdcae723413718ed390a56c42a1: Status 404 returned error can't find the container with id 7b122177a817dc16789f5204e41ea44a3a33ccdcae723413718ed390a56c42a1 Jul 09 11:45:00.840702 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:00.840670 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726625-4rf6w" event={"ID":"cd88de32-82e5-4b18-8296-2e58997d2519","Type":"ContainerStarted","Data":"7b122177a817dc16789f5204e41ea44a3a33ccdcae723413718ed390a56c42a1"} Jul 09 11:45:01.846072 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:01.846038 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726625-4rf6w" event={"ID":"cd88de32-82e5-4b18-8296-2e58997d2519","Type":"ContainerStarted","Data":"9188b397c6f6e8493a8bb3d7173da700de8886b3e7741ab63d9fce3f53b4e8c2"} Jul 09 11:45:01.864998 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:01.864936 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726625-4rf6w" podStartSLOduration=1.864911682 podStartE2EDuration="1.864911682s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 11:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:45:01.862233499 +0000 UTC m=+2007.064552649" watchObservedRunningTime="2026-07-09 11:45:01.864911682 +0000 UTC m=+2007.067230834" Jul 09 11:45:53.889416 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:53.889379 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-944007/test-model-944007-kserve-64786dbfdf-z8nfq"] Jul 09 11:45:53.893290 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:53.893264 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-944007/test-model-944007-kserve-64786dbfdf-z8nfq" Jul 09 11:45:53.916126 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:53.916090 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-944007\"/\"kube-root-ca.crt\"" Jul 09 11:45:53.916262 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:53.916157 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-944007\"/\"test-model-944007-kserve-self-signed-certs\"" Jul 09 11:45:53.917074 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:53.917055 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-944007\"/\"openshift-service-ca.crt\"" Jul 09 11:45:53.917201 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:53.917186 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-944007\"/\"default-dockercfg-jzmrr\"" Jul 09 11:45:53.922686 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:53.922657 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-944007/test-model-944007-kserve-64786dbfdf-z8nfq"] Jul 09 11:45:53.954047 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:53.954006 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e7429a5-fd33-41d5-a04c-187585c014f2-tmp-dir\") pod \"test-model-944007-kserve-64786dbfdf-z8nfq\" (UID: \"4e7429a5-fd33-41d5-a04c-187585c014f2\") " pod="ai-tenant-e2e-inf-a-944007/test-model-944007-kserve-64786dbfdf-z8nfq" Jul 09 11:45:53.954252 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:53.954094 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e7429a5-fd33-41d5-a04c-187585c014f2-home\") pod \"test-model-944007-kserve-64786dbfdf-z8nfq\" (UID: \"4e7429a5-fd33-41d5-a04c-187585c014f2\") " pod="ai-tenant-e2e-inf-a-944007/test-model-944007-kserve-64786dbfdf-z8nfq" Jul 09 11:45:53.954252 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:53.954119 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hj5z\" (UniqueName: \"kubernetes.io/projected/4e7429a5-fd33-41d5-a04c-187585c014f2-kube-api-access-4hj5z\") pod \"test-model-944007-kserve-64786dbfdf-z8nfq\" (UID: \"4e7429a5-fd33-41d5-a04c-187585c014f2\") " pod="ai-tenant-e2e-inf-a-944007/test-model-944007-kserve-64786dbfdf-z8nfq" Jul 09 11:45:53.954252 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:53.954168 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e7429a5-fd33-41d5-a04c-187585c014f2-model-cache\") pod \"test-model-944007-kserve-64786dbfdf-z8nfq\" (UID: \"4e7429a5-fd33-41d5-a04c-187585c014f2\") " pod="ai-tenant-e2e-inf-a-944007/test-model-944007-kserve-64786dbfdf-z8nfq" Jul 09 11:45:53.954252 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:53.954198 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e7429a5-fd33-41d5-a04c-187585c014f2-dshm\") pod \"test-model-944007-kserve-64786dbfdf-z8nfq\" (UID: \"4e7429a5-fd33-41d5-a04c-187585c014f2\") " pod="ai-tenant-e2e-inf-a-944007/test-model-944007-kserve-64786dbfdf-z8nfq" Jul 09 11:45:53.954252 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:53.954222 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e7429a5-fd33-41d5-a04c-187585c014f2-tls-certs\") pod \"test-model-944007-kserve-64786dbfdf-z8nfq\" (UID: \"4e7429a5-fd33-41d5-a04c-187585c014f2\") " pod="ai-tenant-e2e-inf-a-944007/test-model-944007-kserve-64786dbfdf-z8nfq" Jul 09 11:45:54.055380 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:54.055312 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e7429a5-fd33-41d5-a04c-187585c014f2-home\") pod \"test-model-944007-kserve-64786dbfdf-z8nfq\" (UID: \"4e7429a5-fd33-41d5-a04c-187585c014f2\") " pod="ai-tenant-e2e-inf-a-944007/test-model-944007-kserve-64786dbfdf-z8nfq" Jul 09 11:45:54.055380 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:54.055376 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hj5z\" (UniqueName: \"kubernetes.io/projected/4e7429a5-fd33-41d5-a04c-187585c014f2-kube-api-access-4hj5z\") pod \"test-model-944007-kserve-64786dbfdf-z8nfq\" (UID: \"4e7429a5-fd33-41d5-a04c-187585c014f2\") " pod="ai-tenant-e2e-inf-a-944007/test-model-944007-kserve-64786dbfdf-z8nfq" Jul 09 11:45:54.055646 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:54.055424 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e7429a5-fd33-41d5-a04c-187585c014f2-model-cache\") pod \"test-model-944007-kserve-64786dbfdf-z8nfq\" (UID: \"4e7429a5-fd33-41d5-a04c-187585c014f2\") " pod="ai-tenant-e2e-inf-a-944007/test-model-944007-kserve-64786dbfdf-z8nfq" Jul 09 11:45:54.055646 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:54.055449 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e7429a5-fd33-41d5-a04c-187585c014f2-dshm\") pod \"test-model-944007-kserve-64786dbfdf-z8nfq\" (UID: \"4e7429a5-fd33-41d5-a04c-187585c014f2\") " pod="ai-tenant-e2e-inf-a-944007/test-model-944007-kserve-64786dbfdf-z8nfq" Jul 09 11:45:54.055646 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:54.055474 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e7429a5-fd33-41d5-a04c-187585c014f2-tls-certs\") pod \"test-model-944007-kserve-64786dbfdf-z8nfq\" (UID: \"4e7429a5-fd33-41d5-a04c-187585c014f2\") " pod="ai-tenant-e2e-inf-a-944007/test-model-944007-kserve-64786dbfdf-z8nfq" Jul 09 11:45:54.055646 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:54.055515 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e7429a5-fd33-41d5-a04c-187585c014f2-tmp-dir\") pod \"test-model-944007-kserve-64786dbfdf-z8nfq\" (UID: \"4e7429a5-fd33-41d5-a04c-187585c014f2\") " pod="ai-tenant-e2e-inf-a-944007/test-model-944007-kserve-64786dbfdf-z8nfq" Jul 09 11:45:54.055839 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:54.055801 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e7429a5-fd33-41d5-a04c-187585c014f2-home\") pod \"test-model-944007-kserve-64786dbfdf-z8nfq\" (UID: \"4e7429a5-fd33-41d5-a04c-187585c014f2\") " pod="ai-tenant-e2e-inf-a-944007/test-model-944007-kserve-64786dbfdf-z8nfq" Jul 09 11:45:54.055839 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:54.055817 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e7429a5-fd33-41d5-a04c-187585c014f2-model-cache\") pod \"test-model-944007-kserve-64786dbfdf-z8nfq\" (UID: \"4e7429a5-fd33-41d5-a04c-187585c014f2\") " pod="ai-tenant-e2e-inf-a-944007/test-model-944007-kserve-64786dbfdf-z8nfq" Jul 09 11:45:54.055904 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:54.055850 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e7429a5-fd33-41d5-a04c-187585c014f2-tmp-dir\") pod \"test-model-944007-kserve-64786dbfdf-z8nfq\" (UID: \"4e7429a5-fd33-41d5-a04c-187585c014f2\") " pod="ai-tenant-e2e-inf-a-944007/test-model-944007-kserve-64786dbfdf-z8nfq" Jul 09 11:45:54.057860 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:54.057822 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e7429a5-fd33-41d5-a04c-187585c014f2-dshm\") pod \"test-model-944007-kserve-64786dbfdf-z8nfq\" (UID: \"4e7429a5-fd33-41d5-a04c-187585c014f2\") " pod="ai-tenant-e2e-inf-a-944007/test-model-944007-kserve-64786dbfdf-z8nfq" Jul 09 11:45:54.058032 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:54.058014 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e7429a5-fd33-41d5-a04c-187585c014f2-tls-certs\") pod \"test-model-944007-kserve-64786dbfdf-z8nfq\" (UID: \"4e7429a5-fd33-41d5-a04c-187585c014f2\") " pod="ai-tenant-e2e-inf-a-944007/test-model-944007-kserve-64786dbfdf-z8nfq" Jul 09 11:45:54.072642 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:54.072616 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hj5z\" (UniqueName: \"kubernetes.io/projected/4e7429a5-fd33-41d5-a04c-187585c014f2-kube-api-access-4hj5z\") pod \"test-model-944007-kserve-64786dbfdf-z8nfq\" (UID: \"4e7429a5-fd33-41d5-a04c-187585c014f2\") " pod="ai-tenant-e2e-inf-a-944007/test-model-944007-kserve-64786dbfdf-z8nfq" Jul 09 11:45:54.202817 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:54.202725 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-944007/test-model-944007-kserve-64786dbfdf-z8nfq" Jul 09 11:45:54.342772 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:54.342747 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-944007/test-model-944007-kserve-64786dbfdf-z8nfq"] Jul 09 11:45:54.344037 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:45:54.344018 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e7429a5_fd33_41d5_a04c_187585c014f2.slice/crio-fb30ace44f0cd8473ee48ab54e72b8f3387c4031eb60dc307ad4fb746b5c1211 WatchSource:0}: Error finding container fb30ace44f0cd8473ee48ab54e72b8f3387c4031eb60dc307ad4fb746b5c1211: Status 404 returned error can't find the container with id fb30ace44f0cd8473ee48ab54e72b8f3387c4031eb60dc307ad4fb746b5c1211 Jul 09 11:45:55.088707 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:55.088669 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-944007/test-model-944007-kserve-64786dbfdf-z8nfq" event={"ID":"4e7429a5-fd33-41d5-a04c-187585c014f2","Type":"ContainerStarted","Data":"dc795eb9cd296988fb545ad8ea127abe686602b37014c6a755bd4b1ccbf541e8"} Jul 09 11:45:55.089121 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:55.088716 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-944007/test-model-944007-kserve-64786dbfdf-z8nfq" event={"ID":"4e7429a5-fd33-41d5-a04c-187585c014f2","Type":"ContainerStarted","Data":"fb30ace44f0cd8473ee48ab54e72b8f3387c4031eb60dc307ad4fb746b5c1211"} Jul 09 11:45:55.111324 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:45:55.111268 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-944007/test-model-944007-kserve-64786dbfdf-z8nfq" podStartSLOduration=2.1112515 podStartE2EDuration="2.1112515s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 11:45:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:45:55.109786561 +0000 UTC m=+2060.312105710" watchObservedRunningTime="2026-07-09 11:45:55.1112515 +0000 UTC m=+2060.313570649" Jul 09 11:46:00.011614 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:00.011522 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726625-4rf6w"] Jul 09 11:46:00.012013 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:00.011760 2550 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726625-4rf6w" podUID="cd88de32-82e5-4b18-8296-2e58997d2519" containerName="cleanup" containerID="cri-o://9188b397c6f6e8493a8bb3d7173da700de8886b3e7741ab63d9fce3f53b4e8c2" gracePeriod=30 Jul 09 11:46:04.203268 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:04.203232 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-944007/test-model-944007-kserve-64786dbfdf-z8nfq" Jul 09 11:46:04.203694 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:04.203287 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-944007/test-model-944007-kserve-64786dbfdf-z8nfq" Jul 09 11:46:04.215511 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:04.215486 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-944007/test-model-944007-kserve-64786dbfdf-z8nfq" Jul 09 11:46:05.144444 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:05.144411 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-944007/test-model-944007-kserve-64786dbfdf-z8nfq" Jul 09 11:46:15.951812 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:15.951776 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-0d824f/test-model-0d824f-kserve-5ddd94b549-fdt6b"] Jul 09 11:46:15.955106 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:15.955081 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0d824f/test-model-0d824f-kserve-5ddd94b549-fdt6b" Jul 09 11:46:15.957601 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:15.957573 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8-model-cache\") pod \"test-model-0d824f-kserve-5ddd94b549-fdt6b\" (UID: \"1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8\") " pod="ai-tenant-e2e-inf-b-0d824f/test-model-0d824f-kserve-5ddd94b549-fdt6b" Jul 09 11:46:15.957742 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:15.957654 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8-tls-certs\") pod \"test-model-0d824f-kserve-5ddd94b549-fdt6b\" (UID: \"1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8\") " pod="ai-tenant-e2e-inf-b-0d824f/test-model-0d824f-kserve-5ddd94b549-fdt6b" Jul 09 11:46:15.957742 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:15.957696 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmzj5\" (UniqueName: \"kubernetes.io/projected/1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8-kube-api-access-xmzj5\") pod \"test-model-0d824f-kserve-5ddd94b549-fdt6b\" (UID: \"1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8\") " pod="ai-tenant-e2e-inf-b-0d824f/test-model-0d824f-kserve-5ddd94b549-fdt6b" Jul 09 11:46:15.957870 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:15.957750 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8-dshm\") pod \"test-model-0d824f-kserve-5ddd94b549-fdt6b\" (UID: \"1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8\") " pod="ai-tenant-e2e-inf-b-0d824f/test-model-0d824f-kserve-5ddd94b549-fdt6b" Jul 09 11:46:15.957870 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:15.957778 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8-tmp-dir\") pod \"test-model-0d824f-kserve-5ddd94b549-fdt6b\" (UID: \"1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8\") " pod="ai-tenant-e2e-inf-b-0d824f/test-model-0d824f-kserve-5ddd94b549-fdt6b" Jul 09 11:46:15.957870 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:15.957830 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8-home\") pod \"test-model-0d824f-kserve-5ddd94b549-fdt6b\" (UID: \"1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8\") " pod="ai-tenant-e2e-inf-b-0d824f/test-model-0d824f-kserve-5ddd94b549-fdt6b" Jul 09 11:46:15.958236 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:15.958213 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-0d824f\"/\"kube-root-ca.crt\"" Jul 09 11:46:15.958390 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:15.958264 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-0d824f\"/\"default-dockercfg-bdt4k\"" Jul 09 11:46:15.958390 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:15.958273 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-0d824f\"/\"openshift-service-ca.crt\"" Jul 09 11:46:15.958522 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:15.958406 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-0d824f\"/\"test-model-0d824f-kserve-self-signed-certs\"" Jul 09 11:46:15.959740 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:15.959716 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-0d824f/test-model-0d824f-kserve-5ddd94b549-fdt6b"] Jul 09 11:46:16.058538 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:16.058505 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8-home\") pod \"test-model-0d824f-kserve-5ddd94b549-fdt6b\" (UID: \"1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8\") " pod="ai-tenant-e2e-inf-b-0d824f/test-model-0d824f-kserve-5ddd94b549-fdt6b" Jul 09 11:46:16.058759 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:16.058563 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8-model-cache\") pod \"test-model-0d824f-kserve-5ddd94b549-fdt6b\" (UID: \"1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8\") " pod="ai-tenant-e2e-inf-b-0d824f/test-model-0d824f-kserve-5ddd94b549-fdt6b" Jul 09 11:46:16.058759 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:16.058592 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8-tls-certs\") pod \"test-model-0d824f-kserve-5ddd94b549-fdt6b\" (UID: \"1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8\") " pod="ai-tenant-e2e-inf-b-0d824f/test-model-0d824f-kserve-5ddd94b549-fdt6b" Jul 09 11:46:16.058759 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:16.058609 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmzj5\" (UniqueName: \"kubernetes.io/projected/1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8-kube-api-access-xmzj5\") pod \"test-model-0d824f-kserve-5ddd94b549-fdt6b\" (UID: \"1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8\") " pod="ai-tenant-e2e-inf-b-0d824f/test-model-0d824f-kserve-5ddd94b549-fdt6b" Jul 09 11:46:16.058759 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:16.058668 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8-dshm\") pod \"test-model-0d824f-kserve-5ddd94b549-fdt6b\" (UID: \"1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8\") " pod="ai-tenant-e2e-inf-b-0d824f/test-model-0d824f-kserve-5ddd94b549-fdt6b" Jul 09 11:46:16.058759 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:16.058691 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8-tmp-dir\") pod \"test-model-0d824f-kserve-5ddd94b549-fdt6b\" (UID: \"1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8\") " pod="ai-tenant-e2e-inf-b-0d824f/test-model-0d824f-kserve-5ddd94b549-fdt6b" Jul 09 11:46:16.059034 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:16.058950 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8-home\") pod \"test-model-0d824f-kserve-5ddd94b549-fdt6b\" (UID: \"1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8\") " pod="ai-tenant-e2e-inf-b-0d824f/test-model-0d824f-kserve-5ddd94b549-fdt6b" Jul 09 11:46:16.059034 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:16.058986 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8-model-cache\") pod \"test-model-0d824f-kserve-5ddd94b549-fdt6b\" (UID: \"1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8\") " pod="ai-tenant-e2e-inf-b-0d824f/test-model-0d824f-kserve-5ddd94b549-fdt6b" Jul 09 11:46:16.059106 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:16.059086 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8-tmp-dir\") pod \"test-model-0d824f-kserve-5ddd94b549-fdt6b\" (UID: \"1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8\") " pod="ai-tenant-e2e-inf-b-0d824f/test-model-0d824f-kserve-5ddd94b549-fdt6b" Jul 09 11:46:16.060867 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:16.060842 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8-dshm\") pod \"test-model-0d824f-kserve-5ddd94b549-fdt6b\" (UID: \"1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8\") " pod="ai-tenant-e2e-inf-b-0d824f/test-model-0d824f-kserve-5ddd94b549-fdt6b" Jul 09 11:46:16.061128 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:16.061113 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8-tls-certs\") pod \"test-model-0d824f-kserve-5ddd94b549-fdt6b\" (UID: \"1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8\") " pod="ai-tenant-e2e-inf-b-0d824f/test-model-0d824f-kserve-5ddd94b549-fdt6b" Jul 09 11:46:16.067700 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:16.067677 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmzj5\" (UniqueName: \"kubernetes.io/projected/1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8-kube-api-access-xmzj5\") pod \"test-model-0d824f-kserve-5ddd94b549-fdt6b\" (UID: \"1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8\") " pod="ai-tenant-e2e-inf-b-0d824f/test-model-0d824f-kserve-5ddd94b549-fdt6b" Jul 09 11:46:16.266528 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:16.266430 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0d824f/test-model-0d824f-kserve-5ddd94b549-fdt6b" Jul 09 11:46:16.403710 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:16.403625 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-0d824f/test-model-0d824f-kserve-5ddd94b549-fdt6b"] Jul 09 11:46:16.406021 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:46:16.405994 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1efc278e_fdf4_4266_aafe_d2b9f6c2f7f8.slice/crio-7738c1723a610d6ffbcb45ac5b5701d05413a4e9c916b9a5d1922b27ab8853a1 WatchSource:0}: Error finding container 7738c1723a610d6ffbcb45ac5b5701d05413a4e9c916b9a5d1922b27ab8853a1: Status 404 returned error can't find the container with id 7738c1723a610d6ffbcb45ac5b5701d05413a4e9c916b9a5d1922b27ab8853a1 Jul 09 11:46:17.191494 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:17.191458 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0d824f/test-model-0d824f-kserve-5ddd94b549-fdt6b" event={"ID":"1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8","Type":"ContainerStarted","Data":"f84838a1b9f1bf67bdf5f52be19ff81107fbe31b1453d7855da1dbad9f0efd95"} Jul 09 11:46:17.191494 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:17.191502 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0d824f/test-model-0d824f-kserve-5ddd94b549-fdt6b" event={"ID":"1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8","Type":"ContainerStarted","Data":"7738c1723a610d6ffbcb45ac5b5701d05413a4e9c916b9a5d1922b27ab8853a1"} Jul 09 11:46:17.209132 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:17.209082 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-0d824f/test-model-0d824f-kserve-5ddd94b549-fdt6b" podStartSLOduration=2.209065853 podStartE2EDuration="2.209065853s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 11:46:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:46:17.208275865 +0000 UTC m=+2082.410595004" watchObservedRunningTime="2026-07-09 11:46:17.209065853 +0000 UTC m=+2082.411385002" Jul 09 11:46:26.266983 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:26.266938 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-0d824f/test-model-0d824f-kserve-5ddd94b549-fdt6b" Jul 09 11:46:26.267516 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:26.267019 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-0d824f/test-model-0d824f-kserve-5ddd94b549-fdt6b" Jul 09 11:46:26.279628 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:26.279604 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-0d824f/test-model-0d824f-kserve-5ddd94b549-fdt6b" Jul 09 11:46:27.251132 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:27.251106 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-0d824f/test-model-0d824f-kserve-5ddd94b549-fdt6b" Jul 09 11:46:30.203697 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:30.203671 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726625-4rf6w" Jul 09 11:46:30.255695 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:30.255662 2550 generic.go:356] "Generic (PLEG): container finished" podID="cd88de32-82e5-4b18-8296-2e58997d2519" containerID="9188b397c6f6e8493a8bb3d7173da700de8886b3e7741ab63d9fce3f53b4e8c2" exitCode=137 Jul 09 11:46:30.255883 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:30.255714 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726625-4rf6w" Jul 09 11:46:30.255883 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:30.255743 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726625-4rf6w" event={"ID":"cd88de32-82e5-4b18-8296-2e58997d2519","Type":"ContainerDied","Data":"9188b397c6f6e8493a8bb3d7173da700de8886b3e7741ab63d9fce3f53b4e8c2"} Jul 09 11:46:30.255883 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:30.255796 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726625-4rf6w" event={"ID":"cd88de32-82e5-4b18-8296-2e58997d2519","Type":"ContainerDied","Data":"7b122177a817dc16789f5204e41ea44a3a33ccdcae723413718ed390a56c42a1"} Jul 09 11:46:30.255883 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:30.255818 2550 scope.go:117] "RemoveContainer" containerID="9188b397c6f6e8493a8bb3d7173da700de8886b3e7741ab63d9fce3f53b4e8c2" Jul 09 11:46:30.264044 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:30.264024 2550 scope.go:117] "RemoveContainer" containerID="9188b397c6f6e8493a8bb3d7173da700de8886b3e7741ab63d9fce3f53b4e8c2" Jul 09 11:46:30.264317 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:46:30.264297 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9188b397c6f6e8493a8bb3d7173da700de8886b3e7741ab63d9fce3f53b4e8c2\": container with ID starting with 9188b397c6f6e8493a8bb3d7173da700de8886b3e7741ab63d9fce3f53b4e8c2 not found: ID does not exist" containerID="9188b397c6f6e8493a8bb3d7173da700de8886b3e7741ab63d9fce3f53b4e8c2" Jul 09 11:46:30.264413 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:30.264359 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9188b397c6f6e8493a8bb3d7173da700de8886b3e7741ab63d9fce3f53b4e8c2"} err="failed to get container status \"9188b397c6f6e8493a8bb3d7173da700de8886b3e7741ab63d9fce3f53b4e8c2\": rpc error: code = NotFound desc = could not find container \"9188b397c6f6e8493a8bb3d7173da700de8886b3e7741ab63d9fce3f53b4e8c2\": container with ID starting with 9188b397c6f6e8493a8bb3d7173da700de8886b3e7741ab63d9fce3f53b4e8c2 not found: ID does not exist" Jul 09 11:46:30.268733 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:30.268691 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ktnp7\" (UniqueName: \"kubernetes.io/projected/cd88de32-82e5-4b18-8296-2e58997d2519-kube-api-access-ktnp7\") pod \"cd88de32-82e5-4b18-8296-2e58997d2519\" (UID: \"cd88de32-82e5-4b18-8296-2e58997d2519\") " Jul 09 11:46:30.270674 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:30.270649 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd88de32-82e5-4b18-8296-2e58997d2519-kube-api-access-ktnp7" (OuterVolumeSpecName: "kube-api-access-ktnp7") pod "cd88de32-82e5-4b18-8296-2e58997d2519" (UID: "cd88de32-82e5-4b18-8296-2e58997d2519"). InnerVolumeSpecName "kube-api-access-ktnp7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 11:46:30.369639 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:30.369587 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ktnp7\" (UniqueName: \"kubernetes.io/projected/cd88de32-82e5-4b18-8296-2e58997d2519-kube-api-access-ktnp7\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:46:30.579695 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:30.579659 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726625-4rf6w"] Jul 09 11:46:30.584179 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:30.584152 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726625-4rf6w"] Jul 09 11:46:31.300552 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:31.300517 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd88de32-82e5-4b18-8296-2e58997d2519" path="/var/lib/kubelet/pods/cd88de32-82e5-4b18-8296-2e58997d2519/volumes" Jul 09 11:46:35.330711 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:35.330683 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-98ghf_fccccfad-9a68-4bd9-a44f-af22e4339085/console-operator/2.log" Jul 09 11:46:35.336429 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:35.336404 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-98ghf_fccccfad-9a68-4bd9-a44f-af22e4339085/console-operator/2.log" Jul 09 11:46:48.857917 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:48.857879 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-944007/test-model-944007-kserve-64786dbfdf-z8nfq"] Jul 09 11:46:49.844866 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:46:49.844834 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-0d824f/test-model-0d824f-kserve-5ddd94b549-fdt6b"] Jul 09 11:47:03.877980 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:03.877931 2550 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-944007/test-model-944007-kserve-64786dbfdf-z8nfq" podUID="4e7429a5-fd33-41d5-a04c-187585c014f2" containerName="main" containerID="cri-o://dc795eb9cd296988fb545ad8ea127abe686602b37014c6a755bd4b1ccbf541e8" gracePeriod=45 Jul 09 11:47:04.119156 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:04.119131 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-944007/test-model-944007-kserve-64786dbfdf-z8nfq" Jul 09 11:47:04.200164 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:04.200078 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4hj5z\" (UniqueName: \"kubernetes.io/projected/4e7429a5-fd33-41d5-a04c-187585c014f2-kube-api-access-4hj5z\") pod \"4e7429a5-fd33-41d5-a04c-187585c014f2\" (UID: \"4e7429a5-fd33-41d5-a04c-187585c014f2\") " Jul 09 11:47:04.200164 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:04.200127 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e7429a5-fd33-41d5-a04c-187585c014f2-tmp-dir\") pod \"4e7429a5-fd33-41d5-a04c-187585c014f2\" (UID: \"4e7429a5-fd33-41d5-a04c-187585c014f2\") " Jul 09 11:47:04.200164 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:04.200162 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e7429a5-fd33-41d5-a04c-187585c014f2-tls-certs\") pod \"4e7429a5-fd33-41d5-a04c-187585c014f2\" (UID: \"4e7429a5-fd33-41d5-a04c-187585c014f2\") " Jul 09 11:47:04.200429 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:04.200200 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e7429a5-fd33-41d5-a04c-187585c014f2-model-cache\") pod \"4e7429a5-fd33-41d5-a04c-187585c014f2\" (UID: \"4e7429a5-fd33-41d5-a04c-187585c014f2\") " Jul 09 11:47:04.200429 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:04.200277 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e7429a5-fd33-41d5-a04c-187585c014f2-home\") pod \"4e7429a5-fd33-41d5-a04c-187585c014f2\" (UID: \"4e7429a5-fd33-41d5-a04c-187585c014f2\") " Jul 09 11:47:04.200429 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:04.200300 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e7429a5-fd33-41d5-a04c-187585c014f2-dshm\") pod \"4e7429a5-fd33-41d5-a04c-187585c014f2\" (UID: \"4e7429a5-fd33-41d5-a04c-187585c014f2\") " Jul 09 11:47:04.200590 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:04.200484 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e7429a5-fd33-41d5-a04c-187585c014f2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4e7429a5-fd33-41d5-a04c-187585c014f2" (UID: "4e7429a5-fd33-41d5-a04c-187585c014f2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 11:47:04.200590 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:04.200517 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e7429a5-fd33-41d5-a04c-187585c014f2-model-cache" (OuterVolumeSpecName: "model-cache") pod "4e7429a5-fd33-41d5-a04c-187585c014f2" (UID: "4e7429a5-fd33-41d5-a04c-187585c014f2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 11:47:04.200590 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:04.200536 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e7429a5-fd33-41d5-a04c-187585c014f2-home" (OuterVolumeSpecName: "home") pod "4e7429a5-fd33-41d5-a04c-187585c014f2" (UID: "4e7429a5-fd33-41d5-a04c-187585c014f2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 11:47:04.200734 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:04.200688 2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e7429a5-fd33-41d5-a04c-187585c014f2-tmp-dir\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:47:04.200734 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:04.200704 2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e7429a5-fd33-41d5-a04c-187585c014f2-model-cache\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:47:04.200734 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:04.200721 2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e7429a5-fd33-41d5-a04c-187585c014f2-home\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:47:04.202468 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:04.202436 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e7429a5-fd33-41d5-a04c-187585c014f2-dshm" (OuterVolumeSpecName: "dshm") pod "4e7429a5-fd33-41d5-a04c-187585c014f2" (UID: "4e7429a5-fd33-41d5-a04c-187585c014f2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 11:47:04.202468 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:04.202451 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e7429a5-fd33-41d5-a04c-187585c014f2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4e7429a5-fd33-41d5-a04c-187585c014f2" (UID: "4e7429a5-fd33-41d5-a04c-187585c014f2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 11:47:04.202829 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:04.202525 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e7429a5-fd33-41d5-a04c-187585c014f2-kube-api-access-4hj5z" (OuterVolumeSpecName: "kube-api-access-4hj5z") pod "4e7429a5-fd33-41d5-a04c-187585c014f2" (UID: "4e7429a5-fd33-41d5-a04c-187585c014f2"). InnerVolumeSpecName "kube-api-access-4hj5z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 11:47:04.301514 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:04.301481 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4hj5z\" (UniqueName: \"kubernetes.io/projected/4e7429a5-fd33-41d5-a04c-187585c014f2-kube-api-access-4hj5z\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:47:04.301514 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:04.301507 2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e7429a5-fd33-41d5-a04c-187585c014f2-tls-certs\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:47:04.301514 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:04.301517 2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e7429a5-fd33-41d5-a04c-187585c014f2-dshm\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:47:04.419617 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:04.419585 2550 generic.go:356] "Generic (PLEG): container finished" podID="4e7429a5-fd33-41d5-a04c-187585c014f2" containerID="dc795eb9cd296988fb545ad8ea127abe686602b37014c6a755bd4b1ccbf541e8" exitCode=0 Jul 09 11:47:04.419781 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:04.419631 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-944007/test-model-944007-kserve-64786dbfdf-z8nfq" Jul 09 11:47:04.419781 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:04.419659 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-944007/test-model-944007-kserve-64786dbfdf-z8nfq" event={"ID":"4e7429a5-fd33-41d5-a04c-187585c014f2","Type":"ContainerDied","Data":"dc795eb9cd296988fb545ad8ea127abe686602b37014c6a755bd4b1ccbf541e8"} Jul 09 11:47:04.419781 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:04.419691 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-944007/test-model-944007-kserve-64786dbfdf-z8nfq" event={"ID":"4e7429a5-fd33-41d5-a04c-187585c014f2","Type":"ContainerDied","Data":"fb30ace44f0cd8473ee48ab54e72b8f3387c4031eb60dc307ad4fb746b5c1211"} Jul 09 11:47:04.419781 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:04.419707 2550 scope.go:117] "RemoveContainer" containerID="dc795eb9cd296988fb545ad8ea127abe686602b37014c6a755bd4b1ccbf541e8" Jul 09 11:47:04.429083 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:04.429067 2550 scope.go:117] "RemoveContainer" containerID="dc795eb9cd296988fb545ad8ea127abe686602b37014c6a755bd4b1ccbf541e8" Jul 09 11:47:04.429442 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:47:04.429418 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc795eb9cd296988fb545ad8ea127abe686602b37014c6a755bd4b1ccbf541e8\": container with ID starting with dc795eb9cd296988fb545ad8ea127abe686602b37014c6a755bd4b1ccbf541e8 not found: ID does not exist" containerID="dc795eb9cd296988fb545ad8ea127abe686602b37014c6a755bd4b1ccbf541e8" Jul 09 11:47:04.429505 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:04.429451 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc795eb9cd296988fb545ad8ea127abe686602b37014c6a755bd4b1ccbf541e8"} err="failed to get container status \"dc795eb9cd296988fb545ad8ea127abe686602b37014c6a755bd4b1ccbf541e8\": rpc error: code = NotFound desc = could not find container \"dc795eb9cd296988fb545ad8ea127abe686602b37014c6a755bd4b1ccbf541e8\": container with ID starting with dc795eb9cd296988fb545ad8ea127abe686602b37014c6a755bd4b1ccbf541e8 not found: ID does not exist" Jul 09 11:47:04.440737 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:04.440712 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-944007/test-model-944007-kserve-64786dbfdf-z8nfq"] Jul 09 11:47:04.444903 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:04.444881 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-944007/test-model-944007-kserve-64786dbfdf-z8nfq"] Jul 09 11:47:04.864565 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:04.864503 2550 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-0d824f/test-model-0d824f-kserve-5ddd94b549-fdt6b" podUID="1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8" containerName="main" containerID="cri-o://f84838a1b9f1bf67bdf5f52be19ff81107fbe31b1453d7855da1dbad9f0efd95" gracePeriod=45 Jul 09 11:47:05.114082 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:05.114059 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0d824f/test-model-0d824f-kserve-5ddd94b549-fdt6b" Jul 09 11:47:05.209534 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:05.209457 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xmzj5\" (UniqueName: \"kubernetes.io/projected/1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8-kube-api-access-xmzj5\") pod \"1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8\" (UID: \"1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8\") " Jul 09 11:47:05.209534 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:05.209496 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8-model-cache\") pod \"1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8\" (UID: \"1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8\") " Jul 09 11:47:05.209534 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:05.209533 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8-home\") pod \"1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8\" (UID: \"1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8\") " Jul 09 11:47:05.209800 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:05.209554 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8-tls-certs\") pod \"1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8\" (UID: \"1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8\") " Jul 09 11:47:05.209800 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:05.209568 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8-tmp-dir\") pod \"1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8\" (UID: \"1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8\") " Jul 09 11:47:05.209800 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:05.209609 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8-dshm\") pod \"1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8\" (UID: \"1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8\") " Jul 09 11:47:05.209800 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:05.209786 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8-home" (OuterVolumeSpecName: "home") pod "1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8" (UID: "1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 11:47:05.210022 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:05.209895 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8" (UID: "1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 11:47:05.210022 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:05.209914 2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8-home\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:47:05.210022 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:05.209912 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8-model-cache" (OuterVolumeSpecName: "model-cache") pod "1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8" (UID: "1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 11:47:05.211766 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:05.211745 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8" (UID: "1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 11:47:05.211861 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:05.211794 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8-dshm" (OuterVolumeSpecName: "dshm") pod "1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8" (UID: "1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 11:47:05.211861 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:05.211799 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8-kube-api-access-xmzj5" (OuterVolumeSpecName: "kube-api-access-xmzj5") pod "1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8" (UID: "1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8"). InnerVolumeSpecName "kube-api-access-xmzj5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 11:47:05.299814 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:05.299781 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e7429a5-fd33-41d5-a04c-187585c014f2" path="/var/lib/kubelet/pods/4e7429a5-fd33-41d5-a04c-187585c014f2/volumes" Jul 09 11:47:05.311295 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:05.311273 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmzj5\" (UniqueName: \"kubernetes.io/projected/1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8-kube-api-access-xmzj5\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:47:05.311295 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:05.311296 2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8-model-cache\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:47:05.311456 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:05.311305 2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8-tls-certs\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:47:05.311456 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:05.311358 2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8-tmp-dir\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:47:05.311456 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:05.311370 2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8-dshm\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\"" Jul 09 11:47:05.426099 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:05.426069 2550 generic.go:356] "Generic (PLEG): container finished" podID="1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8" containerID="f84838a1b9f1bf67bdf5f52be19ff81107fbe31b1453d7855da1dbad9f0efd95" exitCode=0 Jul 09 11:47:05.426263 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:05.426112 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0d824f/test-model-0d824f-kserve-5ddd94b549-fdt6b" Jul 09 11:47:05.426263 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:05.426136 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0d824f/test-model-0d824f-kserve-5ddd94b549-fdt6b" event={"ID":"1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8","Type":"ContainerDied","Data":"f84838a1b9f1bf67bdf5f52be19ff81107fbe31b1453d7855da1dbad9f0efd95"} Jul 09 11:47:05.426263 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:05.426165 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0d824f/test-model-0d824f-kserve-5ddd94b549-fdt6b" event={"ID":"1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8","Type":"ContainerDied","Data":"7738c1723a610d6ffbcb45ac5b5701d05413a4e9c916b9a5d1922b27ab8853a1"} Jul 09 11:47:05.426263 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:05.426183 2550 scope.go:117] "RemoveContainer" containerID="f84838a1b9f1bf67bdf5f52be19ff81107fbe31b1453d7855da1dbad9f0efd95" Jul 09 11:47:05.434648 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:05.434630 2550 scope.go:117] "RemoveContainer" containerID="f84838a1b9f1bf67bdf5f52be19ff81107fbe31b1453d7855da1dbad9f0efd95" Jul 09 11:47:05.434923 ip-10-0-143-142 kubenswrapper[2550]: E0709 11:47:05.434901 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f84838a1b9f1bf67bdf5f52be19ff81107fbe31b1453d7855da1dbad9f0efd95\": container with ID starting with f84838a1b9f1bf67bdf5f52be19ff81107fbe31b1453d7855da1dbad9f0efd95 not found: ID does not exist" containerID="f84838a1b9f1bf67bdf5f52be19ff81107fbe31b1453d7855da1dbad9f0efd95" Jul 09 11:47:05.434969 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:05.434933 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f84838a1b9f1bf67bdf5f52be19ff81107fbe31b1453d7855da1dbad9f0efd95"} err="failed to get container status \"f84838a1b9f1bf67bdf5f52be19ff81107fbe31b1453d7855da1dbad9f0efd95\": rpc error: code = NotFound desc = could not find container \"f84838a1b9f1bf67bdf5f52be19ff81107fbe31b1453d7855da1dbad9f0efd95\": container with ID starting with f84838a1b9f1bf67bdf5f52be19ff81107fbe31b1453d7855da1dbad9f0efd95 not found: ID does not exist" Jul 09 11:47:05.442621 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:05.442598 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-0d824f/test-model-0d824f-kserve-5ddd94b549-fdt6b"] Jul 09 11:47:05.446314 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:05.446288 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-0d824f/test-model-0d824f-kserve-5ddd94b549-fdt6b"] Jul 09 11:47:07.299764 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:47:07.299723 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8" path="/var/lib/kubelet/pods/1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8/volumes" Jul 09 11:51:35.357470 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:51:35.357442 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-98ghf_fccccfad-9a68-4bd9-a44f-af22e4339085/console-operator/2.log" Jul 09 11:51:35.363519 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:51:35.363495 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-98ghf_fccccfad-9a68-4bd9-a44f-af22e4339085/console-operator/2.log" Jul 09 11:52:00.434169 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:00.434100 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65b4d55d4c-k9d8r_5950a9fd-7f13-4ff3-b4eb-770d730c94f8/manager/0.log" Jul 09 11:52:00.442064 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:00.442043 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65b4d55d4c-k9d8r_5950a9fd-7f13-4ff3-b4eb-770d730c94f8/copy-manifests/0.log" Jul 09 11:52:02.196933 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:02.196905 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-jjr77_d5aae076-2bd6-40a4-b379-c3e7f1365d27/manager/0.log" Jul 09 11:52:02.315359 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:02.315307 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-xzgnw_9cb6ed43-eb23-467c-82ef-18e489d53960/kuadrant-console-plugin/0.log" Jul 09 11:52:03.435440 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:03.435408 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6579588b8b-5wdsw_b394d164-b21a-4e26-a9ae-4a46e60ffa05/kube-auth-proxy/0.log" Jul 09 11:52:03.804169 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:03.804144 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-gsdht_14411769-06f0-4fec-9e81-915e42460c39/payload-pre-processing/0.log" Jul 09 11:52:03.922213 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:03.922184 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-g6whk_e0243f69-6970-4783-98c2-590c6884422a/payload-processing/0.log" Jul 09 11:52:04.520090 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:04.520052 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-bbvgz_ee68bc2a-5b49-4282-8b75-6a47ec4231c6/main/0.log" Jul 09 11:52:04.874753 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:04.874720 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-k2bm7_2125e350-8d19-4618-baa3-dbe9fbc19ba3/main/0.log" Jul 09 11:52:04.992777 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:04.992740 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-zsw5p_7f6df83b-e283-40e4-aa83-e2cf7a8c1f2f/main/0.log" Jul 09 11:52:12.816985 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:12.816936 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-sg6nt_aefb1aae-31d5-4c63-8245-3b4544e21e49/global-pull-secret-syncer/0.log" Jul 09 11:52:12.952563 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:12.952535 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-lpprd_47077616-7ca2-4b7d-ba31-92d393ab7439/konnectivity-agent/0.log" Jul 09 11:52:13.036300 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:13.036270 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-142.ec2.internal_c0a0fd4f8c4f29eccb1f0904d07ef791/haproxy/0.log" Jul 09 11:52:18.074606 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:18.074564 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-jjr77_d5aae076-2bd6-40a4-b379-c3e7f1365d27/manager/0.log" Jul 09 11:52:18.104192 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:18.104164 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-xzgnw_9cb6ed43-eb23-467c-82ef-18e489d53960/kuadrant-console-plugin/0.log" Jul 09 11:52:20.426357 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:20.426317 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-skljh_8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f/kube-state-metrics/0.log" Jul 09 11:52:20.445958 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:20.445919 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-skljh_8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f/kube-rbac-proxy-main/0.log" Jul 09 11:52:20.486817 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:20.486789 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-skljh_8c947a1f-aa08-45aa-9a5d-ae3f7fa40a4f/kube-rbac-proxy-self/0.log" Jul 09 11:52:20.666052 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:20.665953 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f62dk_0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55/node-exporter/0.log" Jul 09 11:52:20.686264 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:20.686235 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f62dk_0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55/kube-rbac-proxy/0.log" Jul 09 11:52:20.712948 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:20.712926 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f62dk_0eb1f5d9-f0d7-48d3-be2b-65ccfcccbf55/init-textfile/0.log" Jul 09 11:52:20.929666 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:20.929595 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_54f88ee4-f654-4b86-bce9-b17d7f76835b/prometheus/0.log" Jul 09 11:52:20.950464 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:20.950439 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_54f88ee4-f654-4b86-bce9-b17d7f76835b/config-reloader/0.log" Jul 09 11:52:20.972702 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:20.972677 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_54f88ee4-f654-4b86-bce9-b17d7f76835b/thanos-sidecar/0.log" Jul 09 11:52:21.002847 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:21.002822 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_54f88ee4-f654-4b86-bce9-b17d7f76835b/kube-rbac-proxy-web/0.log" Jul 09 11:52:21.024662 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:21.024638 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_54f88ee4-f654-4b86-bce9-b17d7f76835b/kube-rbac-proxy/0.log" Jul 09 11:52:21.047601 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:21.047578 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_54f88ee4-f654-4b86-bce9-b17d7f76835b/kube-rbac-proxy-thanos/0.log" Jul 09 11:52:21.069182 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:21.069162 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_54f88ee4-f654-4b86-bce9-b17d7f76835b/init-config-reloader/0.log" Jul 09 11:52:23.108107 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:23.108076 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8tktg/perf-node-gather-daemonset-5jqzg"] Jul 09 11:52:23.108559 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:23.108451 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8" containerName="main" Jul 09 11:52:23.108559 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:23.108463 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8" containerName="main" Jul 09 11:52:23.108559 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:23.108480 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e7429a5-fd33-41d5-a04c-187585c014f2" containerName="main" Jul 09 11:52:23.108559 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:23.108485 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e7429a5-fd33-41d5-a04c-187585c014f2" containerName="main" Jul 09 11:52:23.108559 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:23.108492 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd88de32-82e5-4b18-8296-2e58997d2519" containerName="cleanup" Jul 09 11:52:23.108559 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:23.108499 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd88de32-82e5-4b18-8296-2e58997d2519" containerName="cleanup" Jul 09 11:52:23.108559 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:23.108563 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e7429a5-fd33-41d5-a04c-187585c014f2" containerName="main" Jul 09 11:52:23.108942 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:23.108579 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd88de32-82e5-4b18-8296-2e58997d2519" containerName="cleanup" Jul 09 11:52:23.108942 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:23.108589 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="1efc278e-fdf4-4266-aafe-d2b9f6c2f7f8" containerName="main" Jul 09 11:52:23.111747 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:23.111729 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8tktg/perf-node-gather-daemonset-5jqzg" Jul 09 11:52:23.114268 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:23.114248 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8tktg\"/\"openshift-service-ca.crt\"" Jul 09 11:52:23.115075 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:23.115041 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8tktg\"/\"kube-root-ca.crt\"" Jul 09 11:52:23.115075 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:23.115052 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8tktg\"/\"default-dockercfg-rl825\"" Jul 09 11:52:23.122675 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:23.122654 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8tktg/perf-node-gather-daemonset-5jqzg"] Jul 09 11:52:23.258062 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:23.258025 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/918473f8-36f7-4cce-8d8d-b9c1059c1f93-podres\") pod \"perf-node-gather-daemonset-5jqzg\" (UID: \"918473f8-36f7-4cce-8d8d-b9c1059c1f93\") " pod="openshift-must-gather-8tktg/perf-node-gather-daemonset-5jqzg" Jul 09 11:52:23.258062 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:23.258062 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/918473f8-36f7-4cce-8d8d-b9c1059c1f93-proc\") pod \"perf-node-gather-daemonset-5jqzg\" (UID: \"918473f8-36f7-4cce-8d8d-b9c1059c1f93\") " pod="openshift-must-gather-8tktg/perf-node-gather-daemonset-5jqzg" Jul 09 11:52:23.258285 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:23.258161 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/918473f8-36f7-4cce-8d8d-b9c1059c1f93-sys\") pod \"perf-node-gather-daemonset-5jqzg\" (UID: \"918473f8-36f7-4cce-8d8d-b9c1059c1f93\") " pod="openshift-must-gather-8tktg/perf-node-gather-daemonset-5jqzg" Jul 09 11:52:23.258285 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:23.258201 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/918473f8-36f7-4cce-8d8d-b9c1059c1f93-lib-modules\") pod \"perf-node-gather-daemonset-5jqzg\" (UID: \"918473f8-36f7-4cce-8d8d-b9c1059c1f93\") " pod="openshift-must-gather-8tktg/perf-node-gather-daemonset-5jqzg" Jul 09 11:52:23.258411 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:23.258300 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qvrz\" (UniqueName: \"kubernetes.io/projected/918473f8-36f7-4cce-8d8d-b9c1059c1f93-kube-api-access-6qvrz\") pod \"perf-node-gather-daemonset-5jqzg\" (UID: \"918473f8-36f7-4cce-8d8d-b9c1059c1f93\") " pod="openshift-must-gather-8tktg/perf-node-gather-daemonset-5jqzg" Jul 09 11:52:23.304246 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:23.304216 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-98ghf_fccccfad-9a68-4bd9-a44f-af22e4339085/console-operator/2.log" Jul 09 11:52:23.308283 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:23.308263 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-98ghf_fccccfad-9a68-4bd9-a44f-af22e4339085/console-operator/3.log" Jul 09 11:52:23.359454 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:23.359382 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/918473f8-36f7-4cce-8d8d-b9c1059c1f93-sys\") pod \"perf-node-gather-daemonset-5jqzg\" (UID: \"918473f8-36f7-4cce-8d8d-b9c1059c1f93\") " pod="openshift-must-gather-8tktg/perf-node-gather-daemonset-5jqzg" Jul 09 11:52:23.359454 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:23.359423 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/918473f8-36f7-4cce-8d8d-b9c1059c1f93-lib-modules\") pod \"perf-node-gather-daemonset-5jqzg\" (UID: \"918473f8-36f7-4cce-8d8d-b9c1059c1f93\") " pod="openshift-must-gather-8tktg/perf-node-gather-daemonset-5jqzg" Jul 09 11:52:23.359621 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:23.359484 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qvrz\" (UniqueName: \"kubernetes.io/projected/918473f8-36f7-4cce-8d8d-b9c1059c1f93-kube-api-access-6qvrz\") pod \"perf-node-gather-daemonset-5jqzg\" (UID: \"918473f8-36f7-4cce-8d8d-b9c1059c1f93\") " pod="openshift-must-gather-8tktg/perf-node-gather-daemonset-5jqzg" Jul 09 11:52:23.359621 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:23.359501 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/918473f8-36f7-4cce-8d8d-b9c1059c1f93-sys\") pod \"perf-node-gather-daemonset-5jqzg\" (UID: \"918473f8-36f7-4cce-8d8d-b9c1059c1f93\") " pod="openshift-must-gather-8tktg/perf-node-gather-daemonset-5jqzg" Jul 09 11:52:23.359621 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:23.359515 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/918473f8-36f7-4cce-8d8d-b9c1059c1f93-podres\") pod \"perf-node-gather-daemonset-5jqzg\" (UID: \"918473f8-36f7-4cce-8d8d-b9c1059c1f93\") " pod="openshift-must-gather-8tktg/perf-node-gather-daemonset-5jqzg" Jul 09 11:52:23.359621 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:23.359545 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/918473f8-36f7-4cce-8d8d-b9c1059c1f93-proc\") pod \"perf-node-gather-daemonset-5jqzg\" (UID: \"918473f8-36f7-4cce-8d8d-b9c1059c1f93\") " pod="openshift-must-gather-8tktg/perf-node-gather-daemonset-5jqzg" Jul 09 11:52:23.359774 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:23.359619 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/918473f8-36f7-4cce-8d8d-b9c1059c1f93-lib-modules\") pod \"perf-node-gather-daemonset-5jqzg\" (UID: \"918473f8-36f7-4cce-8d8d-b9c1059c1f93\") " pod="openshift-must-gather-8tktg/perf-node-gather-daemonset-5jqzg" Jul 09 11:52:23.359774 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:23.359664 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/918473f8-36f7-4cce-8d8d-b9c1059c1f93-proc\") pod \"perf-node-gather-daemonset-5jqzg\" (UID: \"918473f8-36f7-4cce-8d8d-b9c1059c1f93\") " pod="openshift-must-gather-8tktg/perf-node-gather-daemonset-5jqzg" Jul 09 11:52:23.359774 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:23.359676 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/918473f8-36f7-4cce-8d8d-b9c1059c1f93-podres\") pod \"perf-node-gather-daemonset-5jqzg\" (UID: \"918473f8-36f7-4cce-8d8d-b9c1059c1f93\") " pod="openshift-must-gather-8tktg/perf-node-gather-daemonset-5jqzg" Jul 09 11:52:23.367427 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:23.367404 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qvrz\" (UniqueName: \"kubernetes.io/projected/918473f8-36f7-4cce-8d8d-b9c1059c1f93-kube-api-access-6qvrz\") pod \"perf-node-gather-daemonset-5jqzg\" (UID: \"918473f8-36f7-4cce-8d8d-b9c1059c1f93\") " pod="openshift-must-gather-8tktg/perf-node-gather-daemonset-5jqzg" Jul 09 11:52:23.422108 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:23.422083 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8tktg/perf-node-gather-daemonset-5jqzg" Jul 09 11:52:23.561395 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:23.561353 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8tktg/perf-node-gather-daemonset-5jqzg"] Jul 09 11:52:23.564249 ip-10-0-143-142 kubenswrapper[2550]: W0709 11:52:23.564222 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod918473f8_36f7_4cce_8d8d_b9c1059c1f93.slice/crio-6477665f760820c0b9feb00756403dbaf596fe80537d90ed2f273e48977f274e WatchSource:0}: Error finding container 6477665f760820c0b9feb00756403dbaf596fe80537d90ed2f273e48977f274e: Status 404 returned error can't find the container with id 6477665f760820c0b9feb00756403dbaf596fe80537d90ed2f273e48977f274e Jul 09 11:52:23.852361 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:23.852310 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8tktg/perf-node-gather-daemonset-5jqzg" event={"ID":"918473f8-36f7-4cce-8d8d-b9c1059c1f93","Type":"ContainerStarted","Data":"97ea0c00bb79f87fb814adca7ada5d39e96cb5828938e42e015cc58a6b1dc051"} Jul 09 11:52:23.852532 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:23.852385 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8tktg/perf-node-gather-daemonset-5jqzg" event={"ID":"918473f8-36f7-4cce-8d8d-b9c1059c1f93","Type":"ContainerStarted","Data":"6477665f760820c0b9feb00756403dbaf596fe80537d90ed2f273e48977f274e"} Jul 09 11:52:23.852532 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:23.852412 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8tktg/perf-node-gather-daemonset-5jqzg" Jul 09 11:52:23.869351 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:23.869220 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8tktg/perf-node-gather-daemonset-5jqzg" podStartSLOduration=0.869204322 podStartE2EDuration="869.204322ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 11:52:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 11:52:23.868116387 +0000 UTC m=+2449.070435536" watchObservedRunningTime="2026-07-09 11:52:23.869204322 +0000 UTC m=+2449.071523470" Jul 09 11:52:24.633825 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:24.633794 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-7wm65_97cba2fd-d06f-4c64-82e1-9e9d65be3908/volume-data-source-validator/0.log" Jul 09 11:52:25.758744 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:25.758716 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vsfwj_127cd1f0-bf8a-4bae-94fa-d21833536f5f/dns/0.log" Jul 09 11:52:25.785010 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:25.784984 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vsfwj_127cd1f0-bf8a-4bae-94fa-d21833536f5f/kube-rbac-proxy/0.log" Jul 09 11:52:25.921455 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:25.921429 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wsfmn_8117b6cb-dfab-462d-ae86-50a342e88a9a/dns-node-resolver/0.log" Jul 09 11:52:26.449213 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:26.449184 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7448f9db94-86hhs_02acda78-f3fd-42ff-8e98-5f9131421fbc/registry/0.log" Jul 09 11:52:26.469308 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:26.469282 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ppz4r_8207614d-56bb-42cd-a5e6-d7fd5da8dac1/node-ca/0.log" Jul 09 11:52:27.606051 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:27.606021 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6579588b8b-5wdsw_b394d164-b21a-4e26-a9ae-4a46e60ffa05/kube-auth-proxy/0.log" Jul 09 11:52:27.749749 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:27.749718 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-gsdht_14411769-06f0-4fec-9e81-915e42460c39/payload-pre-processing/0.log" Jul 09 11:52:27.793213 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:27.793184 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-g6whk_e0243f69-6970-4783-98c2-590c6884422a/payload-processing/0.log" Jul 09 11:52:28.569790 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:28.569751 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-96bj9_330acdbe-92a6-4ad6-b11a-e941fc7217fc/serve-healthcheck-canary/0.log" Jul 09 11:52:29.272794 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:29.272766 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tmb84_110148eb-6117-4ca1-b49c-911089247193/kube-rbac-proxy/0.log" Jul 09 11:52:29.300671 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:29.300622 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tmb84_110148eb-6117-4ca1-b49c-911089247193/exporter/0.log" Jul 09 11:52:29.321364 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:29.321321 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tmb84_110148eb-6117-4ca1-b49c-911089247193/extractor/0.log" Jul 09 11:52:29.867611 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:29.867586 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8tktg/perf-node-gather-daemonset-5jqzg" Jul 09 11:52:32.127631 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:32.127597 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65b4d55d4c-k9d8r_5950a9fd-7f13-4ff3-b4eb-770d730c94f8/manager/0.log" Jul 09 11:52:32.156516 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:32.156487 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65b4d55d4c-k9d8r_5950a9fd-7f13-4ff3-b4eb-770d730c94f8/copy-manifests/0.log" Jul 09 11:52:33.727487 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:33.727462 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-cbdc75774-jwm2l_3567c8ce-832f-4fa0-a14b-ac2aa4276bfe/manager/0.log" Jul 09 11:52:33.776881 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:33.776853 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-z9nv7_5c4a144a-965a-44e6-9172-323dc3e7f580/openshift-lws-operator/0.log" Jul 09 11:52:34.153855 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:34.153822 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6579588b8b-5wdsw_b394d164-b21a-4e26-a9ae-4a46e60ffa05/kube-auth-proxy/0.log" Jul 09 11:52:34.263231 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:34.263202 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-gsdht_14411769-06f0-4fec-9e81-915e42460c39/payload-pre-processing/0.log" Jul 09 11:52:34.306087 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:34.306065 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-g6whk_e0243f69-6970-4783-98c2-590c6884422a/payload-processing/0.log" Jul 09 11:52:35.657695 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:35.657669 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6579588b8b-5wdsw_b394d164-b21a-4e26-a9ae-4a46e60ffa05/kube-auth-proxy/0.log" Jul 09 11:52:35.768129 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:35.768095 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-gsdht_14411769-06f0-4fec-9e81-915e42460c39/payload-pre-processing/0.log" Jul 09 11:52:35.801420 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:35.801395 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-g6whk_e0243f69-6970-4783-98c2-590c6884422a/payload-processing/0.log" Jul 09 11:52:39.360982 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:39.360953 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-l7t8g_bbe87e2e-7cb1-4822-a5f1-245774196a39/migrator/0.log" Jul 09 11:52:39.388939 ip-10-0-143-142 kubenswrapper[2550]: I0709 11:52:39.388912 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-l7t8g_bbe87e2e-7cb1-4822-a5f1-245774196a39/graceful-termination/0.log"