Jul 08 18:37:38.165960 ip-10-0-143-154 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 08 18:37:38.165971 ip-10-0-143-154 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 08 18:37:38.165978 ip-10-0-143-154 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 08 18:37:38.166194 ip-10-0-143-154 systemd[1]: Failed to start Kubernetes Kubelet. Jul 08 18:37:48.122463 ip-10-0-143-154 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found. Jul 08 18:37:48.122476 ip-10-0-143-154 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 3b494c6b3fa0476aa772762f33cc2758 -- Jul 08 18:39:16.642095 ip-10-0-143-154 systemd[1]: Starting Kubernetes Kubelet... Jul 08 18:39:17.110286 ip-10-0-143-154 kubenswrapper[2559]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 18:39:17.110286 ip-10-0-143-154 kubenswrapper[2559]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 08 18:39:17.110286 ip-10-0-143-154 kubenswrapper[2559]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 18:39:17.110286 ip-10-0-143-154 kubenswrapper[2559]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 08 18:39:17.110286 ip-10-0-143-154 kubenswrapper[2559]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 18:39:17.110286 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.051034 2559 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 08 18:39:17.110286 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058202 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 18:39:17.110286 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058218 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 18:39:17.110286 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058222 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 18:39:17.110286 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058226 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 18:39:17.110286 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058229 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 18:39:17.110286 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058232 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 18:39:17.110286 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058237 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 18:39:17.114346 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058241 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 18:39:17.114346 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058244 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 18:39:17.114346 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058248 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 18:39:17.114346 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058251 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 18:39:17.114346 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058253 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 18:39:17.114346 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058256 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 08 18:39:17.114346 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058259 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 18:39:17.114346 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058262 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 18:39:17.114346 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058264 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 18:39:17.114346 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058268 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 18:39:17.114346 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058273 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 18:39:17.114346 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058276 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 18:39:17.114346 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058279 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 18:39:17.114346 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058282 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 18:39:17.114346 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058285 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 18:39:17.114346 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058288 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 18:39:17.114346 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058291 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 18:39:17.114346 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058294 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 18:39:17.114346 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058296 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 18:39:17.114346 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058299 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 18:39:17.115246 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058302 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 18:39:17.115246 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058307 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 18:39:17.115246 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058310 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 18:39:17.115246 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058313 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 18:39:17.115246 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058316 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 18:39:17.115246 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058319 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 18:39:17.115246 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058322 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 18:39:17.115246 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058325 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 18:39:17.115246 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058327 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 18:39:17.115246 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058330 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 18:39:17.115246 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058333 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 18:39:17.115246 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058335 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 18:39:17.115246 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058338 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 18:39:17.115246 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058340 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 18:39:17.115246 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058343 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 18:39:17.115246 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058345 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 18:39:17.115246 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058348 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 18:39:17.115246 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058351 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 18:39:17.115246 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058353 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 18:39:17.115246 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058355 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 18:39:17.117361 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058358 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 18:39:17.117361 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058361 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 18:39:17.117361 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058363 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 18:39:17.117361 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058368 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 18:39:17.117361 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058370 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 18:39:17.117361 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058373 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 18:39:17.117361 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058375 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 18:39:17.117361 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058378 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 18:39:17.117361 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058380 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 18:39:17.117361 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058382 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 18:39:17.117361 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058385 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 18:39:17.117361 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058387 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 18:39:17.117361 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058390 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 18:39:17.117361 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058392 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 18:39:17.117361 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058395 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 18:39:17.117361 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058397 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 18:39:17.117361 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058400 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 18:39:17.117361 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058402 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 18:39:17.117361 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058405 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 18:39:17.117361 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058408 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 18:39:17.118206 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058411 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 18:39:17.118206 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058413 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 18:39:17.118206 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058417 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 18:39:17.118206 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058420 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 18:39:17.118206 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058423 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 18:39:17.118206 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058425 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 18:39:17.118206 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058428 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 18:39:17.118206 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058430 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 18:39:17.118206 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058433 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 18:39:17.118206 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058436 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 18:39:17.118206 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058439 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 18:39:17.118206 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058442 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 18:39:17.118206 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058444 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 18:39:17.118206 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058447 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 18:39:17.118206 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058449 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 18:39:17.118206 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058452 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 18:39:17.118206 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058455 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 18:39:17.118206 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058457 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 18:39:17.118206 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058460 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 18:39:17.120711 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058463 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 18:39:17.120711 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058465 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 18:39:17.120711 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058467 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 18:39:17.120711 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058470 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 18:39:17.120711 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058473 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 18:39:17.120711 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058476 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 18:39:17.120711 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058479 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 18:39:17.120711 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058481 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 18:39:17.120711 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058483 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 18:39:17.120711 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058486 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 18:39:17.120711 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058489 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 18:39:17.120711 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058492 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 18:39:17.120711 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.058495 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 18:39:17.120711 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060827 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 18:39:17.120711 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060838 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 18:39:17.120711 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060842 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 18:39:17.120711 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060845 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 18:39:17.120711 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060849 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 18:39:17.120711 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060852 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 18:39:17.120711 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060854 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 18:39:17.121528 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060857 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 18:39:17.121528 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060860 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 18:39:17.121528 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060862 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 18:39:17.121528 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060866 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 18:39:17.121528 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060868 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 18:39:17.121528 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060871 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 18:39:17.121528 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060874 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 18:39:17.121528 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060876 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 18:39:17.121528 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060879 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 18:39:17.121528 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060881 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 18:39:17.121528 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060885 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 18:39:17.121528 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060887 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 18:39:17.121528 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060890 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 18:39:17.121528 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060892 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 18:39:17.121528 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060895 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 18:39:17.121528 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060897 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 18:39:17.121528 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060900 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 18:39:17.121528 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060903 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 18:39:17.121528 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060905 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 18:39:17.125060 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060908 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 18:39:17.125060 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060910 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 18:39:17.125060 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060913 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 18:39:17.125060 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060916 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 18:39:17.125060 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060918 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 18:39:17.125060 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060921 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 18:39:17.125060 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060923 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 18:39:17.125060 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060926 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 18:39:17.125060 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060928 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 18:39:17.125060 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060931 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 18:39:17.125060 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060935 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 18:39:17.125060 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060939 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 18:39:17.125060 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060942 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 18:39:17.125060 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060945 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 18:39:17.125060 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060949 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 18:39:17.125060 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060952 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 18:39:17.125060 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060956 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 18:39:17.125060 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060959 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 18:39:17.125060 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060962 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 18:39:17.125867 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060965 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 18:39:17.125867 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060968 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 18:39:17.125867 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060970 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 18:39:17.125867 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060973 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 18:39:17.125867 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060975 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 18:39:17.125867 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060979 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 18:39:17.125867 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060981 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 18:39:17.125867 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060984 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 08 18:39:17.125867 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060986 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 18:39:17.125867 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060989 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 18:39:17.125867 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060991 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 18:39:17.125867 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060994 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 18:39:17.125867 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060996 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 18:39:17.125867 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.060998 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 18:39:17.125867 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061001 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 18:39:17.125867 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061003 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 18:39:17.125867 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061007 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 18:39:17.125867 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061010 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 18:39:17.125867 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061012 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 18:39:17.125867 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061014 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 18:39:17.129313 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061017 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 18:39:17.129313 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061020 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 18:39:17.129313 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061023 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 18:39:17.129313 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061025 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 18:39:17.129313 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061028 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 18:39:17.129313 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061031 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 18:39:17.129313 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061033 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 18:39:17.129313 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061036 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 18:39:17.129313 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061038 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 18:39:17.129313 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061042 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 18:39:17.129313 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061045 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 18:39:17.129313 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061048 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 18:39:17.129313 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061050 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 18:39:17.129313 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061053 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 18:39:17.129313 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061056 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 18:39:17.129313 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061058 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 18:39:17.129313 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061061 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 18:39:17.129313 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061064 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 18:39:17.129313 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061068 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 18:39:17.129313 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061071 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 18:39:17.129313 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061073 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 18:39:17.130224 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061077 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 18:39:17.130224 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061080 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 18:39:17.130224 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061082 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 18:39:17.130224 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061085 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 18:39:17.130224 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061087 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 18:39:17.130224 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061090 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 18:39:17.130224 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061092 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 18:39:17.130224 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061095 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 18:39:17.130224 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061098 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 18:39:17.130224 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061101 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 18:39:17.130224 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061103 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 18:39:17.130224 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061106 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 18:39:17.130224 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061109 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 18:39:17.130224 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061508 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 18:39:17.130224 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061513 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 18:39:17.130224 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061516 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 18:39:17.130224 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061519 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 18:39:17.130224 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061521 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 18:39:17.130224 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061524 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 18:39:17.135477 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061526 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 18:39:17.135477 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061529 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 18:39:17.135477 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061532 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 18:39:17.135477 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061534 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 18:39:17.135477 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061537 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 18:39:17.135477 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061539 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 18:39:17.135477 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061542 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 18:39:17.135477 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061545 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 18:39:17.135477 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061547 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 18:39:17.135477 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061549 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 18:39:17.135477 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061552 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 18:39:17.135477 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061555 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 18:39:17.135477 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061558 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 18:39:17.135477 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061561 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 18:39:17.135477 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061563 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 18:39:17.135477 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061566 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 18:39:17.135477 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061568 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 18:39:17.135477 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061572 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 18:39:17.135477 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061576 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 18:39:17.136551 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061580 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 18:39:17.136551 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061583 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 18:39:17.136551 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061586 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 18:39:17.136551 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061589 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 18:39:17.136551 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061592 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 18:39:17.136551 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061595 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 18:39:17.136551 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061599 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 18:39:17.136551 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061602 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 18:39:17.136551 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061605 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 18:39:17.136551 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061607 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 18:39:17.136551 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061610 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 18:39:17.136551 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061613 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 18:39:17.136551 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061615 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 18:39:17.136551 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061617 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 18:39:17.136551 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061620 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 18:39:17.136551 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061622 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 18:39:17.136551 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061625 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 18:39:17.136551 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061627 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 18:39:17.136551 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061630 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 18:39:17.136551 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061632 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 18:39:17.141870 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061634 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 08 18:39:17.141870 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061637 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 18:39:17.141870 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061640 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 18:39:17.141870 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061643 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 18:39:17.141870 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061646 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 18:39:17.141870 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061649 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 18:39:17.141870 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061652 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 18:39:17.141870 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061654 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 18:39:17.141870 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061656 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 18:39:17.141870 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061663 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 18:39:17.141870 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061665 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 18:39:17.141870 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061668 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 18:39:17.141870 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061670 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 18:39:17.141870 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061673 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 18:39:17.141870 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061676 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 18:39:17.141870 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061679 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 18:39:17.141870 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061681 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 18:39:17.141870 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061684 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 18:39:17.141870 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061687 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 18:39:17.140507 ip-10-0-143-154 systemd[1]: Started Kubernetes Kubelet. Jul 08 18:39:17.143590 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061689 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 18:39:17.143590 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061692 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 18:39:17.143590 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061694 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 18:39:17.143590 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061697 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 18:39:17.143590 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061699 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 18:39:17.143590 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061702 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 18:39:17.143590 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061704 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 18:39:17.143590 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061707 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 18:39:17.143590 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061709 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 18:39:17.143590 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061712 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 18:39:17.143590 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061714 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 18:39:17.143590 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061717 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 18:39:17.143590 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061719 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 18:39:17.143590 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061722 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 18:39:17.143590 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061724 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 18:39:17.143590 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061727 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 18:39:17.143590 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061730 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 18:39:17.143590 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061733 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 18:39:17.143590 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061736 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 18:39:17.143590 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061739 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 18:39:17.147652 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061741 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 18:39:17.147652 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061744 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 18:39:17.147652 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061747 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 18:39:17.147652 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061750 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 18:39:17.147652 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061752 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 18:39:17.147652 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061755 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 18:39:17.147652 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061758 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 18:39:17.147652 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061760 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 18:39:17.147652 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061762 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 18:39:17.147652 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061782 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 18:39:17.147652 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061786 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 18:39:17.147652 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061790 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 18:39:17.147652 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061792 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 18:39:17.147652 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061795 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 18:39:17.147652 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.061797 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 18:39:17.147652 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.061869 2559 flags.go:64] FLAG: --address="0.0.0.0" Jul 08 18:39:17.147652 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.061875 2559 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 08 18:39:17.147652 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.061881 2559 flags.go:64] FLAG: --anonymous-auth="true" Jul 08 18:39:17.147652 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.061885 2559 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 08 18:39:17.147652 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.061890 2559 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 08 18:39:17.147652 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.061893 2559 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 08 18:39:17.148458 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.061897 2559 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 08 18:39:17.148458 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.061901 2559 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 08 18:39:17.148458 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.061904 2559 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 08 18:39:17.148458 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.061907 2559 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 08 18:39:17.148458 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.061911 2559 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 08 18:39:17.148458 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.061914 2559 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 08 18:39:17.148458 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.061918 2559 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 08 18:39:17.148458 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.061920 2559 flags.go:64] FLAG: --cgroup-root="" Jul 08 18:39:17.148458 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.061923 2559 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 08 18:39:17.148458 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.061926 2559 flags.go:64] FLAG: --client-ca-file="" Jul 08 18:39:17.148458 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.061930 2559 flags.go:64] FLAG: --cloud-provider="external" Jul 08 18:39:17.148458 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.061933 2559 flags.go:64] FLAG: --cluster-dns="[]" Jul 08 18:39:17.148458 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.061938 2559 flags.go:64] FLAG: --cluster-domain="" Jul 08 18:39:17.148458 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.061943 2559 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 08 18:39:17.148458 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.061946 2559 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 08 18:39:17.148458 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.061949 2559 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 08 18:39:17.148458 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.061952 2559 flags.go:64] FLAG: --container-log-max-files="5" Jul 08 18:39:17.148458 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.061957 2559 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 08 18:39:17.148458 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.061959 2559 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 08 18:39:17.148458 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.061962 2559 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 08 18:39:17.148458 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.061966 2559 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 08 18:39:17.148458 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.061969 2559 flags.go:64] FLAG: --contention-profiling="false" Jul 08 18:39:17.148458 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.061972 2559 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 08 18:39:17.148458 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.061975 2559 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 08 18:39:17.152683 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.061978 2559 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 08 18:39:17.152683 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.061980 2559 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 08 18:39:17.152683 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.061984 2559 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 08 18:39:17.152683 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.061988 2559 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 08 18:39:17.152683 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.061990 2559 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 08 18:39:17.152683 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.061993 2559 flags.go:64] FLAG: --enable-load-reader="false" Jul 08 18:39:17.152683 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.061996 2559 flags.go:64] FLAG: --enable-server="true" Jul 08 18:39:17.152683 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.061999 2559 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 08 18:39:17.152683 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062003 2559 flags.go:64] FLAG: --event-burst="100" Jul 08 18:39:17.152683 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062006 2559 flags.go:64] FLAG: --event-qps="50" Jul 08 18:39:17.152683 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062009 2559 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 08 18:39:17.152683 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062012 2559 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 08 18:39:17.152683 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062015 2559 flags.go:64] FLAG: --eviction-hard="" Jul 08 18:39:17.152683 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062019 2559 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 08 18:39:17.152683 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062022 2559 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 08 18:39:17.152683 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062025 2559 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 08 18:39:17.152683 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062028 2559 flags.go:64] FLAG: --eviction-soft="" Jul 08 18:39:17.152683 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062031 2559 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 08 18:39:17.152683 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062033 2559 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 08 18:39:17.152683 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062037 2559 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 08 18:39:17.152683 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062040 2559 flags.go:64] FLAG: --experimental-mounter-path="" Jul 08 18:39:17.152683 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062044 2559 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 08 18:39:17.152683 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062047 2559 flags.go:64] FLAG: --fail-swap-on="true" Jul 08 18:39:17.152683 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062050 2559 flags.go:64] FLAG: --feature-gates="" Jul 08 18:39:17.152683 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062054 2559 flags.go:64] FLAG: --file-check-frequency="20s" Jul 08 18:39:17.153895 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062057 2559 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 08 18:39:17.153895 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062060 2559 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 08 18:39:17.153895 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062063 2559 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 08 18:39:17.153895 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062066 2559 flags.go:64] FLAG: --healthz-port="10248" Jul 08 18:39:17.153895 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062069 2559 flags.go:64] FLAG: --help="false" Jul 08 18:39:17.153895 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062072 2559 flags.go:64] FLAG: --hostname-override="ip-10-0-143-154.ec2.internal" Jul 08 18:39:17.153895 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062075 2559 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 08 18:39:17.153895 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062078 2559 flags.go:64] FLAG: --http-check-frequency="20s" Jul 08 18:39:17.153895 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062081 2559 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 08 18:39:17.153895 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062084 2559 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 08 18:39:17.153895 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062088 2559 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 08 18:39:17.153895 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062091 2559 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 08 18:39:17.153895 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062093 2559 flags.go:64] FLAG: --image-service-endpoint="" Jul 08 18:39:17.153895 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062097 2559 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 08 18:39:17.153895 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062099 2559 flags.go:64] FLAG: --kube-api-burst="100" Jul 08 18:39:17.153895 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062102 2559 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 08 18:39:17.153895 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062105 2559 flags.go:64] FLAG: --kube-api-qps="50" Jul 08 18:39:17.153895 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062108 2559 flags.go:64] FLAG: --kube-reserved="" Jul 08 18:39:17.153895 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062111 2559 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 08 18:39:17.153895 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062114 2559 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 08 18:39:17.153895 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062117 2559 flags.go:64] FLAG: --kubelet-cgroups="" Jul 08 18:39:17.153895 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062120 2559 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 08 18:39:17.153895 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062123 2559 flags.go:64] FLAG: --lock-file="" Jul 08 18:39:17.153895 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062125 2559 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 08 18:39:17.258180 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062128 2559 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 08 18:39:17.258180 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062131 2559 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 08 18:39:17.258180 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062137 2559 flags.go:64] FLAG: --log-json-split-stream="false" Jul 08 18:39:17.258180 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062140 2559 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 08 18:39:17.258180 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062144 2559 flags.go:64] FLAG: --log-text-split-stream="false" Jul 08 18:39:17.258180 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062147 2559 flags.go:64] FLAG: --logging-format="text" Jul 08 18:39:17.258180 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062149 2559 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 08 18:39:17.258180 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062153 2559 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 08 18:39:17.258180 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062155 2559 flags.go:64] FLAG: --manifest-url="" Jul 08 18:39:17.258180 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062159 2559 flags.go:64] FLAG: --manifest-url-header="" Jul 08 18:39:17.258180 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062163 2559 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 08 18:39:17.258180 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062166 2559 flags.go:64] FLAG: --max-open-files="1000000" Jul 08 18:39:17.258180 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062170 2559 flags.go:64] FLAG: --max-pods="110" Jul 08 18:39:17.258180 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062173 2559 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 08 18:39:17.258180 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062176 2559 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 08 18:39:17.258180 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062179 2559 flags.go:64] FLAG: --memory-manager-policy="None" Jul 08 18:39:17.258180 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062182 2559 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 08 18:39:17.258180 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062184 2559 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 08 18:39:17.258180 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062187 2559 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 08 18:39:17.258180 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062190 2559 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 08 18:39:17.258180 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062198 2559 flags.go:64] FLAG: --node-status-max-images="50" Jul 08 18:39:17.258180 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062201 2559 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 08 18:39:17.258180 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062203 2559 flags.go:64] FLAG: --oom-score-adj="-999" Jul 08 18:39:17.258180 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062206 2559 flags.go:64] FLAG: --pod-cidr="" Jul 08 18:39:17.259410 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062209 2559 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 08 18:39:17.259410 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062214 2559 flags.go:64] FLAG: --pod-manifest-path="" Jul 08 18:39:17.259410 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062217 2559 flags.go:64] FLAG: --pod-max-pids="-1" Jul 08 18:39:17.259410 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062220 2559 flags.go:64] FLAG: --pods-per-core="0" Jul 08 18:39:17.259410 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062223 2559 flags.go:64] FLAG: --port="10250" Jul 08 18:39:17.259410 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062226 2559 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 08 18:39:17.259410 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062229 2559 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-03601bd9e0c9f4367" Jul 08 18:39:17.259410 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062232 2559 flags.go:64] FLAG: --qos-reserved="" Jul 08 18:39:17.259410 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062235 2559 flags.go:64] FLAG: --read-only-port="10255" Jul 08 18:39:17.259410 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062238 2559 flags.go:64] FLAG: --register-node="true" Jul 08 18:39:17.259410 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062241 2559 flags.go:64] FLAG: --register-with-taints="" Jul 08 18:39:17.259410 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062245 2559 flags.go:64] FLAG: --registry-burst="10" Jul 08 18:39:17.259410 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062249 2559 flags.go:64] FLAG: --registry-qps="5" Jul 08 18:39:17.259410 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062252 2559 flags.go:64] FLAG: --reserved-cpus="" Jul 08 18:39:17.259410 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062255 2559 flags.go:64] FLAG: --reserved-memory="" Jul 08 18:39:17.259410 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062259 2559 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 08 18:39:17.259410 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062262 2559 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 08 18:39:17.259410 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062265 2559 flags.go:64] FLAG: --rotate-certificates="false" Jul 08 18:39:17.259410 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062268 2559 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 08 18:39:17.259410 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062270 2559 flags.go:64] FLAG: --runonce="false" Jul 08 18:39:17.259410 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062273 2559 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 08 18:39:17.259410 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062276 2559 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 08 18:39:17.259410 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062279 2559 flags.go:64] FLAG: --seccomp-default="false" Jul 08 18:39:17.259410 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062282 2559 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 08 18:39:17.259410 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062285 2559 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 08 18:39:17.263611 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062288 2559 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 08 18:39:17.263611 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062291 2559 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 08 18:39:17.263611 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062294 2559 flags.go:64] FLAG: --storage-driver-password="root" Jul 08 18:39:17.263611 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062297 2559 flags.go:64] FLAG: --storage-driver-secure="false" Jul 08 18:39:17.263611 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062300 2559 flags.go:64] FLAG: --storage-driver-table="stats" Jul 08 18:39:17.263611 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062303 2559 flags.go:64] FLAG: --storage-driver-user="root" Jul 08 18:39:17.263611 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062305 2559 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 08 18:39:17.263611 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062308 2559 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 08 18:39:17.263611 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062311 2559 flags.go:64] FLAG: --system-cgroups="" Jul 08 18:39:17.263611 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062314 2559 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 08 18:39:17.263611 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062321 2559 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 08 18:39:17.263611 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062323 2559 flags.go:64] FLAG: --tls-cert-file="" Jul 08 18:39:17.263611 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062326 2559 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 08 18:39:17.263611 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062331 2559 flags.go:64] FLAG: --tls-min-version="" Jul 08 18:39:17.263611 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062333 2559 flags.go:64] FLAG: --tls-private-key-file="" Jul 08 18:39:17.263611 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062336 2559 flags.go:64] FLAG: --topology-manager-policy="none" Jul 08 18:39:17.263611 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062339 2559 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 08 18:39:17.263611 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062342 2559 flags.go:64] FLAG: --topology-manager-scope="container" Jul 08 18:39:17.263611 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062345 2559 flags.go:64] FLAG: --v="2" Jul 08 18:39:17.263611 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062350 2559 flags.go:64] FLAG: --version="false" Jul 08 18:39:17.263611 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062355 2559 flags.go:64] FLAG: --vmodule="" Jul 08 18:39:17.263611 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062359 2559 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 08 18:39:17.263611 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062362 2559 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 08 18:39:17.263611 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062496 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 18:39:17.263611 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062500 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 18:39:17.264420 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062503 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 18:39:17.264420 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062506 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 18:39:17.264420 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062509 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 18:39:17.264420 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062512 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 18:39:17.264420 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062514 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 18:39:17.264420 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062517 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 18:39:17.264420 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062519 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 18:39:17.264420 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062522 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 18:39:17.264420 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062524 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 18:39:17.264420 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062527 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 18:39:17.264420 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062529 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 18:39:17.264420 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062532 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 18:39:17.264420 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062534 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 18:39:17.264420 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062537 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 18:39:17.264420 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062540 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 18:39:17.264420 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062542 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 18:39:17.264420 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062545 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 18:39:17.264420 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062548 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 18:39:17.264420 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062551 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 18:39:17.264420 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062554 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 18:39:17.267335 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062556 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 18:39:17.267335 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062559 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 18:39:17.267335 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062563 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 18:39:17.267335 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062567 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 18:39:17.267335 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062572 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 18:39:17.267335 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062575 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 18:39:17.267335 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062579 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 18:39:17.267335 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062582 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 18:39:17.267335 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062589 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 18:39:17.267335 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062592 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 18:39:17.267335 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062595 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 18:39:17.267335 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062598 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 18:39:17.267335 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062601 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 18:39:17.267335 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062604 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 18:39:17.267335 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062607 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 18:39:17.267335 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062610 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 18:39:17.267335 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062612 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 18:39:17.267335 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062615 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 18:39:17.267335 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062618 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 18:39:17.268155 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062620 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 18:39:17.268155 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062623 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 18:39:17.268155 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062625 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 18:39:17.268155 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062628 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 18:39:17.268155 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062630 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 18:39:17.268155 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062633 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 18:39:17.268155 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062635 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 18:39:17.268155 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062638 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 18:39:17.268155 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062640 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 18:39:17.268155 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062643 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 18:39:17.268155 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062646 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 18:39:17.268155 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062649 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 18:39:17.268155 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062652 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 18:39:17.268155 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062654 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 18:39:17.268155 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062657 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 18:39:17.268155 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062659 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 18:39:17.268155 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062662 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 18:39:17.268155 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062664 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 18:39:17.268155 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062667 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 18:39:17.268155 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062671 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 18:39:17.271090 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062673 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 18:39:17.271090 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062677 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 18:39:17.271090 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062682 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 18:39:17.271090 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062685 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 18:39:17.271090 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062688 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 18:39:17.271090 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062692 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 18:39:17.271090 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062694 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 18:39:17.271090 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062697 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 18:39:17.271090 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062699 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 18:39:17.271090 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062702 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 18:39:17.271090 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062704 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 18:39:17.271090 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062707 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 18:39:17.271090 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062709 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 18:39:17.271090 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062712 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 18:39:17.271090 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062714 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 18:39:17.271090 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062717 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 18:39:17.271090 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062720 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 18:39:17.271090 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062722 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 18:39:17.271090 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062724 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 18:39:17.271090 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062727 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 18:39:17.271913 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062730 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 18:39:17.271913 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062732 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 18:39:17.271913 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062736 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 18:39:17.271913 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062739 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 18:39:17.271913 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062741 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 18:39:17.271913 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062744 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 18:39:17.271913 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062747 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 18:39:17.271913 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062750 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 18:39:17.271913 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062752 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 18:39:17.271913 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062755 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 08 18:39:17.271913 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062757 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 18:39:17.271913 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062761 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 18:39:17.271913 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062764 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 18:39:17.271913 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062787 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 18:39:17.271913 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062789 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 18:39:17.271913 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062801 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 18:39:17.271913 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062804 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 18:39:17.271913 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.062807 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 18:39:17.276003 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.062812 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 18:39:17.276003 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.070286 2559 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 08 18:39:17.276003 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.070301 2559 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 08 18:39:17.276003 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.070316 2559 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 08 18:39:17.276003 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.070321 2559 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 08 18:39:17.276003 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070376 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 18:39:17.276003 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070380 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 18:39:17.276003 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070383 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 18:39:17.276003 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070387 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 18:39:17.276003 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070390 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 18:39:17.276003 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070393 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 18:39:17.276003 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070396 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 18:39:17.276003 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070398 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 18:39:17.276003 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070401 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 18:39:17.276430 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070404 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 18:39:17.276430 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070407 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 18:39:17.276430 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070409 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 18:39:17.276430 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070412 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 18:39:17.276430 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070415 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 18:39:17.276430 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070417 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 18:39:17.276430 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070420 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 18:39:17.276430 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070422 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 18:39:17.276430 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070425 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 18:39:17.276430 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070427 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 18:39:17.276430 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070430 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 18:39:17.276430 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070434 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 18:39:17.276430 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070437 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 18:39:17.276430 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070440 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 18:39:17.276430 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070443 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 18:39:17.276430 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070446 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 18:39:17.276430 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070449 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 18:39:17.276430 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070451 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 18:39:17.276430 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070454 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 18:39:17.276430 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070456 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 18:39:17.281316 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070459 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 18:39:17.281316 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070461 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 18:39:17.281316 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070464 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 18:39:17.281316 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070467 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 18:39:17.281316 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070470 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 18:39:17.281316 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070473 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 18:39:17.281316 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070476 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 18:39:17.281316 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070479 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 18:39:17.281316 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070482 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 18:39:17.281316 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070484 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 18:39:17.281316 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070487 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 18:39:17.281316 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070490 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 18:39:17.281316 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070493 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 18:39:17.281316 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070495 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 18:39:17.281316 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070498 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 18:39:17.281316 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070501 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 18:39:17.281316 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070503 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 18:39:17.281316 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070506 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 18:39:17.281316 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070508 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 18:39:17.281316 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070511 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 18:39:17.282149 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070513 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 18:39:17.282149 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070516 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 18:39:17.282149 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070518 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 18:39:17.282149 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070521 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 18:39:17.282149 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070524 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 18:39:17.282149 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070527 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 18:39:17.282149 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070531 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 18:39:17.282149 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070533 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 18:39:17.282149 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070536 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 18:39:17.282149 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070539 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 18:39:17.282149 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070541 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 18:39:17.282149 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070544 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 18:39:17.282149 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070546 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 18:39:17.282149 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070549 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 18:39:17.282149 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070551 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 18:39:17.282149 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070554 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 18:39:17.282149 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070556 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 18:39:17.282149 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070559 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 18:39:17.282149 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070561 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 18:39:17.282149 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070564 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 18:39:17.283002 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070567 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 18:39:17.283002 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070569 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 18:39:17.283002 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070572 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 18:39:17.283002 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070575 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 18:39:17.283002 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070578 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 18:39:17.283002 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070580 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 18:39:17.283002 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070584 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 18:39:17.283002 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070587 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 18:39:17.283002 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070589 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 18:39:17.283002 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070591 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 18:39:17.283002 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070595 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 18:39:17.283002 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070599 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 18:39:17.283002 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070602 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 18:39:17.283002 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070605 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 18:39:17.283002 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070608 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 18:39:17.283002 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070611 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 18:39:17.283002 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070615 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 18:39:17.283002 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070621 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 18:39:17.283002 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070625 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 18:39:17.287519 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070628 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 18:39:17.287519 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070631 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 18:39:17.287519 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070634 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 18:39:17.287519 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070637 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 18:39:17.287519 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070640 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 18:39:17.287519 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070643 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 18:39:17.287519 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070646 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 18:39:17.287519 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070648 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 08 18:39:17.287519 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070651 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 18:39:17.287519 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070654 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 18:39:17.287519 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070656 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 18:39:17.287519 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.070661 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 18:39:17.287519 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070788 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 18:39:17.287519 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070793 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 18:39:17.287519 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070796 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 18:39:17.288367 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070799 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 18:39:17.288367 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070802 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 18:39:17.288367 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070805 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 18:39:17.288367 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070807 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 18:39:17.288367 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070810 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 18:39:17.288367 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070813 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 18:39:17.288367 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070815 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 18:39:17.288367 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070818 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 18:39:17.288367 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070820 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 18:39:17.288367 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070823 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 18:39:17.288367 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070825 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 18:39:17.288367 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070829 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 18:39:17.288367 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070832 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 18:39:17.288367 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070835 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 18:39:17.288367 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070837 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 18:39:17.288367 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070841 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 18:39:17.288367 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070844 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 18:39:17.288367 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070847 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 18:39:17.288367 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070850 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 18:39:17.288367 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070852 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 18:39:17.289190 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070856 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 18:39:17.289190 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070859 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 18:39:17.289190 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070861 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 18:39:17.289190 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070864 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 18:39:17.289190 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070866 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 18:39:17.289190 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070869 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 18:39:17.289190 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070871 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 18:39:17.289190 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070874 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 18:39:17.289190 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070876 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 18:39:17.289190 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070879 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 18:39:17.289190 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070881 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 18:39:17.289190 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070884 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 18:39:17.289190 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070886 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 18:39:17.289190 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070890 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 18:39:17.289190 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070893 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 18:39:17.289190 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070896 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 18:39:17.289190 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070899 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 18:39:17.289190 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070902 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 18:39:17.289190 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070905 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 18:39:17.289190 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070907 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 18:39:17.391552 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070910 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 18:39:17.391552 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070912 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 18:39:17.391552 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070915 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 18:39:17.391552 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070919 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 18:39:17.391552 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070921 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 18:39:17.391552 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070924 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 18:39:17.391552 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070927 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 18:39:17.391552 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070931 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 18:39:17.391552 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070934 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 18:39:17.391552 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070936 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 18:39:17.391552 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070939 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 18:39:17.391552 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070941 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 18:39:17.391552 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070943 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 18:39:17.391552 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070946 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 18:39:17.391552 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070949 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 18:39:17.391552 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070951 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 18:39:17.391552 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070954 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 18:39:17.391552 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070956 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 08 18:39:17.391552 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070959 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 18:39:17.392352 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070961 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 18:39:17.392352 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070964 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 18:39:17.392352 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070966 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 18:39:17.392352 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070969 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 18:39:17.392352 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070971 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 18:39:17.392352 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070974 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 18:39:17.392352 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070976 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 18:39:17.392352 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070979 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 18:39:17.392352 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070981 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 18:39:17.392352 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070984 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 18:39:17.392352 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070986 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 18:39:17.392352 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070988 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 18:39:17.392352 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070992 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 18:39:17.392352 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070995 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 18:39:17.392352 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.070998 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 18:39:17.392352 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.071001 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 18:39:17.392352 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.071005 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 18:39:17.392352 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.071007 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 18:39:17.392352 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.071017 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 18:39:17.395668 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.071020 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 18:39:17.395668 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.071026 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 18:39:17.395668 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.071028 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 18:39:17.395668 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.071031 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 18:39:17.395668 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.071035 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 18:39:17.395668 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.071039 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 18:39:17.395668 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.071041 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 18:39:17.395668 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.071044 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 18:39:17.395668 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.071047 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 18:39:17.395668 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.071050 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 18:39:17.395668 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.071052 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 18:39:17.395668 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.071055 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 18:39:17.395668 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.071057 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 18:39:17.395668 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.071059 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 18:39:17.395668 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.071062 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 18:39:17.395668 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.071065 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 18:39:17.395668 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.071067 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 18:39:17.395668 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:17.071070 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 18:39:17.396510 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.071075 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 18:39:17.396510 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.071182 2559 server.go:962] "Client rotation is on, will bootstrap in background" Jul 08 18:39:17.396510 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.073083 2559 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 08 18:39:17.396510 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.074274 2559 server.go:1019] "Starting client certificate rotation" Jul 08 18:39:17.396510 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.074375 2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 18:39:17.396510 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.074416 2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 18:39:17.396510 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.077229 2559 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 18:39:17.396510 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.082156 2559 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 18:39:17.396510 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.095123 2559 log.go:25] "Validated CRI v1 runtime API" Jul 08 18:39:17.396510 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.098718 2559 log.go:25] "Validated CRI v1 image API" Jul 08 18:39:17.396510 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.099493 2559 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 08 18:39:17.396510 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.105548 2559 fs.go:135] Filesystem UUIDs: map[208e44d2-5857-46a8-ae67-8755f44d7aba:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 c9d4ecfa-a57d-41fa-ba58-3883f339ac78:/dev/nvme0n1p4] Jul 08 18:39:17.399053 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.105563 2559 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 08 18:39:17.399053 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.109170 2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 18:39:17.508568 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.114789 2559 manager.go:216] Machine: {Timestamp:2026-07-08 18:39:17.112448427 +0000 UTC m=+0.348746508 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3136188 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e260e13e77250f244c5cd377e0ee0 SystemUUID:ec2e260e-13e7-7250-f244-c5cd377e0ee0 BootID:3b494c6b-3fa0-476a-a772-762f33cc2758 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:dd:a7:24:09:0d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:dd:a7:24:09:0d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d2:e0:46:ca:56:93 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 08 18:39:17.508568 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.114956 2559 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 08 18:39:17.508568 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.115108 2559 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 08 18:39:17.508568 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.119617 2559 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 08 18:39:17.508568 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.119650 2559 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-154.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 08 18:39:17.508568 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.120334 2559 topology_manager.go:138] "Creating topology manager with none policy" Jul 08 18:39:17.508568 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.120346 2559 container_manager_linux.go:309] "Creating device plugin manager" Jul 08 18:39:17.508568 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.120357 2559 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 18:39:17.508568 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.120375 2559 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 18:39:17.508568 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.121561 2559 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 08 18:39:17.508568 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.124244 2559 state_mem.go:36] "Initialized new in-memory state store" Jul 08 18:39:17.508568 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.124384 2559 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 08 18:39:17.508568 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.124414 2559 kubelet.go:486] "Attempting to sync node with API server" Jul 08 18:39:17.508568 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.124428 2559 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 08 18:39:17.508568 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.124452 2559 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 08 18:39:17.508568 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.124465 2559 kubelet.go:398] "Adding apiserver pod source" Jul 08 18:39:17.508568 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.124479 2559 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 08 18:39:17.508568 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.124916 2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 18:39:17.510253 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.124939 2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 18:39:17.510253 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.130015 2559 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 08 18:39:17.510253 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.132110 2559 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 08 18:39:17.510253 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.132148 2559 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 08 18:39:17.510253 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.133861 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 08 18:39:17.510253 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.133877 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 08 18:39:17.510253 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.133886 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 08 18:39:17.510253 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.133892 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 08 18:39:17.510253 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.133904 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 08 18:39:17.510253 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.133912 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 08 18:39:17.510253 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.133924 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 08 18:39:17.510253 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.133931 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 08 18:39:17.510253 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.133940 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 08 18:39:17.510253 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.133948 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 08 18:39:17.510253 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.133972 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 08 18:39:17.510253 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.133981 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 08 18:39:17.510253 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.135296 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 08 18:39:17.510253 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.135306 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 08 18:39:17.510253 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.139266 2559 server.go:1268] "Started kubelet" Jul 08 18:39:17.510253 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.140272 2559 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 08 18:39:17.510253 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.142866 2559 server.go:310] "Adding debug handlers to kubelet server" Jul 08 18:39:17.510253 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.143888 2559 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 08 18:39:17.510253 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.143931 2559 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 08 18:39:17.510253 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.148763 2559 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 08 18:39:17.510253 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:17.151516 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-154.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 08 18:39:17.510253 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:17.151564 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 08 18:39:17.510253 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.151672 2559 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-154.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 18:39:17.510253 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.155032 2559 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 08 18:39:17.510253 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.155057 2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 08 18:39:17.510253 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.155891 2559 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 08 18:39:17.510253 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.155903 2559 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 08 18:39:17.510253 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.155995 2559 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 08 18:39:17.510253 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.156034 2559 reconstruct.go:97] "Volume reconstruction finished" Jul 08 18:39:17.510253 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.156039 2559 reconciler.go:29] "Reconciler: start to sync state" Jul 08 18:39:17.510253 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:17.156432 2559 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 08 18:39:17.510253 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:17.156563 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-154.ec2.internal\" not found" Jul 08 18:39:17.611132 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:17.610488 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-154.ec2.internal\" not found" Jul 08 18:39:17.611132 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.611076 2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zt8hl" Jul 08 18:39:17.611132 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.611125 2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zt8hl" Jul 08 18:39:17.612953 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.612848 2559 factory.go:153] Registering CRI-O factory Jul 08 18:39:17.613079 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.612969 2559 factory.go:223] Registration of the crio container factory successfully Jul 08 18:39:17.613079 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.613023 2559 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 08 18:39:17.613079 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.613033 2559 factory.go:55] Registering systemd factory Jul 08 18:39:17.613079 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.613051 2559 factory.go:223] Registration of the systemd container factory successfully Jul 08 18:39:17.613079 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.613074 2559 factory.go:103] Registering Raw factory Jul 08 18:39:17.613305 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.613088 2559 manager.go:1196] Started watching for new ooms in manager Jul 08 18:39:17.613542 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.613526 2559 manager.go:349] Starting recovery of all containers Jul 08 18:39:17.621073 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.621048 2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 18:39:17.624487 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.624470 2559 manager.go:354] Recovery completed Jul 08 18:39:17.626227 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:17.626205 2559 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-143-154.ec2.internal\" not found" node="ip-10-0-143-154.ec2.internal" Jul 08 18:39:17.628425 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.628410 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:39:17.630338 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.630323 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-154.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:39:17.630407 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.630353 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-154.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:39:17.630407 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.630366 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-154.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:39:17.630906 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.630894 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:39:17.630969 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.630896 2559 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 08 18:39:17.630969 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.630954 2559 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 08 18:39:17.631031 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.630972 2559 state_mem.go:36] "Initialized new in-memory state store" Jul 08 18:39:17.632210 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.632198 2559 policy_none.go:49] "None policy: Start" Jul 08 18:39:17.632261 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.632221 2559 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 08 18:39:17.632261 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.632231 2559 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 08 18:39:17.632327 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.632201 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-154.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:39:17.632327 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.632284 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-154.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:39:17.632327 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.632293 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-154.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:39:17.633247 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.633234 2559 policy_none.go:47] "Start" Jul 08 18:39:17.678638 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.678456 2559 manager.go:337] "Starting Device Plugin manager" Jul 08 18:39:17.684924 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:17.678664 2559 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 08 18:39:17.684924 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.678680 2559 server.go:87] "Starting device plugin registration server" Jul 08 18:39:17.684924 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.678946 2559 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 08 18:39:17.684924 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.678959 2559 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 08 18:39:17.684924 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.679063 2559 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 08 18:39:17.684924 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.679160 2559 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 08 18:39:17.684924 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.679170 2559 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 08 18:39:17.684924 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.679444 2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 08 18:39:17.684924 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:17.679737 2559 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 08 18:39:17.684924 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:17.679811 2559 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-154.ec2.internal\" not found" Jul 08 18:39:17.684924 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.680779 2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 08 18:39:17.684924 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.680820 2559 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 08 18:39:17.684924 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.680843 2559 kubelet.go:2448] "Starting kubelet main sync loop" Jul 08 18:39:17.684924 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:17.680883 2559 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 08 18:39:17.684924 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.683372 2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 18:39:17.781583 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.781557 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:39:17.781583 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.781563 2559 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-154.ec2.internal"] Jul 08 18:39:17.781744 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.781661 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:39:17.782719 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.782699 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-154.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:39:17.782854 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.782737 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-154.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:39:17.782854 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.782752 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-154.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:39:17.782975 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.782751 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-154.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:39:17.782975 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.782897 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-154.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:39:17.782975 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.782908 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-154.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:39:17.782975 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.782934 2559 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-154.ec2.internal" Jul 08 18:39:17.784963 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.784948 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:39:17.785102 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.785085 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal" Jul 08 18:39:17.785159 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.785119 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:39:17.786124 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.786101 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-154.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:39:17.786231 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.786135 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-154.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:39:17.786231 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.786150 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-154.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:39:17.786231 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.786108 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-154.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:39:17.786231 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.786184 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-154.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:39:17.786231 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.786196 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-154.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:39:17.788369 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.788352 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-154.ec2.internal" Jul 08 18:39:17.788451 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.788385 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:39:17.789422 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.789199 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-154.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:39:17.789531 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.789447 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-154.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:39:17.789531 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.789466 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-154.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:39:17.794315 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.794292 2559 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-154.ec2.internal" Jul 08 18:39:17.794409 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:17.794317 2559 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-154.ec2.internal\": node \"ip-10-0-143-154.ec2.internal\" not found" Jul 08 18:39:17.808464 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:17.808438 2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-154.ec2.internal\" not found" node="ip-10-0-143-154.ec2.internal" Jul 08 18:39:17.812088 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:17.812070 2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-154.ec2.internal\" not found" node="ip-10-0-143-154.ec2.internal" Jul 08 18:39:17.822099 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:17.822080 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-154.ec2.internal\" not found" Jul 08 18:39:17.913196 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.913110 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b82dabbd147d65c747169cc61141ecbe-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal\" (UID: \"b82dabbd147d65c747169cc61141ecbe\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal" Jul 08 18:39:17.913196 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.913143 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b82dabbd147d65c747169cc61141ecbe-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal\" (UID: \"b82dabbd147d65c747169cc61141ecbe\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal" Jul 08 18:39:17.913196 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:17.913164 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3872d75f39f988d68af7ca611d18fcd5-config\") pod \"kube-apiserver-proxy-ip-10-0-143-154.ec2.internal\" (UID: \"3872d75f39f988d68af7ca611d18fcd5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-154.ec2.internal" Jul 08 18:39:17.922905 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:17.922883 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-154.ec2.internal\" not found" Jul 08 18:39:18.015057 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:18.015003 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3872d75f39f988d68af7ca611d18fcd5-config\") pod \"kube-apiserver-proxy-ip-10-0-143-154.ec2.internal\" (UID: \"3872d75f39f988d68af7ca611d18fcd5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-154.ec2.internal" Jul 08 18:39:18.015057 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:18.015064 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b82dabbd147d65c747169cc61141ecbe-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal\" (UID: \"b82dabbd147d65c747169cc61141ecbe\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal" Jul 08 18:39:18.015192 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:18.015082 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b82dabbd147d65c747169cc61141ecbe-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal\" (UID: \"b82dabbd147d65c747169cc61141ecbe\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal" Jul 08 18:39:18.015192 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:18.015100 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3872d75f39f988d68af7ca611d18fcd5-config\") pod \"kube-apiserver-proxy-ip-10-0-143-154.ec2.internal\" (UID: \"3872d75f39f988d68af7ca611d18fcd5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-154.ec2.internal" Jul 08 18:39:18.015192 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:18.015127 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b82dabbd147d65c747169cc61141ecbe-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal\" (UID: \"b82dabbd147d65c747169cc61141ecbe\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal" Jul 08 18:39:18.015287 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:18.015190 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b82dabbd147d65c747169cc61141ecbe-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal\" (UID: \"b82dabbd147d65c747169cc61141ecbe\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal" Jul 08 18:39:18.023846 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:18.023813 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-154.ec2.internal\" not found" Jul 08 18:39:18.074870 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:18.074831 2559 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 08 18:39:18.075041 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:18.075015 2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 18:39:18.075105 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:18.075026 2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 18:39:18.120674 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:18.113447 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal" Jul 08 18:39:18.120674 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:18.117006 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-154.ec2.internal" Jul 08 18:39:18.124546 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:18.124525 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-154.ec2.internal\" not found" Jul 08 18:39:18.144516 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:18.144494 2559 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 18:39:18.155398 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:18.155368 2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 08 18:39:18.157286 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:18.157266 2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-154.ec2.internal" Jul 08 18:39:18.170796 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:18.170727 2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 18:39:18.172908 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:18.172893 2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal" Jul 08 18:39:18.173344 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:18.173328 2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 18:39:18.197248 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:18.197216 2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-ftjcw" Jul 08 18:39:18.204402 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:18.204384 2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-ftjcw" Jul 08 18:39:18.240609 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:18.240589 2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 18:39:18.579190 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:18.579102 2559 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 18:39:18.614618 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:18.614557 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 18:34:17 +0000 UTC" deadline="2028-01-30 19:46:33.800558303 +0000 UTC" Jul 08 18:39:18.614618 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:18.614610 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13705h7m15.185952979s" Jul 08 18:39:18.865238 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:18.865202 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb82dabbd147d65c747169cc61141ecbe.slice/crio-92c9ebc8399bc9624403fa2ce2bcd917197f55ef3e6d396f437aef5c886d8968 WatchSource:0}: Error finding container 92c9ebc8399bc9624403fa2ce2bcd917197f55ef3e6d396f437aef5c886d8968: Status 404 returned error can't find the container with id 92c9ebc8399bc9624403fa2ce2bcd917197f55ef3e6d396f437aef5c886d8968 Jul 08 18:39:18.872855 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:18.872824 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 18:39:18.874493 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:18.874471 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3872d75f39f988d68af7ca611d18fcd5.slice/crio-102f7a9d7976ce4c8f8ca6d678e42b8093064aabd7b505ac1c82ec92e2c9dd3a WatchSource:0}: Error finding container 102f7a9d7976ce4c8f8ca6d678e42b8093064aabd7b505ac1c82ec92e2c9dd3a: Status 404 returned error can't find the container with id 102f7a9d7976ce4c8f8ca6d678e42b8093064aabd7b505ac1c82ec92e2c9dd3a Jul 08 18:39:18.989042 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:18.989011 2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 18:39:19.127828 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.127729 2559 apiserver.go:52] "Watching apiserver" Jul 08 18:39:19.136450 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.136425 2559 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 08 18:39:19.136841 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.136818 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-7sxcd","openshift-dns/node-resolver-x72mc","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal","openshift-multus/network-metrics-daemon-dc669","openshift-ovn-kubernetes/ovnkube-node-rj4xk","kube-system/konnectivity-agent-qklqp","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4w8l","openshift-image-registry/node-ca-q9hp4","openshift-multus/multus-additional-cni-plugins-qw6cl","openshift-multus/multus-kqlfn","openshift-network-diagnostics/network-check-target-88zv8","openshift-network-operator/iptables-alerter-fqjq8","kube-system/global-pull-secret-syncer-l9l9x","kube-system/kube-apiserver-proxy-ip-10-0-143-154.ec2.internal"] Jul 08 18:39:19.140860 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.140827 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.143058 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.143038 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-x72mc" Jul 08 18:39:19.144739 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.144718 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 08 18:39:19.144842 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.144745 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7f9jt\"" Jul 08 18:39:19.145146 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.145132 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 08 18:39:19.145366 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.145349 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dc669" Jul 08 18:39:19.145452 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:19.145429 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dc669" podUID="14cb8ce5-3cb8-4827-8034-d74d7a8d75bf" Jul 08 18:39:19.145840 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.145819 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 08 18:39:19.145840 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.145851 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-rvg4v\"" Jul 08 18:39:19.148613 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.146453 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 08 18:39:19.150106 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.150086 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.152497 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.152481 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qklqp" Jul 08 18:39:19.154909 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.154889 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 08 18:39:19.155003 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.154931 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4w8l" Jul 08 18:39:19.156491 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.156471 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 08 18:39:19.156688 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.156670 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 08 18:39:19.156784 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.156768 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 08 18:39:19.156879 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.156863 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-hxv7t\"" Jul 08 18:39:19.157197 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.157180 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vgr5m\"" Jul 08 18:39:19.157314 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.157297 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 08 18:39:19.157383 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.157332 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-q9hp4" Jul 08 18:39:19.157383 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.157350 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 08 18:39:19.157546 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.157529 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 08 18:39:19.158036 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.158019 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 08 18:39:19.158218 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.158201 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 08 18:39:19.158414 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.158399 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 08 18:39:19.158745 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.158730 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 08 18:39:19.159058 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.159041 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mp6qk\"" Jul 08 18:39:19.159901 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.159882 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qw6cl" Jul 08 18:39:19.162290 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.162269 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.165876 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.165743 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88zv8" Jul 08 18:39:19.165876 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:19.165804 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88zv8" podUID="205d4eb0-c6f3-4800-aa14-ece6eb2c362d" Jul 08 18:39:19.168254 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.168237 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fqjq8" Jul 08 18:39:19.169047 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.169031 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 08 18:39:19.169125 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.169033 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 08 18:39:19.169175 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.169130 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 08 18:39:19.169227 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.169188 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 08 18:39:19.169365 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.169351 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 08 18:39:19.169452 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.169436 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 08 18:39:19.169516 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.169494 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 08 18:39:19.169638 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.169617 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-dhdwn\"" Jul 08 18:39:19.169809 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.169740 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 08 18:39:19.169809 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.169798 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-qp58f\"" Jul 08 18:39:19.170056 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.170041 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-b7m75\"" Jul 08 18:39:19.170184 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.170167 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 08 18:39:19.170759 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.170737 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9l9x" Jul 08 18:39:19.170841 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:19.170806 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9l9x" podUID="a0618904-8d08-404a-9a82-a0bb55351381" Jul 08 18:39:19.171199 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.171180 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 08 18:39:19.172020 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.172004 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-kd6mz\"" Jul 08 18:39:19.172081 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.172047 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 08 18:39:19.172258 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.172242 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 08 18:39:19.207660 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.207622 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 18:34:18 +0000 UTC" deadline="2028-04-04 21:55:01.540659778 +0000 UTC" Jul 08 18:39:19.207660 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.207658 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15267h15m42.333004121s" Jul 08 18:39:19.231549 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.231514 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-run-systemd\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.231549 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.231548 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-host-cni-netd\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.231737 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.231564 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/56014c3a-384a-4775-96e5-290968b196b1-env-overrides\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.231737 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.231580 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/145206ea-0a5b-4a75-b0dc-81e6412873ba-host\") pod \"tuned-7sxcd\" (UID: \"145206ea-0a5b-4a75-b0dc-81e6412873ba\") " pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.231737 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.231611 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0b0ede89-a5f3-4ce8-b2e1-45451e6c443d-device-dir\") pod \"aws-ebs-csi-driver-node-p4w8l\" (UID: \"0b0ede89-a5f3-4ce8-b2e1-45451e6c443d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4w8l" Jul 08 18:39:19.231737 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.231629 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5btgw\" (UniqueName: \"kubernetes.io/projected/ceb0a0a0-5ba1-4906-b214-5b1de0abbec3-kube-api-access-5btgw\") pod \"node-ca-q9hp4\" (UID: \"ceb0a0a0-5ba1-4906-b214-5b1de0abbec3\") " pod="openshift-image-registry/node-ca-q9hp4" Jul 08 18:39:19.231737 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.231646 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-systemd-units\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.231737 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.231661 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-etc-openvswitch\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.231737 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.231677 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-host-run-ovn-kubernetes\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.231737 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.231720 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/56014c3a-384a-4775-96e5-290968b196b1-ovnkube-config\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.232051 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.231751 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/145206ea-0a5b-4a75-b0dc-81e6412873ba-etc-modprobe-d\") pod \"tuned-7sxcd\" (UID: \"145206ea-0a5b-4a75-b0dc-81e6412873ba\") " pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.232051 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.231768 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/145206ea-0a5b-4a75-b0dc-81e6412873ba-etc-sysconfig\") pod \"tuned-7sxcd\" (UID: \"145206ea-0a5b-4a75-b0dc-81e6412873ba\") " pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.232051 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.231810 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/145206ea-0a5b-4a75-b0dc-81e6412873ba-sys\") pod \"tuned-7sxcd\" (UID: \"145206ea-0a5b-4a75-b0dc-81e6412873ba\") " pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.232051 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.231826 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/145206ea-0a5b-4a75-b0dc-81e6412873ba-var-lib-kubelet\") pod \"tuned-7sxcd\" (UID: \"145206ea-0a5b-4a75-b0dc-81e6412873ba\") " pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.232051 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.231840 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/145206ea-0a5b-4a75-b0dc-81e6412873ba-etc-tuned\") pod \"tuned-7sxcd\" (UID: \"145206ea-0a5b-4a75-b0dc-81e6412873ba\") " pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.232051 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.231856 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dnt8\" (UniqueName: \"kubernetes.io/projected/145206ea-0a5b-4a75-b0dc-81e6412873ba-kube-api-access-7dnt8\") pod \"tuned-7sxcd\" (UID: \"145206ea-0a5b-4a75-b0dc-81e6412873ba\") " pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.232051 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.231881 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/32185124-ea41-4309-9674-13230b4d7a6e-hosts-file\") pod \"node-resolver-x72mc\" (UID: \"32185124-ea41-4309-9674-13230b4d7a6e\") " pod="openshift-dns/node-resolver-x72mc" Jul 08 18:39:19.232051 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.231916 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14cb8ce5-3cb8-4827-8034-d74d7a8d75bf-metrics-certs\") pod \"network-metrics-daemon-dc669\" (UID: \"14cb8ce5-3cb8-4827-8034-d74d7a8d75bf\") " pod="openshift-multus/network-metrics-daemon-dc669" Jul 08 18:39:19.232051 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.231967 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32185124-ea41-4309-9674-13230b4d7a6e-tmp-dir\") pod \"node-resolver-x72mc\" (UID: \"32185124-ea41-4309-9674-13230b4d7a6e\") " pod="openshift-dns/node-resolver-x72mc" Jul 08 18:39:19.232051 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.231992 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d155f1e7-7109-445e-b884-7a3e49df96f8-konnectivity-ca\") pod \"konnectivity-agent-qklqp\" (UID: \"d155f1e7-7109-445e-b884-7a3e49df96f8\") " pod="kube-system/konnectivity-agent-qklqp" Jul 08 18:39:19.232051 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.232008 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0b0ede89-a5f3-4ce8-b2e1-45451e6c443d-socket-dir\") pod \"aws-ebs-csi-driver-node-p4w8l\" (UID: \"0b0ede89-a5f3-4ce8-b2e1-45451e6c443d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4w8l" Jul 08 18:39:19.232051 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.232038 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0b0ede89-a5f3-4ce8-b2e1-45451e6c443d-etc-selinux\") pod \"aws-ebs-csi-driver-node-p4w8l\" (UID: \"0b0ede89-a5f3-4ce8-b2e1-45451e6c443d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4w8l" Jul 08 18:39:19.232529 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.232059 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-var-lib-openvswitch\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.232529 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.232078 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-run-ovn\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.232529 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.232097 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/145206ea-0a5b-4a75-b0dc-81e6412873ba-etc-sysctl-d\") pod \"tuned-7sxcd\" (UID: \"145206ea-0a5b-4a75-b0dc-81e6412873ba\") " pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.232529 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.232113 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/145206ea-0a5b-4a75-b0dc-81e6412873ba-etc-systemd\") pod \"tuned-7sxcd\" (UID: \"145206ea-0a5b-4a75-b0dc-81e6412873ba\") " pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.232529 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.232130 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/145206ea-0a5b-4a75-b0dc-81e6412873ba-tmp\") pod \"tuned-7sxcd\" (UID: \"145206ea-0a5b-4a75-b0dc-81e6412873ba\") " pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.232529 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.232151 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-host-kubelet\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.232529 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.232170 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-host-slash\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.232529 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.232185 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-host-cni-bin\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.232529 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.232200 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d155f1e7-7109-445e-b884-7a3e49df96f8-agent-certs\") pod \"konnectivity-agent-qklqp\" (UID: \"d155f1e7-7109-445e-b884-7a3e49df96f8\") " pod="kube-system/konnectivity-agent-qklqp" Jul 08 18:39:19.232529 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.232216 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0b0ede89-a5f3-4ce8-b2e1-45451e6c443d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p4w8l\" (UID: \"0b0ede89-a5f3-4ce8-b2e1-45451e6c443d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4w8l" Jul 08 18:39:19.232529 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.232233 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0b0ede89-a5f3-4ce8-b2e1-45451e6c443d-registration-dir\") pod \"aws-ebs-csi-driver-node-p4w8l\" (UID: \"0b0ede89-a5f3-4ce8-b2e1-45451e6c443d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4w8l" Jul 08 18:39:19.232529 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.232255 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-node-log\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.232529 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.232268 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-log-socket\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.232529 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.232284 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/56014c3a-384a-4775-96e5-290968b196b1-ovn-node-metrics-cert\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.232529 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.232300 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/56014c3a-384a-4775-96e5-290968b196b1-ovnkube-script-lib\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.232529 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.232316 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/145206ea-0a5b-4a75-b0dc-81e6412873ba-etc-kubernetes\") pod \"tuned-7sxcd\" (UID: \"145206ea-0a5b-4a75-b0dc-81e6412873ba\") " pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.233072 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.232336 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/145206ea-0a5b-4a75-b0dc-81e6412873ba-etc-sysctl-conf\") pod \"tuned-7sxcd\" (UID: \"145206ea-0a5b-4a75-b0dc-81e6412873ba\") " pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.233072 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.232350 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/145206ea-0a5b-4a75-b0dc-81e6412873ba-run\") pod \"tuned-7sxcd\" (UID: \"145206ea-0a5b-4a75-b0dc-81e6412873ba\") " pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.233072 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.232365 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6ktv\" (UniqueName: \"kubernetes.io/projected/14cb8ce5-3cb8-4827-8034-d74d7a8d75bf-kube-api-access-w6ktv\") pod \"network-metrics-daemon-dc669\" (UID: \"14cb8ce5-3cb8-4827-8034-d74d7a8d75bf\") " pod="openshift-multus/network-metrics-daemon-dc669" Jul 08 18:39:19.233072 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.232384 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sc88z\" (UniqueName: \"kubernetes.io/projected/32185124-ea41-4309-9674-13230b4d7a6e-kube-api-access-sc88z\") pod \"node-resolver-x72mc\" (UID: \"32185124-ea41-4309-9674-13230b4d7a6e\") " pod="openshift-dns/node-resolver-x72mc" Jul 08 18:39:19.233072 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.232402 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqh6b\" (UniqueName: \"kubernetes.io/projected/0b0ede89-a5f3-4ce8-b2e1-45451e6c443d-kube-api-access-bqh6b\") pod \"aws-ebs-csi-driver-node-p4w8l\" (UID: \"0b0ede89-a5f3-4ce8-b2e1-45451e6c443d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4w8l" Jul 08 18:39:19.233072 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.232439 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-host-run-netns\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.233072 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.232480 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-run-openvswitch\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.233072 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.232508 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.233072 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.232536 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbm5v\" (UniqueName: \"kubernetes.io/projected/56014c3a-384a-4775-96e5-290968b196b1-kube-api-access-hbm5v\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.233072 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.232562 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/145206ea-0a5b-4a75-b0dc-81e6412873ba-lib-modules\") pod \"tuned-7sxcd\" (UID: \"145206ea-0a5b-4a75-b0dc-81e6412873ba\") " pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.233072 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.232587 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0b0ede89-a5f3-4ce8-b2e1-45451e6c443d-sys-fs\") pod \"aws-ebs-csi-driver-node-p4w8l\" (UID: \"0b0ede89-a5f3-4ce8-b2e1-45451e6c443d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4w8l" Jul 08 18:39:19.233072 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.232610 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ceb0a0a0-5ba1-4906-b214-5b1de0abbec3-host\") pod \"node-ca-q9hp4\" (UID: \"ceb0a0a0-5ba1-4906-b214-5b1de0abbec3\") " pod="openshift-image-registry/node-ca-q9hp4" Jul 08 18:39:19.233072 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.232634 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ceb0a0a0-5ba1-4906-b214-5b1de0abbec3-serviceca\") pod \"node-ca-q9hp4\" (UID: \"ceb0a0a0-5ba1-4906-b214-5b1de0abbec3\") " pod="openshift-image-registry/node-ca-q9hp4" Jul 08 18:39:19.257380 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.257342 2559 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 08 18:39:19.333506 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.333476 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b958669d-dcaf-40b5-8b8d-36ff1ce96694-host-run-k8s-cni-cncf-io\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.333506 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.333510 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/145206ea-0a5b-4a75-b0dc-81e6412873ba-host\") pod \"tuned-7sxcd\" (UID: \"145206ea-0a5b-4a75-b0dc-81e6412873ba\") " pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.333733 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.333530 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0b0ede89-a5f3-4ce8-b2e1-45451e6c443d-device-dir\") pod \"aws-ebs-csi-driver-node-p4w8l\" (UID: \"0b0ede89-a5f3-4ce8-b2e1-45451e6c443d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4w8l" Jul 08 18:39:19.333733 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.333547 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/abf012dc-98c1-40d7-aca4-8e9b5ccccea6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qw6cl\" (UID: \"abf012dc-98c1-40d7-aca4-8e9b5ccccea6\") " pod="openshift-multus/multus-additional-cni-plugins-qw6cl" Jul 08 18:39:19.333733 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.333583 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b958669d-dcaf-40b5-8b8d-36ff1ce96694-multus-conf-dir\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.333733 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.333614 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b958669d-dcaf-40b5-8b8d-36ff1ce96694-etc-kubernetes\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.333733 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.333635 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/145206ea-0a5b-4a75-b0dc-81e6412873ba-host\") pod \"tuned-7sxcd\" (UID: \"145206ea-0a5b-4a75-b0dc-81e6412873ba\") " pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.333733 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.333679 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9zzh\" (UniqueName: \"kubernetes.io/projected/b958669d-dcaf-40b5-8b8d-36ff1ce96694-kube-api-access-k9zzh\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.333733 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.333692 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0b0ede89-a5f3-4ce8-b2e1-45451e6c443d-device-dir\") pod \"aws-ebs-csi-driver-node-p4w8l\" (UID: \"0b0ede89-a5f3-4ce8-b2e1-45451e6c443d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4w8l" Jul 08 18:39:19.334063 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.333739 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/145206ea-0a5b-4a75-b0dc-81e6412873ba-etc-sysconfig\") pod \"tuned-7sxcd\" (UID: \"145206ea-0a5b-4a75-b0dc-81e6412873ba\") " pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.334063 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.333762 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/145206ea-0a5b-4a75-b0dc-81e6412873ba-sys\") pod \"tuned-7sxcd\" (UID: \"145206ea-0a5b-4a75-b0dc-81e6412873ba\") " pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.334063 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.333794 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/145206ea-0a5b-4a75-b0dc-81e6412873ba-var-lib-kubelet\") pod \"tuned-7sxcd\" (UID: \"145206ea-0a5b-4a75-b0dc-81e6412873ba\") " pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.334063 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.333837 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dnt8\" (UniqueName: \"kubernetes.io/projected/145206ea-0a5b-4a75-b0dc-81e6412873ba-kube-api-access-7dnt8\") pod \"tuned-7sxcd\" (UID: \"145206ea-0a5b-4a75-b0dc-81e6412873ba\") " pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.334063 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.333840 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/145206ea-0a5b-4a75-b0dc-81e6412873ba-etc-sysconfig\") pod \"tuned-7sxcd\" (UID: \"145206ea-0a5b-4a75-b0dc-81e6412873ba\") " pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.334063 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.333838 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/145206ea-0a5b-4a75-b0dc-81e6412873ba-sys\") pod \"tuned-7sxcd\" (UID: \"145206ea-0a5b-4a75-b0dc-81e6412873ba\") " pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.334063 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.333870 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/145206ea-0a5b-4a75-b0dc-81e6412873ba-var-lib-kubelet\") pod \"tuned-7sxcd\" (UID: \"145206ea-0a5b-4a75-b0dc-81e6412873ba\") " pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.334063 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.333909 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14cb8ce5-3cb8-4827-8034-d74d7a8d75bf-metrics-certs\") pod \"network-metrics-daemon-dc669\" (UID: \"14cb8ce5-3cb8-4827-8034-d74d7a8d75bf\") " pod="openshift-multus/network-metrics-daemon-dc669" Jul 08 18:39:19.334063 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.333928 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-var-lib-openvswitch\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.334063 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.333951 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/abf012dc-98c1-40d7-aca4-8e9b5ccccea6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qw6cl\" (UID: \"abf012dc-98c1-40d7-aca4-8e9b5ccccea6\") " pod="openshift-multus/multus-additional-cni-plugins-qw6cl" Jul 08 18:39:19.334063 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.334011 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-var-lib-openvswitch\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.334063 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.334038 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/abf012dc-98c1-40d7-aca4-8e9b5ccccea6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qw6cl\" (UID: \"abf012dc-98c1-40d7-aca4-8e9b5ccccea6\") " pod="openshift-multus/multus-additional-cni-plugins-qw6cl" Jul 08 18:39:19.334063 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:19.334066 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:39:19.334685 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.334105 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32185124-ea41-4309-9674-13230b4d7a6e-tmp-dir\") pod \"node-resolver-x72mc\" (UID: \"32185124-ea41-4309-9674-13230b4d7a6e\") " pod="openshift-dns/node-resolver-x72mc" Jul 08 18:39:19.334685 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:19.334142 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14cb8ce5-3cb8-4827-8034-d74d7a8d75bf-metrics-certs podName:14cb8ce5-3cb8-4827-8034-d74d7a8d75bf nodeName:}" failed. No retries permitted until 2026-07-08 18:39:19.834119696 +0000 UTC m=+3.070417768 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/14cb8ce5-3cb8-4827-8034-d74d7a8d75bf-metrics-certs") pod "network-metrics-daemon-dc669" (UID: "14cb8ce5-3cb8-4827-8034-d74d7a8d75bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:39:19.334685 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.334210 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0b0ede89-a5f3-4ce8-b2e1-45451e6c443d-socket-dir\") pod \"aws-ebs-csi-driver-node-p4w8l\" (UID: \"0b0ede89-a5f3-4ce8-b2e1-45451e6c443d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4w8l" Jul 08 18:39:19.334685 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.334241 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-run-ovn\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.334685 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.334269 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/abf012dc-98c1-40d7-aca4-8e9b5ccccea6-cni-binary-copy\") pod \"multus-additional-cni-plugins-qw6cl\" (UID: \"abf012dc-98c1-40d7-aca4-8e9b5ccccea6\") " pod="openshift-multus/multus-additional-cni-plugins-qw6cl" Jul 08 18:39:19.334685 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.334304 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b958669d-dcaf-40b5-8b8d-36ff1ce96694-host-var-lib-cni-bin\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.334685 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.334317 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-run-ovn\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.334685 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.334331 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b958669d-dcaf-40b5-8b8d-36ff1ce96694-host-var-lib-kubelet\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.334685 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.334356 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a0618904-8d08-404a-9a82-a0bb55351381-dbus\") pod \"global-pull-secret-syncer-l9l9x\" (UID: \"a0618904-8d08-404a-9a82-a0bb55351381\") " pod="kube-system/global-pull-secret-syncer-l9l9x" Jul 08 18:39:19.334685 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.334397 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32185124-ea41-4309-9674-13230b4d7a6e-tmp-dir\") pod \"node-resolver-x72mc\" (UID: \"32185124-ea41-4309-9674-13230b4d7a6e\") " pod="openshift-dns/node-resolver-x72mc" Jul 08 18:39:19.334685 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.334402 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/145206ea-0a5b-4a75-b0dc-81e6412873ba-etc-sysctl-d\") pod \"tuned-7sxcd\" (UID: \"145206ea-0a5b-4a75-b0dc-81e6412873ba\") " pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.334685 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.334393 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0b0ede89-a5f3-4ce8-b2e1-45451e6c443d-socket-dir\") pod \"aws-ebs-csi-driver-node-p4w8l\" (UID: \"0b0ede89-a5f3-4ce8-b2e1-45451e6c443d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4w8l" Jul 08 18:39:19.334685 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.334434 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/145206ea-0a5b-4a75-b0dc-81e6412873ba-etc-systemd\") pod \"tuned-7sxcd\" (UID: \"145206ea-0a5b-4a75-b0dc-81e6412873ba\") " pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.334685 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.334476 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/145206ea-0a5b-4a75-b0dc-81e6412873ba-tmp\") pod \"tuned-7sxcd\" (UID: \"145206ea-0a5b-4a75-b0dc-81e6412873ba\") " pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.334685 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.334489 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/145206ea-0a5b-4a75-b0dc-81e6412873ba-etc-systemd\") pod \"tuned-7sxcd\" (UID: \"145206ea-0a5b-4a75-b0dc-81e6412873ba\") " pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.334685 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.334492 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-host-slash\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.334685 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.334517 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d155f1e7-7109-445e-b884-7a3e49df96f8-agent-certs\") pod \"konnectivity-agent-qklqp\" (UID: \"d155f1e7-7109-445e-b884-7a3e49df96f8\") " pod="kube-system/konnectivity-agent-qklqp" Jul 08 18:39:19.335512 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.334531 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-host-slash\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.335512 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.334534 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/145206ea-0a5b-4a75-b0dc-81e6412873ba-etc-sysctl-d\") pod \"tuned-7sxcd\" (UID: \"145206ea-0a5b-4a75-b0dc-81e6412873ba\") " pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.335512 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.334539 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/56014c3a-384a-4775-96e5-290968b196b1-ovn-node-metrics-cert\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.335512 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.334619 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/abf012dc-98c1-40d7-aca4-8e9b5ccccea6-cnibin\") pod \"multus-additional-cni-plugins-qw6cl\" (UID: \"abf012dc-98c1-40d7-aca4-8e9b5ccccea6\") " pod="openshift-multus/multus-additional-cni-plugins-qw6cl" Jul 08 18:39:19.335512 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.334646 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b958669d-dcaf-40b5-8b8d-36ff1ce96694-hostroot\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.335512 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.334676 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/145206ea-0a5b-4a75-b0dc-81e6412873ba-etc-kubernetes\") pod \"tuned-7sxcd\" (UID: \"145206ea-0a5b-4a75-b0dc-81e6412873ba\") " pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.335512 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.334705 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/145206ea-0a5b-4a75-b0dc-81e6412873ba-etc-sysctl-conf\") pod \"tuned-7sxcd\" (UID: \"145206ea-0a5b-4a75-b0dc-81e6412873ba\") " pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.335512 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.334735 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/145206ea-0a5b-4a75-b0dc-81e6412873ba-etc-kubernetes\") pod \"tuned-7sxcd\" (UID: \"145206ea-0a5b-4a75-b0dc-81e6412873ba\") " pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.335512 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.334744 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/abf012dc-98c1-40d7-aca4-8e9b5ccccea6-system-cni-dir\") pod \"multus-additional-cni-plugins-qw6cl\" (UID: \"abf012dc-98c1-40d7-aca4-8e9b5ccccea6\") " pod="openshift-multus/multus-additional-cni-plugins-qw6cl" Jul 08 18:39:19.335512 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.334775 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wmt9\" (UniqueName: \"kubernetes.io/projected/45985403-baf5-4513-a47e-be19f11afd47-kube-api-access-5wmt9\") pod \"iptables-alerter-fqjq8\" (UID: \"45985403-baf5-4513-a47e-be19f11afd47\") " pod="openshift-network-operator/iptables-alerter-fqjq8" Jul 08 18:39:19.335512 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.334820 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqh6b\" (UniqueName: \"kubernetes.io/projected/0b0ede89-a5f3-4ce8-b2e1-45451e6c443d-kube-api-access-bqh6b\") pod \"aws-ebs-csi-driver-node-p4w8l\" (UID: \"0b0ede89-a5f3-4ce8-b2e1-45451e6c443d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4w8l" Jul 08 18:39:19.335512 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.334817 2559 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 08 18:39:19.335512 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.334849 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-host-run-netns\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.335512 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.334874 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-run-openvswitch\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.335512 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.334874 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/145206ea-0a5b-4a75-b0dc-81e6412873ba-etc-sysctl-conf\") pod \"tuned-7sxcd\" (UID: \"145206ea-0a5b-4a75-b0dc-81e6412873ba\") " pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.335512 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.334899 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.335512 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.334923 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-host-run-netns\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.336284 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.334927 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ceb0a0a0-5ba1-4906-b214-5b1de0abbec3-host\") pod \"node-ca-q9hp4\" (UID: \"ceb0a0a0-5ba1-4906-b214-5b1de0abbec3\") " pod="openshift-image-registry/node-ca-q9hp4" Jul 08 18:39:19.336284 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.334928 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-run-openvswitch\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.336284 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.334966 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-run-systemd\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.336284 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.334971 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.336284 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.334991 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ceb0a0a0-5ba1-4906-b214-5b1de0abbec3-host\") pod \"node-ca-q9hp4\" (UID: \"ceb0a0a0-5ba1-4906-b214-5b1de0abbec3\") " pod="openshift-image-registry/node-ca-q9hp4" Jul 08 18:39:19.336284 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.334993 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/56014c3a-384a-4775-96e5-290968b196b1-env-overrides\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.336284 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335044 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b958669d-dcaf-40b5-8b8d-36ff1ce96694-host-run-multus-certs\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.336284 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335035 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-run-systemd\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.336284 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335073 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5btgw\" (UniqueName: \"kubernetes.io/projected/ceb0a0a0-5ba1-4906-b214-5b1de0abbec3-kube-api-access-5btgw\") pod \"node-ca-q9hp4\" (UID: \"ceb0a0a0-5ba1-4906-b214-5b1de0abbec3\") " pod="openshift-image-registry/node-ca-q9hp4" Jul 08 18:39:19.336284 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335098 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-systemd-units\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.336284 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335114 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-etc-openvswitch\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.336284 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335134 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-host-run-ovn-kubernetes\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.336284 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335159 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/56014c3a-384a-4775-96e5-290968b196b1-ovnkube-config\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.336284 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335166 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-systemd-units\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.336284 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335186 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/abf012dc-98c1-40d7-aca4-8e9b5ccccea6-os-release\") pod \"multus-additional-cni-plugins-qw6cl\" (UID: \"abf012dc-98c1-40d7-aca4-8e9b5ccccea6\") " pod="openshift-multus/multus-additional-cni-plugins-qw6cl" Jul 08 18:39:19.336284 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335192 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-etc-openvswitch\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.336284 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335213 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkdzl\" (UniqueName: \"kubernetes.io/projected/abf012dc-98c1-40d7-aca4-8e9b5ccccea6-kube-api-access-xkdzl\") pod \"multus-additional-cni-plugins-qw6cl\" (UID: \"abf012dc-98c1-40d7-aca4-8e9b5ccccea6\") " pod="openshift-multus/multus-additional-cni-plugins-qw6cl" Jul 08 18:39:19.337180 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335260 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-host-run-ovn-kubernetes\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.337180 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335263 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b958669d-dcaf-40b5-8b8d-36ff1ce96694-multus-daemon-config\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.337180 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335311 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/145206ea-0a5b-4a75-b0dc-81e6412873ba-etc-modprobe-d\") pod \"tuned-7sxcd\" (UID: \"145206ea-0a5b-4a75-b0dc-81e6412873ba\") " pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.337180 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335338 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/145206ea-0a5b-4a75-b0dc-81e6412873ba-etc-tuned\") pod \"tuned-7sxcd\" (UID: \"145206ea-0a5b-4a75-b0dc-81e6412873ba\") " pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.337180 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335363 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/32185124-ea41-4309-9674-13230b4d7a6e-hosts-file\") pod \"node-resolver-x72mc\" (UID: \"32185124-ea41-4309-9674-13230b4d7a6e\") " pod="openshift-dns/node-resolver-x72mc" Jul 08 18:39:19.337180 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335406 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b958669d-dcaf-40b5-8b8d-36ff1ce96694-multus-cni-dir\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.337180 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335436 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b958669d-dcaf-40b5-8b8d-36ff1ce96694-cni-binary-copy\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.337180 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335445 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/145206ea-0a5b-4a75-b0dc-81e6412873ba-etc-modprobe-d\") pod \"tuned-7sxcd\" (UID: \"145206ea-0a5b-4a75-b0dc-81e6412873ba\") " pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.337180 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335462 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/56014c3a-384a-4775-96e5-290968b196b1-env-overrides\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.337180 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335479 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b958669d-dcaf-40b5-8b8d-36ff1ce96694-host-run-netns\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.337180 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335498 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/32185124-ea41-4309-9674-13230b4d7a6e-hosts-file\") pod \"node-resolver-x72mc\" (UID: \"32185124-ea41-4309-9674-13230b4d7a6e\") " pod="openshift-dns/node-resolver-x72mc" Jul 08 18:39:19.337180 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335510 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlzw9\" (UniqueName: \"kubernetes.io/projected/205d4eb0-c6f3-4800-aa14-ece6eb2c362d-kube-api-access-qlzw9\") pod \"network-check-target-88zv8\" (UID: \"205d4eb0-c6f3-4800-aa14-ece6eb2c362d\") " pod="openshift-network-diagnostics/network-check-target-88zv8" Jul 08 18:39:19.337180 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335538 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/45985403-baf5-4513-a47e-be19f11afd47-iptables-alerter-script\") pod \"iptables-alerter-fqjq8\" (UID: \"45985403-baf5-4513-a47e-be19f11afd47\") " pod="openshift-network-operator/iptables-alerter-fqjq8" Jul 08 18:39:19.337180 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335592 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d155f1e7-7109-445e-b884-7a3e49df96f8-konnectivity-ca\") pod \"konnectivity-agent-qklqp\" (UID: \"d155f1e7-7109-445e-b884-7a3e49df96f8\") " pod="kube-system/konnectivity-agent-qklqp" Jul 08 18:39:19.337180 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335618 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0b0ede89-a5f3-4ce8-b2e1-45451e6c443d-etc-selinux\") pod \"aws-ebs-csi-driver-node-p4w8l\" (UID: \"0b0ede89-a5f3-4ce8-b2e1-45451e6c443d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4w8l" Jul 08 18:39:19.337180 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335644 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b958669d-dcaf-40b5-8b8d-36ff1ce96694-system-cni-dir\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.337180 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335669 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-host-kubelet\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.337695 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335690 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-host-cni-bin\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.337695 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335716 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b958669d-dcaf-40b5-8b8d-36ff1ce96694-host-var-lib-cni-multus\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.337695 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335739 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0b0ede89-a5f3-4ce8-b2e1-45451e6c443d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p4w8l\" (UID: \"0b0ede89-a5f3-4ce8-b2e1-45451e6c443d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4w8l" Jul 08 18:39:19.337695 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335760 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0b0ede89-a5f3-4ce8-b2e1-45451e6c443d-registration-dir\") pod \"aws-ebs-csi-driver-node-p4w8l\" (UID: \"0b0ede89-a5f3-4ce8-b2e1-45451e6c443d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4w8l" Jul 08 18:39:19.337695 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335783 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-node-log\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.337695 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335788 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/56014c3a-384a-4775-96e5-290968b196b1-ovnkube-config\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.337695 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335803 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-log-socket\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.337695 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335824 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/56014c3a-384a-4775-96e5-290968b196b1-ovnkube-script-lib\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.337695 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335867 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b958669d-dcaf-40b5-8b8d-36ff1ce96694-multus-socket-dir-parent\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.337695 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335870 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-host-cni-bin\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.337695 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335890 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a0618904-8d08-404a-9a82-a0bb55351381-kubelet-config\") pod \"global-pull-secret-syncer-l9l9x\" (UID: \"a0618904-8d08-404a-9a82-a0bb55351381\") " pod="kube-system/global-pull-secret-syncer-l9l9x" Jul 08 18:39:19.337695 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335914 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a0618904-8d08-404a-9a82-a0bb55351381-original-pull-secret\") pod \"global-pull-secret-syncer-l9l9x\" (UID: \"a0618904-8d08-404a-9a82-a0bb55351381\") " pod="kube-system/global-pull-secret-syncer-l9l9x" Jul 08 18:39:19.337695 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335927 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0b0ede89-a5f3-4ce8-b2e1-45451e6c443d-registration-dir\") pod \"aws-ebs-csi-driver-node-p4w8l\" (UID: \"0b0ede89-a5f3-4ce8-b2e1-45451e6c443d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4w8l" Jul 08 18:39:19.337695 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335952 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0b0ede89-a5f3-4ce8-b2e1-45451e6c443d-etc-selinux\") pod \"aws-ebs-csi-driver-node-p4w8l\" (UID: \"0b0ede89-a5f3-4ce8-b2e1-45451e6c443d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4w8l" Jul 08 18:39:19.337695 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.336002 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-host-kubelet\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.337695 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.336023 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/145206ea-0a5b-4a75-b0dc-81e6412873ba-run\") pod \"tuned-7sxcd\" (UID: \"145206ea-0a5b-4a75-b0dc-81e6412873ba\") " pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.337695 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.335935 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/145206ea-0a5b-4a75-b0dc-81e6412873ba-run\") pod \"tuned-7sxcd\" (UID: \"145206ea-0a5b-4a75-b0dc-81e6412873ba\") " pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.338281 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.336067 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0b0ede89-a5f3-4ce8-b2e1-45451e6c443d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p4w8l\" (UID: \"0b0ede89-a5f3-4ce8-b2e1-45451e6c443d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4w8l" Jul 08 18:39:19.338281 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.336067 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-node-log\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.338281 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.336097 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6ktv\" (UniqueName: \"kubernetes.io/projected/14cb8ce5-3cb8-4827-8034-d74d7a8d75bf-kube-api-access-w6ktv\") pod \"network-metrics-daemon-dc669\" (UID: \"14cb8ce5-3cb8-4827-8034-d74d7a8d75bf\") " pod="openshift-multus/network-metrics-daemon-dc669" Jul 08 18:39:19.338281 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.336133 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b958669d-dcaf-40b5-8b8d-36ff1ce96694-cnibin\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.338281 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.336208 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-log-socket\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.338281 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.336248 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45985403-baf5-4513-a47e-be19f11afd47-host-slash\") pod \"iptables-alerter-fqjq8\" (UID: \"45985403-baf5-4513-a47e-be19f11afd47\") " pod="openshift-network-operator/iptables-alerter-fqjq8" Jul 08 18:39:19.338281 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.336398 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sc88z\" (UniqueName: \"kubernetes.io/projected/32185124-ea41-4309-9674-13230b4d7a6e-kube-api-access-sc88z\") pod \"node-resolver-x72mc\" (UID: \"32185124-ea41-4309-9674-13230b4d7a6e\") " pod="openshift-dns/node-resolver-x72mc" Jul 08 18:39:19.338281 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.336438 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d155f1e7-7109-445e-b884-7a3e49df96f8-konnectivity-ca\") pod \"konnectivity-agent-qklqp\" (UID: \"d155f1e7-7109-445e-b884-7a3e49df96f8\") " pod="kube-system/konnectivity-agent-qklqp" Jul 08 18:39:19.338281 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.336451 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbm5v\" (UniqueName: \"kubernetes.io/projected/56014c3a-384a-4775-96e5-290968b196b1-kube-api-access-hbm5v\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.338281 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.336478 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/145206ea-0a5b-4a75-b0dc-81e6412873ba-lib-modules\") pod \"tuned-7sxcd\" (UID: \"145206ea-0a5b-4a75-b0dc-81e6412873ba\") " pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.338281 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.336526 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0b0ede89-a5f3-4ce8-b2e1-45451e6c443d-sys-fs\") pod \"aws-ebs-csi-driver-node-p4w8l\" (UID: \"0b0ede89-a5f3-4ce8-b2e1-45451e6c443d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4w8l" Jul 08 18:39:19.338281 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.336549 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ceb0a0a0-5ba1-4906-b214-5b1de0abbec3-serviceca\") pod \"node-ca-q9hp4\" (UID: \"ceb0a0a0-5ba1-4906-b214-5b1de0abbec3\") " pod="openshift-image-registry/node-ca-q9hp4" Jul 08 18:39:19.338281 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.336590 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-host-cni-netd\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.338281 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.336609 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/56014c3a-384a-4775-96e5-290968b196b1-ovnkube-script-lib\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.338281 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.336702 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0b0ede89-a5f3-4ce8-b2e1-45451e6c443d-sys-fs\") pod \"aws-ebs-csi-driver-node-p4w8l\" (UID: \"0b0ede89-a5f3-4ce8-b2e1-45451e6c443d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4w8l" Jul 08 18:39:19.338281 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.336616 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b958669d-dcaf-40b5-8b8d-36ff1ce96694-os-release\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.338281 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.336732 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/145206ea-0a5b-4a75-b0dc-81e6412873ba-lib-modules\") pod \"tuned-7sxcd\" (UID: \"145206ea-0a5b-4a75-b0dc-81e6412873ba\") " pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.338893 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.336742 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/56014c3a-384a-4775-96e5-290968b196b1-host-cni-netd\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.338893 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.337048 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ceb0a0a0-5ba1-4906-b214-5b1de0abbec3-serviceca\") pod \"node-ca-q9hp4\" (UID: \"ceb0a0a0-5ba1-4906-b214-5b1de0abbec3\") " pod="openshift-image-registry/node-ca-q9hp4" Jul 08 18:39:19.338893 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.338577 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/145206ea-0a5b-4a75-b0dc-81e6412873ba-tmp\") pod \"tuned-7sxcd\" (UID: \"145206ea-0a5b-4a75-b0dc-81e6412873ba\") " pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.338893 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.338593 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/56014c3a-384a-4775-96e5-290968b196b1-ovn-node-metrics-cert\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.338893 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.338740 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d155f1e7-7109-445e-b884-7a3e49df96f8-agent-certs\") pod \"konnectivity-agent-qklqp\" (UID: \"d155f1e7-7109-445e-b884-7a3e49df96f8\") " pod="kube-system/konnectivity-agent-qklqp" Jul 08 18:39:19.339176 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.339161 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/145206ea-0a5b-4a75-b0dc-81e6412873ba-etc-tuned\") pod \"tuned-7sxcd\" (UID: \"145206ea-0a5b-4a75-b0dc-81e6412873ba\") " pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.349029 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.349006 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqh6b\" (UniqueName: \"kubernetes.io/projected/0b0ede89-a5f3-4ce8-b2e1-45451e6c443d-kube-api-access-bqh6b\") pod \"aws-ebs-csi-driver-node-p4w8l\" (UID: \"0b0ede89-a5f3-4ce8-b2e1-45451e6c443d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4w8l" Jul 08 18:39:19.349460 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.349433 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dnt8\" (UniqueName: \"kubernetes.io/projected/145206ea-0a5b-4a75-b0dc-81e6412873ba-kube-api-access-7dnt8\") pod \"tuned-7sxcd\" (UID: \"145206ea-0a5b-4a75-b0dc-81e6412873ba\") " pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.349866 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.349846 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sc88z\" (UniqueName: \"kubernetes.io/projected/32185124-ea41-4309-9674-13230b4d7a6e-kube-api-access-sc88z\") pod \"node-resolver-x72mc\" (UID: \"32185124-ea41-4309-9674-13230b4d7a6e\") " pod="openshift-dns/node-resolver-x72mc" Jul 08 18:39:19.350221 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.350202 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbm5v\" (UniqueName: \"kubernetes.io/projected/56014c3a-384a-4775-96e5-290968b196b1-kube-api-access-hbm5v\") pod \"ovnkube-node-rj4xk\" (UID: \"56014c3a-384a-4775-96e5-290968b196b1\") " pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.350580 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.350560 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6ktv\" (UniqueName: \"kubernetes.io/projected/14cb8ce5-3cb8-4827-8034-d74d7a8d75bf-kube-api-access-w6ktv\") pod \"network-metrics-daemon-dc669\" (UID: \"14cb8ce5-3cb8-4827-8034-d74d7a8d75bf\") " pod="openshift-multus/network-metrics-daemon-dc669" Jul 08 18:39:19.350741 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.350723 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5btgw\" (UniqueName: \"kubernetes.io/projected/ceb0a0a0-5ba1-4906-b214-5b1de0abbec3-kube-api-access-5btgw\") pod \"node-ca-q9hp4\" (UID: \"ceb0a0a0-5ba1-4906-b214-5b1de0abbec3\") " pod="openshift-image-registry/node-ca-q9hp4" Jul 08 18:39:19.438380 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.438280 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/abf012dc-98c1-40d7-aca4-8e9b5ccccea6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qw6cl\" (UID: \"abf012dc-98c1-40d7-aca4-8e9b5ccccea6\") " pod="openshift-multus/multus-additional-cni-plugins-qw6cl" Jul 08 18:39:19.438380 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.438323 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/abf012dc-98c1-40d7-aca4-8e9b5ccccea6-cni-binary-copy\") pod \"multus-additional-cni-plugins-qw6cl\" (UID: \"abf012dc-98c1-40d7-aca4-8e9b5ccccea6\") " pod="openshift-multus/multus-additional-cni-plugins-qw6cl" Jul 08 18:39:19.438380 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.438347 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b958669d-dcaf-40b5-8b8d-36ff1ce96694-host-var-lib-cni-bin\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.438380 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.438371 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b958669d-dcaf-40b5-8b8d-36ff1ce96694-host-var-lib-kubelet\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.438731 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.438420 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b958669d-dcaf-40b5-8b8d-36ff1ce96694-host-var-lib-kubelet\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.438731 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.438470 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b958669d-dcaf-40b5-8b8d-36ff1ce96694-host-var-lib-cni-bin\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.438731 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.438495 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a0618904-8d08-404a-9a82-a0bb55351381-dbus\") pod \"global-pull-secret-syncer-l9l9x\" (UID: \"a0618904-8d08-404a-9a82-a0bb55351381\") " pod="kube-system/global-pull-secret-syncer-l9l9x" Jul 08 18:39:19.438731 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.438524 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/abf012dc-98c1-40d7-aca4-8e9b5ccccea6-cnibin\") pod \"multus-additional-cni-plugins-qw6cl\" (UID: \"abf012dc-98c1-40d7-aca4-8e9b5ccccea6\") " pod="openshift-multus/multus-additional-cni-plugins-qw6cl" Jul 08 18:39:19.438731 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.438549 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b958669d-dcaf-40b5-8b8d-36ff1ce96694-hostroot\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.438731 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.438572 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/abf012dc-98c1-40d7-aca4-8e9b5ccccea6-system-cni-dir\") pod \"multus-additional-cni-plugins-qw6cl\" (UID: \"abf012dc-98c1-40d7-aca4-8e9b5ccccea6\") " pod="openshift-multus/multus-additional-cni-plugins-qw6cl" Jul 08 18:39:19.438731 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.438588 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/abf012dc-98c1-40d7-aca4-8e9b5ccccea6-cnibin\") pod \"multus-additional-cni-plugins-qw6cl\" (UID: \"abf012dc-98c1-40d7-aca4-8e9b5ccccea6\") " pod="openshift-multus/multus-additional-cni-plugins-qw6cl" Jul 08 18:39:19.438731 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.438598 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wmt9\" (UniqueName: \"kubernetes.io/projected/45985403-baf5-4513-a47e-be19f11afd47-kube-api-access-5wmt9\") pod \"iptables-alerter-fqjq8\" (UID: \"45985403-baf5-4513-a47e-be19f11afd47\") " pod="openshift-network-operator/iptables-alerter-fqjq8" Jul 08 18:39:19.438731 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.438659 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a0618904-8d08-404a-9a82-a0bb55351381-dbus\") pod \"global-pull-secret-syncer-l9l9x\" (UID: \"a0618904-8d08-404a-9a82-a0bb55351381\") " pod="kube-system/global-pull-secret-syncer-l9l9x" Jul 08 18:39:19.438731 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.438671 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/abf012dc-98c1-40d7-aca4-8e9b5ccccea6-system-cni-dir\") pod \"multus-additional-cni-plugins-qw6cl\" (UID: \"abf012dc-98c1-40d7-aca4-8e9b5ccccea6\") " pod="openshift-multus/multus-additional-cni-plugins-qw6cl" Jul 08 18:39:19.438731 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.438634 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b958669d-dcaf-40b5-8b8d-36ff1ce96694-hostroot\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.438731 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.438692 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b958669d-dcaf-40b5-8b8d-36ff1ce96694-host-run-multus-certs\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.438731 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.438735 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/abf012dc-98c1-40d7-aca4-8e9b5ccccea6-os-release\") pod \"multus-additional-cni-plugins-qw6cl\" (UID: \"abf012dc-98c1-40d7-aca4-8e9b5ccccea6\") " pod="openshift-multus/multus-additional-cni-plugins-qw6cl" Jul 08 18:39:19.439440 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.438749 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b958669d-dcaf-40b5-8b8d-36ff1ce96694-host-run-multus-certs\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.439440 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.438760 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkdzl\" (UniqueName: \"kubernetes.io/projected/abf012dc-98c1-40d7-aca4-8e9b5ccccea6-kube-api-access-xkdzl\") pod \"multus-additional-cni-plugins-qw6cl\" (UID: \"abf012dc-98c1-40d7-aca4-8e9b5ccccea6\") " pod="openshift-multus/multus-additional-cni-plugins-qw6cl" Jul 08 18:39:19.439440 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.438867 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b958669d-dcaf-40b5-8b8d-36ff1ce96694-multus-daemon-config\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.439440 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.438905 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b958669d-dcaf-40b5-8b8d-36ff1ce96694-multus-cni-dir\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.439440 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.438931 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b958669d-dcaf-40b5-8b8d-36ff1ce96694-cni-binary-copy\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.439440 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.438955 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b958669d-dcaf-40b5-8b8d-36ff1ce96694-host-run-netns\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.439440 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.438907 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/abf012dc-98c1-40d7-aca4-8e9b5ccccea6-os-release\") pod \"multus-additional-cni-plugins-qw6cl\" (UID: \"abf012dc-98c1-40d7-aca4-8e9b5ccccea6\") " pod="openshift-multus/multus-additional-cni-plugins-qw6cl" Jul 08 18:39:19.439440 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.438999 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlzw9\" (UniqueName: \"kubernetes.io/projected/205d4eb0-c6f3-4800-aa14-ece6eb2c362d-kube-api-access-qlzw9\") pod \"network-check-target-88zv8\" (UID: \"205d4eb0-c6f3-4800-aa14-ece6eb2c362d\") " pod="openshift-network-diagnostics/network-check-target-88zv8" Jul 08 18:39:19.439440 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.439047 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/45985403-baf5-4513-a47e-be19f11afd47-iptables-alerter-script\") pod \"iptables-alerter-fqjq8\" (UID: \"45985403-baf5-4513-a47e-be19f11afd47\") " pod="openshift-network-operator/iptables-alerter-fqjq8" Jul 08 18:39:19.439440 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.439060 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/abf012dc-98c1-40d7-aca4-8e9b5ccccea6-cni-binary-copy\") pod \"multus-additional-cni-plugins-qw6cl\" (UID: \"abf012dc-98c1-40d7-aca4-8e9b5ccccea6\") " pod="openshift-multus/multus-additional-cni-plugins-qw6cl" Jul 08 18:39:19.439440 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.439079 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b958669d-dcaf-40b5-8b8d-36ff1ce96694-system-cni-dir\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.439440 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.439010 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b958669d-dcaf-40b5-8b8d-36ff1ce96694-multus-cni-dir\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.439440 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.439091 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/abf012dc-98c1-40d7-aca4-8e9b5ccccea6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qw6cl\" (UID: \"abf012dc-98c1-40d7-aca4-8e9b5ccccea6\") " pod="openshift-multus/multus-additional-cni-plugins-qw6cl" Jul 08 18:39:19.439440 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.439049 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b958669d-dcaf-40b5-8b8d-36ff1ce96694-host-run-netns\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.439440 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.439125 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b958669d-dcaf-40b5-8b8d-36ff1ce96694-host-var-lib-cni-multus\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.439440 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.439155 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b958669d-dcaf-40b5-8b8d-36ff1ce96694-multus-socket-dir-parent\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.439440 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.439179 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b958669d-dcaf-40b5-8b8d-36ff1ce96694-system-cni-dir\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.440071 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.439196 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a0618904-8d08-404a-9a82-a0bb55351381-kubelet-config\") pod \"global-pull-secret-syncer-l9l9x\" (UID: \"a0618904-8d08-404a-9a82-a0bb55351381\") " pod="kube-system/global-pull-secret-syncer-l9l9x" Jul 08 18:39:19.440071 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.439203 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b958669d-dcaf-40b5-8b8d-36ff1ce96694-host-var-lib-cni-multus\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.440071 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.439227 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a0618904-8d08-404a-9a82-a0bb55351381-original-pull-secret\") pod \"global-pull-secret-syncer-l9l9x\" (UID: \"a0618904-8d08-404a-9a82-a0bb55351381\") " pod="kube-system/global-pull-secret-syncer-l9l9x" Jul 08 18:39:19.440071 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.439234 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b958669d-dcaf-40b5-8b8d-36ff1ce96694-multus-socket-dir-parent\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.440071 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.439255 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b958669d-dcaf-40b5-8b8d-36ff1ce96694-cnibin\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.440071 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.439292 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45985403-baf5-4513-a47e-be19f11afd47-host-slash\") pod \"iptables-alerter-fqjq8\" (UID: \"45985403-baf5-4513-a47e-be19f11afd47\") " pod="openshift-network-operator/iptables-alerter-fqjq8" Jul 08 18:39:19.440071 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.439260 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a0618904-8d08-404a-9a82-a0bb55351381-kubelet-config\") pod \"global-pull-secret-syncer-l9l9x\" (UID: \"a0618904-8d08-404a-9a82-a0bb55351381\") " pod="kube-system/global-pull-secret-syncer-l9l9x" Jul 08 18:39:19.440071 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.439309 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b958669d-dcaf-40b5-8b8d-36ff1ce96694-cnibin\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.440071 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:19.439336 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 18:39:19.440071 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.439345 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b958669d-dcaf-40b5-8b8d-36ff1ce96694-os-release\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.440071 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.439357 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45985403-baf5-4513-a47e-be19f11afd47-host-slash\") pod \"iptables-alerter-fqjq8\" (UID: \"45985403-baf5-4513-a47e-be19f11afd47\") " pod="openshift-network-operator/iptables-alerter-fqjq8" Jul 08 18:39:19.440071 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.439380 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b958669d-dcaf-40b5-8b8d-36ff1ce96694-host-run-k8s-cni-cncf-io\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.440071 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:19.439396 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0618904-8d08-404a-9a82-a0bb55351381-original-pull-secret podName:a0618904-8d08-404a-9a82-a0bb55351381 nodeName:}" failed. No retries permitted until 2026-07-08 18:39:19.939376546 +0000 UTC m=+3.175674629 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a0618904-8d08-404a-9a82-a0bb55351381-original-pull-secret") pod "global-pull-secret-syncer-l9l9x" (UID: "a0618904-8d08-404a-9a82-a0bb55351381") : object "kube-system"/"original-pull-secret" not registered Jul 08 18:39:19.440071 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.439420 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b958669d-dcaf-40b5-8b8d-36ff1ce96694-os-release\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.440071 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.439426 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b958669d-dcaf-40b5-8b8d-36ff1ce96694-host-run-k8s-cni-cncf-io\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.440071 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.439428 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/abf012dc-98c1-40d7-aca4-8e9b5ccccea6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qw6cl\" (UID: \"abf012dc-98c1-40d7-aca4-8e9b5ccccea6\") " pod="openshift-multus/multus-additional-cni-plugins-qw6cl" Jul 08 18:39:19.440071 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.439457 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b958669d-dcaf-40b5-8b8d-36ff1ce96694-multus-conf-dir\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.440578 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.439484 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b958669d-dcaf-40b5-8b8d-36ff1ce96694-etc-kubernetes\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.440578 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.439483 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b958669d-dcaf-40b5-8b8d-36ff1ce96694-multus-daemon-config\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.440578 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.439496 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b958669d-dcaf-40b5-8b8d-36ff1ce96694-cni-binary-copy\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.440578 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.439529 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b958669d-dcaf-40b5-8b8d-36ff1ce96694-multus-conf-dir\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.440578 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.439533 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b958669d-dcaf-40b5-8b8d-36ff1ce96694-etc-kubernetes\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.440578 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.439548 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9zzh\" (UniqueName: \"kubernetes.io/projected/b958669d-dcaf-40b5-8b8d-36ff1ce96694-kube-api-access-k9zzh\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.440578 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.439575 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/abf012dc-98c1-40d7-aca4-8e9b5ccccea6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qw6cl\" (UID: \"abf012dc-98c1-40d7-aca4-8e9b5ccccea6\") " pod="openshift-multus/multus-additional-cni-plugins-qw6cl" Jul 08 18:39:19.440578 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.439599 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/abf012dc-98c1-40d7-aca4-8e9b5ccccea6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qw6cl\" (UID: \"abf012dc-98c1-40d7-aca4-8e9b5ccccea6\") " pod="openshift-multus/multus-additional-cni-plugins-qw6cl" Jul 08 18:39:19.440578 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.439645 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/45985403-baf5-4513-a47e-be19f11afd47-iptables-alerter-script\") pod \"iptables-alerter-fqjq8\" (UID: \"45985403-baf5-4513-a47e-be19f11afd47\") " pod="openshift-network-operator/iptables-alerter-fqjq8" Jul 08 18:39:19.440578 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.439970 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/abf012dc-98c1-40d7-aca4-8e9b5ccccea6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qw6cl\" (UID: \"abf012dc-98c1-40d7-aca4-8e9b5ccccea6\") " pod="openshift-multus/multus-additional-cni-plugins-qw6cl" Jul 08 18:39:19.446002 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:19.445981 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 18:39:19.446002 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:19.445999 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 18:39:19.446002 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:19.446008 2559 projected.go:196] Error preparing data for projected volume kube-api-access-qlzw9 for pod openshift-network-diagnostics/network-check-target-88zv8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:39:19.446210 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:19.446056 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/205d4eb0-c6f3-4800-aa14-ece6eb2c362d-kube-api-access-qlzw9 podName:205d4eb0-c6f3-4800-aa14-ece6eb2c362d nodeName:}" failed. No retries permitted until 2026-07-08 18:39:19.946043553 +0000 UTC m=+3.182341623 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qlzw9" (UniqueName: "kubernetes.io/projected/205d4eb0-c6f3-4800-aa14-ece6eb2c362d-kube-api-access-qlzw9") pod "network-check-target-88zv8" (UID: "205d4eb0-c6f3-4800-aa14-ece6eb2c362d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:39:19.448936 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.448915 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkdzl\" (UniqueName: \"kubernetes.io/projected/abf012dc-98c1-40d7-aca4-8e9b5ccccea6-kube-api-access-xkdzl\") pod \"multus-additional-cni-plugins-qw6cl\" (UID: \"abf012dc-98c1-40d7-aca4-8e9b5ccccea6\") " pod="openshift-multus/multus-additional-cni-plugins-qw6cl" Jul 08 18:39:19.449017 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.448957 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9zzh\" (UniqueName: \"kubernetes.io/projected/b958669d-dcaf-40b5-8b8d-36ff1ce96694-kube-api-access-k9zzh\") pod \"multus-kqlfn\" (UID: \"b958669d-dcaf-40b5-8b8d-36ff1ce96694\") " pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.449017 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.448966 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wmt9\" (UniqueName: \"kubernetes.io/projected/45985403-baf5-4513-a47e-be19f11afd47-kube-api-access-5wmt9\") pod \"iptables-alerter-fqjq8\" (UID: \"45985403-baf5-4513-a47e-be19f11afd47\") " pod="openshift-network-operator/iptables-alerter-fqjq8" Jul 08 18:39:19.451867 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.451850 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" Jul 08 18:39:19.457438 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.457415 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-x72mc" Jul 08 18:39:19.462514 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:19.462491 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod145206ea_0a5b_4a75_b0dc_81e6412873ba.slice/crio-f4c3b8f43b96b78ccc1cbdd0e1647b2fc6ee2526d5c7f425c1a6f2f6ddadd67c WatchSource:0}: Error finding container f4c3b8f43b96b78ccc1cbdd0e1647b2fc6ee2526d5c7f425c1a6f2f6ddadd67c: Status 404 returned error can't find the container with id f4c3b8f43b96b78ccc1cbdd0e1647b2fc6ee2526d5c7f425c1a6f2f6ddadd67c Jul 08 18:39:19.466463 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:19.466425 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32185124_ea41_4309_9674_13230b4d7a6e.slice/crio-b4d966fc6832d233a35aa1116a8d4e8cfa58f048e762c66af6e2b45f52265570 WatchSource:0}: Error finding container b4d966fc6832d233a35aa1116a8d4e8cfa58f048e762c66af6e2b45f52265570: Status 404 returned error can't find the container with id b4d966fc6832d233a35aa1116a8d4e8cfa58f048e762c66af6e2b45f52265570 Jul 08 18:39:19.467155 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.467128 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:19.470672 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.470654 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qklqp" Jul 08 18:39:19.476914 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.476897 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4w8l" Jul 08 18:39:19.477153 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:19.477120 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56014c3a_384a_4775_96e5_290968b196b1.slice/crio-6e738fcd046dc51764414f15c3a2b50d266faa8153127af81538bcb307c4d869 WatchSource:0}: Error finding container 6e738fcd046dc51764414f15c3a2b50d266faa8153127af81538bcb307c4d869: Status 404 returned error can't find the container with id 6e738fcd046dc51764414f15c3a2b50d266faa8153127af81538bcb307c4d869 Jul 08 18:39:19.479164 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:19.479140 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd155f1e7_7109_445e_b884_7a3e49df96f8.slice/crio-35d9c5e4248ff3efab0dc3e94f12532de4c17d904c9d04acff01a1d0c88bd0a3 WatchSource:0}: Error finding container 35d9c5e4248ff3efab0dc3e94f12532de4c17d904c9d04acff01a1d0c88bd0a3: Status 404 returned error can't find the container with id 35d9c5e4248ff3efab0dc3e94f12532de4c17d904c9d04acff01a1d0c88bd0a3 Jul 08 18:39:19.482438 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.482420 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-q9hp4" Jul 08 18:39:19.488332 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:19.488311 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b0ede89_a5f3_4ce8_b2e1_45451e6c443d.slice/crio-a2ef423fbc0d0f74dbc59003c6d853f0fbc1fa3df53e19c03ec2f4c43b1eddc7 WatchSource:0}: Error finding container a2ef423fbc0d0f74dbc59003c6d853f0fbc1fa3df53e19c03ec2f4c43b1eddc7: Status 404 returned error can't find the container with id a2ef423fbc0d0f74dbc59003c6d853f0fbc1fa3df53e19c03ec2f4c43b1eddc7 Jul 08 18:39:19.489004 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.488983 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qw6cl" Jul 08 18:39:19.493389 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:19.493228 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podceb0a0a0_5ba1_4906_b214_5b1de0abbec3.slice/crio-100985e5ee2c4b463d4768b76b078189c5d905b2485d79a267f2d349b14e7766 WatchSource:0}: Error finding container 100985e5ee2c4b463d4768b76b078189c5d905b2485d79a267f2d349b14e7766: Status 404 returned error can't find the container with id 100985e5ee2c4b463d4768b76b078189c5d905b2485d79a267f2d349b14e7766 Jul 08 18:39:19.493389 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.493351 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kqlfn" Jul 08 18:39:19.499094 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:19.499072 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabf012dc_98c1_40d7_aca4_8e9b5ccccea6.slice/crio-c213c0c4706d249b3ceb8a62856b065a5c6d8051f60e55c80c5cce2cecf1d558 WatchSource:0}: Error finding container c213c0c4706d249b3ceb8a62856b065a5c6d8051f60e55c80c5cce2cecf1d558: Status 404 returned error can't find the container with id c213c0c4706d249b3ceb8a62856b065a5c6d8051f60e55c80c5cce2cecf1d558 Jul 08 18:39:19.499194 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.499132 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fqjq8" Jul 08 18:39:19.503755 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:19.503730 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb958669d_dcaf_40b5_8b8d_36ff1ce96694.slice/crio-631b4ed47a2da2ed4fb80d1026a20df6459e95c428baf7dfe4af5a5f87ba3ac1 WatchSource:0}: Error finding container 631b4ed47a2da2ed4fb80d1026a20df6459e95c428baf7dfe4af5a5f87ba3ac1: Status 404 returned error can't find the container with id 631b4ed47a2da2ed4fb80d1026a20df6459e95c428baf7dfe4af5a5f87ba3ac1 Jul 08 18:39:19.509604 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:19.509569 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45985403_baf5_4513_a47e_be19f11afd47.slice/crio-fe18474daaec0f7ffac551fa1711c0584e548f6ab75058d611c7ae1e8f8ad536 WatchSource:0}: Error finding container fe18474daaec0f7ffac551fa1711c0584e548f6ab75058d611c7ae1e8f8ad536: Status 404 returned error can't find the container with id fe18474daaec0f7ffac551fa1711c0584e548f6ab75058d611c7ae1e8f8ad536 Jul 08 18:39:19.565143 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.565110 2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 18:39:19.686876 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.686805 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" event={"ID":"145206ea-0a5b-4a75-b0dc-81e6412873ba","Type":"ContainerStarted","Data":"f4c3b8f43b96b78ccc1cbdd0e1647b2fc6ee2526d5c7f425c1a6f2f6ddadd67c"} Jul 08 18:39:19.688521 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.688447 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-154.ec2.internal" event={"ID":"3872d75f39f988d68af7ca611d18fcd5","Type":"ContainerStarted","Data":"102f7a9d7976ce4c8f8ca6d678e42b8093064aabd7b505ac1c82ec92e2c9dd3a"} Jul 08 18:39:19.690442 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.690401 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qw6cl" event={"ID":"abf012dc-98c1-40d7-aca4-8e9b5ccccea6","Type":"ContainerStarted","Data":"c213c0c4706d249b3ceb8a62856b065a5c6d8051f60e55c80c5cce2cecf1d558"} Jul 08 18:39:19.692340 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.692314 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" event={"ID":"56014c3a-384a-4775-96e5-290968b196b1","Type":"ContainerStarted","Data":"6e738fcd046dc51764414f15c3a2b50d266faa8153127af81538bcb307c4d869"} Jul 08 18:39:19.693999 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.693915 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-x72mc" event={"ID":"32185124-ea41-4309-9674-13230b4d7a6e","Type":"ContainerStarted","Data":"b4d966fc6832d233a35aa1116a8d4e8cfa58f048e762c66af6e2b45f52265570"} Jul 08 18:39:19.695295 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.695269 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal" event={"ID":"b82dabbd147d65c747169cc61141ecbe","Type":"ContainerStarted","Data":"92c9ebc8399bc9624403fa2ce2bcd917197f55ef3e6d396f437aef5c886d8968"} Jul 08 18:39:19.696758 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.696729 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fqjq8" event={"ID":"45985403-baf5-4513-a47e-be19f11afd47","Type":"ContainerStarted","Data":"fe18474daaec0f7ffac551fa1711c0584e548f6ab75058d611c7ae1e8f8ad536"} Jul 08 18:39:19.698289 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.698203 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kqlfn" event={"ID":"b958669d-dcaf-40b5-8b8d-36ff1ce96694","Type":"ContainerStarted","Data":"631b4ed47a2da2ed4fb80d1026a20df6459e95c428baf7dfe4af5a5f87ba3ac1"} Jul 08 18:39:19.699944 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.699911 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-q9hp4" event={"ID":"ceb0a0a0-5ba1-4906-b214-5b1de0abbec3","Type":"ContainerStarted","Data":"100985e5ee2c4b463d4768b76b078189c5d905b2485d79a267f2d349b14e7766"} Jul 08 18:39:19.701669 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.701643 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4w8l" event={"ID":"0b0ede89-a5f3-4ce8-b2e1-45451e6c443d","Type":"ContainerStarted","Data":"a2ef423fbc0d0f74dbc59003c6d853f0fbc1fa3df53e19c03ec2f4c43b1eddc7"} Jul 08 18:39:19.703233 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.703210 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qklqp" event={"ID":"d155f1e7-7109-445e-b884-7a3e49df96f8","Type":"ContainerStarted","Data":"35d9c5e4248ff3efab0dc3e94f12532de4c17d904c9d04acff01a1d0c88bd0a3"} Jul 08 18:39:19.844212 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.844168 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14cb8ce5-3cb8-4827-8034-d74d7a8d75bf-metrics-certs\") pod \"network-metrics-daemon-dc669\" (UID: \"14cb8ce5-3cb8-4827-8034-d74d7a8d75bf\") " pod="openshift-multus/network-metrics-daemon-dc669" Jul 08 18:39:19.844426 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:19.844362 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:39:19.844426 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:19.844427 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14cb8ce5-3cb8-4827-8034-d74d7a8d75bf-metrics-certs podName:14cb8ce5-3cb8-4827-8034-d74d7a8d75bf nodeName:}" failed. No retries permitted until 2026-07-08 18:39:20.844407654 +0000 UTC m=+4.080705738 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/14cb8ce5-3cb8-4827-8034-d74d7a8d75bf-metrics-certs") pod "network-metrics-daemon-dc669" (UID: "14cb8ce5-3cb8-4827-8034-d74d7a8d75bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:39:19.945515 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:19.945422 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a0618904-8d08-404a-9a82-a0bb55351381-original-pull-secret\") pod \"global-pull-secret-syncer-l9l9x\" (UID: \"a0618904-8d08-404a-9a82-a0bb55351381\") " pod="kube-system/global-pull-secret-syncer-l9l9x" Jul 08 18:39:19.945729 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:19.945553 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 18:39:19.945729 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:19.945621 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0618904-8d08-404a-9a82-a0bb55351381-original-pull-secret podName:a0618904-8d08-404a-9a82-a0bb55351381 nodeName:}" failed. No retries permitted until 2026-07-08 18:39:20.945602268 +0000 UTC m=+4.181900338 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a0618904-8d08-404a-9a82-a0bb55351381-original-pull-secret") pod "global-pull-secret-syncer-l9l9x" (UID: "a0618904-8d08-404a-9a82-a0bb55351381") : object "kube-system"/"original-pull-secret" not registered Jul 08 18:39:20.045859 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:20.045823 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlzw9\" (UniqueName: \"kubernetes.io/projected/205d4eb0-c6f3-4800-aa14-ece6eb2c362d-kube-api-access-qlzw9\") pod \"network-check-target-88zv8\" (UID: \"205d4eb0-c6f3-4800-aa14-ece6eb2c362d\") " pod="openshift-network-diagnostics/network-check-target-88zv8" Jul 08 18:39:20.046064 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:20.046043 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 18:39:20.046130 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:20.046068 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 18:39:20.046130 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:20.046079 2559 projected.go:196] Error preparing data for projected volume kube-api-access-qlzw9 for pod openshift-network-diagnostics/network-check-target-88zv8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:39:20.046245 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:20.046134 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/205d4eb0-c6f3-4800-aa14-ece6eb2c362d-kube-api-access-qlzw9 podName:205d4eb0-c6f3-4800-aa14-ece6eb2c362d nodeName:}" failed. No retries permitted until 2026-07-08 18:39:21.046119444 +0000 UTC m=+4.282417516 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qlzw9" (UniqueName: "kubernetes.io/projected/205d4eb0-c6f3-4800-aa14-ece6eb2c362d-kube-api-access-qlzw9") pod "network-check-target-88zv8" (UID: "205d4eb0-c6f3-4800-aa14-ece6eb2c362d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:39:20.208921 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:20.208656 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 18:34:18 +0000 UTC" deadline="2028-04-23 18:13:48.10405084 +0000 UTC" Jul 08 18:39:20.208921 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:20.208831 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15719h34m27.895230849s" Jul 08 18:39:20.681379 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:20.681279 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dc669" Jul 08 18:39:20.681541 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:20.681442 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dc669" podUID="14cb8ce5-3cb8-4827-8034-d74d7a8d75bf" Jul 08 18:39:20.682135 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:20.681864 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88zv8" Jul 08 18:39:20.682135 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:20.681979 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88zv8" podUID="205d4eb0-c6f3-4800-aa14-ece6eb2c362d" Jul 08 18:39:20.682135 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:20.682012 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9l9x" Jul 08 18:39:20.682135 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:20.682084 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9l9x" podUID="a0618904-8d08-404a-9a82-a0bb55351381" Jul 08 18:39:20.852568 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:20.852525 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14cb8ce5-3cb8-4827-8034-d74d7a8d75bf-metrics-certs\") pod \"network-metrics-daemon-dc669\" (UID: \"14cb8ce5-3cb8-4827-8034-d74d7a8d75bf\") " pod="openshift-multus/network-metrics-daemon-dc669" Jul 08 18:39:20.852751 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:20.852694 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:39:20.852833 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:20.852782 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14cb8ce5-3cb8-4827-8034-d74d7a8d75bf-metrics-certs podName:14cb8ce5-3cb8-4827-8034-d74d7a8d75bf nodeName:}" failed. No retries permitted until 2026-07-08 18:39:22.852762316 +0000 UTC m=+6.089060404 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/14cb8ce5-3cb8-4827-8034-d74d7a8d75bf-metrics-certs") pod "network-metrics-daemon-dc669" (UID: "14cb8ce5-3cb8-4827-8034-d74d7a8d75bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:39:20.953485 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:20.953397 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a0618904-8d08-404a-9a82-a0bb55351381-original-pull-secret\") pod \"global-pull-secret-syncer-l9l9x\" (UID: \"a0618904-8d08-404a-9a82-a0bb55351381\") " pod="kube-system/global-pull-secret-syncer-l9l9x" Jul 08 18:39:20.953642 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:20.953568 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 18:39:20.953642 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:20.953632 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0618904-8d08-404a-9a82-a0bb55351381-original-pull-secret podName:a0618904-8d08-404a-9a82-a0bb55351381 nodeName:}" failed. No retries permitted until 2026-07-08 18:39:22.953614582 +0000 UTC m=+6.189912655 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a0618904-8d08-404a-9a82-a0bb55351381-original-pull-secret") pod "global-pull-secret-syncer-l9l9x" (UID: "a0618904-8d08-404a-9a82-a0bb55351381") : object "kube-system"/"original-pull-secret" not registered Jul 08 18:39:21.054447 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:21.054177 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlzw9\" (UniqueName: \"kubernetes.io/projected/205d4eb0-c6f3-4800-aa14-ece6eb2c362d-kube-api-access-qlzw9\") pod \"network-check-target-88zv8\" (UID: \"205d4eb0-c6f3-4800-aa14-ece6eb2c362d\") " pod="openshift-network-diagnostics/network-check-target-88zv8" Jul 08 18:39:21.054447 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:21.054357 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 18:39:21.054447 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:21.054382 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 18:39:21.054447 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:21.054396 2559 projected.go:196] Error preparing data for projected volume kube-api-access-qlzw9 for pod openshift-network-diagnostics/network-check-target-88zv8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:39:21.054447 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:21.054462 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/205d4eb0-c6f3-4800-aa14-ece6eb2c362d-kube-api-access-qlzw9 podName:205d4eb0-c6f3-4800-aa14-ece6eb2c362d nodeName:}" failed. No retries permitted until 2026-07-08 18:39:23.054441418 +0000 UTC m=+6.290739515 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qlzw9" (UniqueName: "kubernetes.io/projected/205d4eb0-c6f3-4800-aa14-ece6eb2c362d-kube-api-access-qlzw9") pod "network-check-target-88zv8" (UID: "205d4eb0-c6f3-4800-aa14-ece6eb2c362d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:39:22.681387 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:22.681354 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88zv8" Jul 08 18:39:22.681387 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:22.681386 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9l9x" Jul 08 18:39:22.682088 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:22.681482 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88zv8" podUID="205d4eb0-c6f3-4800-aa14-ece6eb2c362d" Jul 08 18:39:22.682088 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:22.681545 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9l9x" podUID="a0618904-8d08-404a-9a82-a0bb55351381" Jul 08 18:39:22.682088 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:22.681559 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dc669" Jul 08 18:39:22.682088 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:22.681658 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dc669" podUID="14cb8ce5-3cb8-4827-8034-d74d7a8d75bf" Jul 08 18:39:22.871480 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:22.871440 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14cb8ce5-3cb8-4827-8034-d74d7a8d75bf-metrics-certs\") pod \"network-metrics-daemon-dc669\" (UID: \"14cb8ce5-3cb8-4827-8034-d74d7a8d75bf\") " pod="openshift-multus/network-metrics-daemon-dc669" Jul 08 18:39:22.871663 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:22.871644 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:39:22.871735 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:22.871716 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14cb8ce5-3cb8-4827-8034-d74d7a8d75bf-metrics-certs podName:14cb8ce5-3cb8-4827-8034-d74d7a8d75bf nodeName:}" failed. No retries permitted until 2026-07-08 18:39:26.871696044 +0000 UTC m=+10.107994127 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/14cb8ce5-3cb8-4827-8034-d74d7a8d75bf-metrics-certs") pod "network-metrics-daemon-dc669" (UID: "14cb8ce5-3cb8-4827-8034-d74d7a8d75bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:39:22.972384 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:22.972178 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a0618904-8d08-404a-9a82-a0bb55351381-original-pull-secret\") pod \"global-pull-secret-syncer-l9l9x\" (UID: \"a0618904-8d08-404a-9a82-a0bb55351381\") " pod="kube-system/global-pull-secret-syncer-l9l9x" Jul 08 18:39:22.972384 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:22.972380 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 18:39:22.972601 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:22.972446 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0618904-8d08-404a-9a82-a0bb55351381-original-pull-secret podName:a0618904-8d08-404a-9a82-a0bb55351381 nodeName:}" failed. No retries permitted until 2026-07-08 18:39:26.972428466 +0000 UTC m=+10.208726542 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a0618904-8d08-404a-9a82-a0bb55351381-original-pull-secret") pod "global-pull-secret-syncer-l9l9x" (UID: "a0618904-8d08-404a-9a82-a0bb55351381") : object "kube-system"/"original-pull-secret" not registered Jul 08 18:39:23.073686 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:23.073614 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlzw9\" (UniqueName: \"kubernetes.io/projected/205d4eb0-c6f3-4800-aa14-ece6eb2c362d-kube-api-access-qlzw9\") pod \"network-check-target-88zv8\" (UID: \"205d4eb0-c6f3-4800-aa14-ece6eb2c362d\") " pod="openshift-network-diagnostics/network-check-target-88zv8" Jul 08 18:39:23.073861 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:23.073845 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 18:39:23.073935 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:23.073865 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 18:39:23.073935 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:23.073878 2559 projected.go:196] Error preparing data for projected volume kube-api-access-qlzw9 for pod openshift-network-diagnostics/network-check-target-88zv8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:39:23.074042 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:23.073984 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/205d4eb0-c6f3-4800-aa14-ece6eb2c362d-kube-api-access-qlzw9 podName:205d4eb0-c6f3-4800-aa14-ece6eb2c362d nodeName:}" failed. No retries permitted until 2026-07-08 18:39:27.073916543 +0000 UTC m=+10.310214620 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qlzw9" (UniqueName: "kubernetes.io/projected/205d4eb0-c6f3-4800-aa14-ece6eb2c362d-kube-api-access-qlzw9") pod "network-check-target-88zv8" (UID: "205d4eb0-c6f3-4800-aa14-ece6eb2c362d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:39:24.681911 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:24.681400 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88zv8" Jul 08 18:39:24.681911 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:24.681432 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9l9x" Jul 08 18:39:24.681911 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:24.681402 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dc669" Jul 08 18:39:24.681911 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:24.681541 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88zv8" podUID="205d4eb0-c6f3-4800-aa14-ece6eb2c362d" Jul 08 18:39:24.681911 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:24.681632 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dc669" podUID="14cb8ce5-3cb8-4827-8034-d74d7a8d75bf" Jul 08 18:39:24.681911 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:24.681715 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9l9x" podUID="a0618904-8d08-404a-9a82-a0bb55351381" Jul 08 18:39:26.682006 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:26.681923 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9l9x" Jul 08 18:39:26.682006 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:26.681940 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dc669" Jul 08 18:39:26.682531 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:26.682053 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9l9x" podUID="a0618904-8d08-404a-9a82-a0bb55351381" Jul 08 18:39:26.682531 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:26.682433 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88zv8" Jul 08 18:39:26.682632 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:26.682551 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88zv8" podUID="205d4eb0-c6f3-4800-aa14-ece6eb2c362d" Jul 08 18:39:26.682690 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:26.682658 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dc669" podUID="14cb8ce5-3cb8-4827-8034-d74d7a8d75bf" Jul 08 18:39:26.906667 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:26.906619 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14cb8ce5-3cb8-4827-8034-d74d7a8d75bf-metrics-certs\") pod \"network-metrics-daemon-dc669\" (UID: \"14cb8ce5-3cb8-4827-8034-d74d7a8d75bf\") " pod="openshift-multus/network-metrics-daemon-dc669" Jul 08 18:39:26.906861 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:26.906824 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:39:26.906926 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:26.906911 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14cb8ce5-3cb8-4827-8034-d74d7a8d75bf-metrics-certs podName:14cb8ce5-3cb8-4827-8034-d74d7a8d75bf nodeName:}" failed. No retries permitted until 2026-07-08 18:39:34.906887328 +0000 UTC m=+18.143185400 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/14cb8ce5-3cb8-4827-8034-d74d7a8d75bf-metrics-certs") pod "network-metrics-daemon-dc669" (UID: "14cb8ce5-3cb8-4827-8034-d74d7a8d75bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:39:27.007403 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:27.007312 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a0618904-8d08-404a-9a82-a0bb55351381-original-pull-secret\") pod \"global-pull-secret-syncer-l9l9x\" (UID: \"a0618904-8d08-404a-9a82-a0bb55351381\") " pod="kube-system/global-pull-secret-syncer-l9l9x" Jul 08 18:39:27.007562 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:27.007518 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 18:39:27.007619 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:27.007581 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0618904-8d08-404a-9a82-a0bb55351381-original-pull-secret podName:a0618904-8d08-404a-9a82-a0bb55351381 nodeName:}" failed. No retries permitted until 2026-07-08 18:39:35.00756245 +0000 UTC m=+18.243860534 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a0618904-8d08-404a-9a82-a0bb55351381-original-pull-secret") pod "global-pull-secret-syncer-l9l9x" (UID: "a0618904-8d08-404a-9a82-a0bb55351381") : object "kube-system"/"original-pull-secret" not registered Jul 08 18:39:27.108127 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:27.107891 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlzw9\" (UniqueName: \"kubernetes.io/projected/205d4eb0-c6f3-4800-aa14-ece6eb2c362d-kube-api-access-qlzw9\") pod \"network-check-target-88zv8\" (UID: \"205d4eb0-c6f3-4800-aa14-ece6eb2c362d\") " pod="openshift-network-diagnostics/network-check-target-88zv8" Jul 08 18:39:27.108127 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:27.108115 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 18:39:27.108127 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:27.108136 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 18:39:27.108407 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:27.108149 2559 projected.go:196] Error preparing data for projected volume kube-api-access-qlzw9 for pod openshift-network-diagnostics/network-check-target-88zv8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:39:27.108407 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:27.108208 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/205d4eb0-c6f3-4800-aa14-ece6eb2c362d-kube-api-access-qlzw9 podName:205d4eb0-c6f3-4800-aa14-ece6eb2c362d nodeName:}" failed. No retries permitted until 2026-07-08 18:39:35.108191719 +0000 UTC m=+18.344489789 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qlzw9" (UniqueName: "kubernetes.io/projected/205d4eb0-c6f3-4800-aa14-ece6eb2c362d-kube-api-access-qlzw9") pod "network-check-target-88zv8" (UID: "205d4eb0-c6f3-4800-aa14-ece6eb2c362d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:39:28.681969 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:28.681934 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9l9x" Jul 08 18:39:28.681969 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:28.681965 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dc669" Jul 08 18:39:28.682485 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:28.682060 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88zv8" Jul 08 18:39:28.682485 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:28.682078 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9l9x" podUID="a0618904-8d08-404a-9a82-a0bb55351381" Jul 08 18:39:28.682485 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:28.682159 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88zv8" podUID="205d4eb0-c6f3-4800-aa14-ece6eb2c362d" Jul 08 18:39:28.682485 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:28.682265 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dc669" podUID="14cb8ce5-3cb8-4827-8034-d74d7a8d75bf" Jul 08 18:39:30.681747 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:30.681697 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dc669" Jul 08 18:39:30.681747 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:30.681726 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88zv8" Jul 08 18:39:30.682260 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:30.681711 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9l9x" Jul 08 18:39:30.682260 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:30.681854 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dc669" podUID="14cb8ce5-3cb8-4827-8034-d74d7a8d75bf" Jul 08 18:39:30.682260 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:30.681917 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88zv8" podUID="205d4eb0-c6f3-4800-aa14-ece6eb2c362d" Jul 08 18:39:30.682260 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:30.681987 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9l9x" podUID="a0618904-8d08-404a-9a82-a0bb55351381" Jul 08 18:39:32.681285 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:32.681247 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88zv8" Jul 08 18:39:32.681673 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:32.681294 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9l9x" Jul 08 18:39:32.681673 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:32.681353 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dc669" Jul 08 18:39:32.681673 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:32.681370 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88zv8" podUID="205d4eb0-c6f3-4800-aa14-ece6eb2c362d" Jul 08 18:39:32.681673 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:32.681438 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dc669" podUID="14cb8ce5-3cb8-4827-8034-d74d7a8d75bf" Jul 08 18:39:32.681673 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:32.681518 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9l9x" podUID="a0618904-8d08-404a-9a82-a0bb55351381" Jul 08 18:39:34.681834 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:34.681793 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88zv8" Jul 08 18:39:34.681834 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:34.681816 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9l9x" Jul 08 18:39:34.682355 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:34.681815 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dc669" Jul 08 18:39:34.682355 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:34.681930 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88zv8" podUID="205d4eb0-c6f3-4800-aa14-ece6eb2c362d" Jul 08 18:39:34.682355 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:34.682039 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dc669" podUID="14cb8ce5-3cb8-4827-8034-d74d7a8d75bf" Jul 08 18:39:34.682355 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:34.682154 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9l9x" podUID="a0618904-8d08-404a-9a82-a0bb55351381" Jul 08 18:39:34.965962 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:34.965877 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14cb8ce5-3cb8-4827-8034-d74d7a8d75bf-metrics-certs\") pod \"network-metrics-daemon-dc669\" (UID: \"14cb8ce5-3cb8-4827-8034-d74d7a8d75bf\") " pod="openshift-multus/network-metrics-daemon-dc669" Jul 08 18:39:34.966105 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:34.966037 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:39:34.966105 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:34.966100 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14cb8ce5-3cb8-4827-8034-d74d7a8d75bf-metrics-certs podName:14cb8ce5-3cb8-4827-8034-d74d7a8d75bf nodeName:}" failed. No retries permitted until 2026-07-08 18:39:50.966083436 +0000 UTC m=+34.202381504 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/14cb8ce5-3cb8-4827-8034-d74d7a8d75bf-metrics-certs") pod "network-metrics-daemon-dc669" (UID: "14cb8ce5-3cb8-4827-8034-d74d7a8d75bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:39:35.066441 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:35.066399 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a0618904-8d08-404a-9a82-a0bb55351381-original-pull-secret\") pod \"global-pull-secret-syncer-l9l9x\" (UID: \"a0618904-8d08-404a-9a82-a0bb55351381\") " pod="kube-system/global-pull-secret-syncer-l9l9x" Jul 08 18:39:35.066616 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:35.066520 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 18:39:35.066616 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:35.066582 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0618904-8d08-404a-9a82-a0bb55351381-original-pull-secret podName:a0618904-8d08-404a-9a82-a0bb55351381 nodeName:}" failed. No retries permitted until 2026-07-08 18:39:51.066564274 +0000 UTC m=+34.302862352 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a0618904-8d08-404a-9a82-a0bb55351381-original-pull-secret") pod "global-pull-secret-syncer-l9l9x" (UID: "a0618904-8d08-404a-9a82-a0bb55351381") : object "kube-system"/"original-pull-secret" not registered Jul 08 18:39:35.167206 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:35.167174 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlzw9\" (UniqueName: \"kubernetes.io/projected/205d4eb0-c6f3-4800-aa14-ece6eb2c362d-kube-api-access-qlzw9\") pod \"network-check-target-88zv8\" (UID: \"205d4eb0-c6f3-4800-aa14-ece6eb2c362d\") " pod="openshift-network-diagnostics/network-check-target-88zv8" Jul 08 18:39:35.167407 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:35.167307 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 18:39:35.167407 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:35.167330 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 18:39:35.167407 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:35.167341 2559 projected.go:196] Error preparing data for projected volume kube-api-access-qlzw9 for pod openshift-network-diagnostics/network-check-target-88zv8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:39:35.167407 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:35.167402 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/205d4eb0-c6f3-4800-aa14-ece6eb2c362d-kube-api-access-qlzw9 podName:205d4eb0-c6f3-4800-aa14-ece6eb2c362d nodeName:}" failed. No retries permitted until 2026-07-08 18:39:51.167385745 +0000 UTC m=+34.403683837 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qlzw9" (UniqueName: "kubernetes.io/projected/205d4eb0-c6f3-4800-aa14-ece6eb2c362d-kube-api-access-qlzw9") pod "network-check-target-88zv8" (UID: "205d4eb0-c6f3-4800-aa14-ece6eb2c362d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:39:36.681304 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:36.681264 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88zv8" Jul 08 18:39:36.681740 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:36.681264 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dc669" Jul 08 18:39:36.681740 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:36.681375 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88zv8" podUID="205d4eb0-c6f3-4800-aa14-ece6eb2c362d" Jul 08 18:39:36.681740 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:36.681489 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dc669" podUID="14cb8ce5-3cb8-4827-8034-d74d7a8d75bf" Jul 08 18:39:36.681740 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:36.681278 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9l9x" Jul 08 18:39:36.681740 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:36.681599 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9l9x" podUID="a0618904-8d08-404a-9a82-a0bb55351381" Jul 08 18:39:38.682034 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:38.681890 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88zv8" Jul 08 18:39:38.682302 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:38.681926 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dc669" Jul 08 18:39:38.682302 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:38.682127 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88zv8" podUID="205d4eb0-c6f3-4800-aa14-ece6eb2c362d" Jul 08 18:39:38.682302 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:38.681925 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9l9x" Jul 08 18:39:38.682302 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:38.682204 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dc669" podUID="14cb8ce5-3cb8-4827-8034-d74d7a8d75bf" Jul 08 18:39:38.682302 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:38.682255 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9l9x" podUID="a0618904-8d08-404a-9a82-a0bb55351381" Jul 08 18:39:39.747967 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:39.747674 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-x72mc" event={"ID":"32185124-ea41-4309-9674-13230b4d7a6e","Type":"ContainerStarted","Data":"3af6ed77f404e8f158b649d90565af47bd3ad698c491b94dd010081ce5517bc0"} Jul 08 18:39:39.750120 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:39.750088 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal" event={"ID":"b82dabbd147d65c747169cc61141ecbe","Type":"ContainerStarted","Data":"3c764468375ad43d408601ac330fb06ffbdf1b27c185aa7d1fd0f6a74ee1b1e9"} Jul 08 18:39:39.751902 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:39.751877 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kqlfn" event={"ID":"b958669d-dcaf-40b5-8b8d-36ff1ce96694","Type":"ContainerStarted","Data":"b1a1fd345a9e7c8dda9a962798ff7630dfdb68e1943dbbe91a56d777379f0cd0"} Jul 08 18:39:39.753662 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:39.753591 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-q9hp4" event={"ID":"ceb0a0a0-5ba1-4906-b214-5b1de0abbec3","Type":"ContainerStarted","Data":"519440d8d3969b5a40671236e0ecb249b8170db517ba88c0e0ca6c4908a40987"} Jul 08 18:39:39.755498 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:39.755464 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4w8l" event={"ID":"0b0ede89-a5f3-4ce8-b2e1-45451e6c443d","Type":"ContainerStarted","Data":"d537d9bf60fb9893f4bf9e8d01f4df10ef8f009d9222b91ff8f5e245c87753ee"} Jul 08 18:39:39.756835 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:39.756811 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qklqp" event={"ID":"d155f1e7-7109-445e-b884-7a3e49df96f8","Type":"ContainerStarted","Data":"294be4f1b26ce045adf71b3d2a08f5d521bd615bb2045aef5a6d19a60fa2a64b"} Jul 08 18:39:39.758858 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:39.758645 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" event={"ID":"145206ea-0a5b-4a75-b0dc-81e6412873ba","Type":"ContainerStarted","Data":"dbb7a6c21ff65af2480827f59ad20293c0ae8cab802fa07c5850b30d34d225a6"} Jul 08 18:39:39.760574 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:39.760529 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-154.ec2.internal" event={"ID":"3872d75f39f988d68af7ca611d18fcd5","Type":"ContainerStarted","Data":"6b546c8e7caacdcdd6daf123f7b824e5033174170e6bcea34ff7276920b46fd0"} Jul 08 18:39:39.763676 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:39.763633 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qw6cl" event={"ID":"abf012dc-98c1-40d7-aca4-8e9b5ccccea6","Type":"ContainerStarted","Data":"1ce24c8569c8103f713b134721e5efc139bb4d06fe6f237ae061859058dc61ab"} Jul 08 18:39:39.767217 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:39.767043 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rj4xk_56014c3a-384a-4775-96e5-290968b196b1/ovn-acl-logging/0.log" Jul 08 18:39:39.767455 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:39.767400 2559 generic.go:356] "Generic (PLEG): container finished" podID="56014c3a-384a-4775-96e5-290968b196b1" containerID="52613cf0715fc1a7457b48ff4230eadda6e725acade034593344256bb3782a1c" exitCode=1 Jul 08 18:39:39.767536 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:39.767432 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" event={"ID":"56014c3a-384a-4775-96e5-290968b196b1","Type":"ContainerStarted","Data":"e7360e8b6f32b362f92c3e97f3f253ca4af92950d2244ae2db0d9dbc1b60fe67"} Jul 08 18:39:39.767536 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:39.767515 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" event={"ID":"56014c3a-384a-4775-96e5-290968b196b1","Type":"ContainerStarted","Data":"2a6b310b460f2a86deefeb3650f44fd06b6efd8c298a93e19c008acd6ca8939a"} Jul 08 18:39:39.767639 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:39.767546 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" event={"ID":"56014c3a-384a-4775-96e5-290968b196b1","Type":"ContainerStarted","Data":"1108541d01e9693336c0657d2605d1df29c5f8341f4f6e7a251899334952ee7e"} Jul 08 18:39:39.767639 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:39.767586 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" event={"ID":"56014c3a-384a-4775-96e5-290968b196b1","Type":"ContainerStarted","Data":"dca57e024994af0c199f9fa5712fc3a3a7bc86d8025da2678875eb097f95dc9d"} Jul 08 18:39:39.767639 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:39.767612 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" event={"ID":"56014c3a-384a-4775-96e5-290968b196b1","Type":"ContainerDied","Data":"52613cf0715fc1a7457b48ff4230eadda6e725acade034593344256bb3782a1c"} Jul 08 18:39:39.767849 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:39.767650 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" event={"ID":"56014c3a-384a-4775-96e5-290968b196b1","Type":"ContainerStarted","Data":"4538f1c5468e924ce71d3ccd5dcb4ca3e286bc9c6baa9fc05d5f3575c2b0bb18"} Jul 08 18:39:39.769315 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:39.769270 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-x72mc" podStartSLOduration=7.076951083 podStartE2EDuration="21.769257962s" totalImagesPullingTime="14.692306879s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:39:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:39:39.768575555 +0000 UTC m=+23.004873646" watchObservedRunningTime="2026-07-08 18:39:39.769257962 +0000 UTC m=+23.005556051" Jul 08 18:39:39.801492 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:39.801282 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-qklqp" podStartSLOduration=7.121816147 podStartE2EDuration="21.801267888s" totalImagesPullingTime="14.679451741s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:39:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:39:39.784985272 +0000 UTC m=+23.021283361" watchObservedRunningTime="2026-07-08 18:39:39.801267888 +0000 UTC m=+23.037565977" Jul 08 18:39:39.801577 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:39.801536 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-q9hp4" podStartSLOduration=2.680840253 podStartE2EDuration="21.801532697s" totalImagesPullingTime="19.120692444s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:39:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:39:39.801199221 +0000 UTC m=+23.037497322" watchObservedRunningTime="2026-07-08 18:39:39.801532697 +0000 UTC m=+23.037830787" Jul 08 18:39:39.859449 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:39.859394 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-154.ec2.internal" podStartSLOduration=21.859379781 podStartE2EDuration="21.859379781s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:39:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:39:39.858872336 +0000 UTC m=+23.095170427" watchObservedRunningTime="2026-07-08 18:39:39.859379781 +0000 UTC m=+23.095677870" Jul 08 18:39:39.878436 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:39.878385 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-kqlfn" podStartSLOduration=2.736540132 podStartE2EDuration="21.878370363s" totalImagesPullingTime="19.141830231s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:39:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:39:39.877929053 +0000 UTC m=+23.114227145" watchObservedRunningTime="2026-07-08 18:39:39.878370363 +0000 UTC m=+23.114668453" Jul 08 18:39:39.896029 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:39.895981 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7sxcd" podStartSLOduration=2.745019613 podStartE2EDuration="21.895967381s" totalImagesPullingTime="19.150947768s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:39:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:39:39.895636814 +0000 UTC m=+23.131934904" watchObservedRunningTime="2026-07-08 18:39:39.895967381 +0000 UTC m=+23.132265454" Jul 08 18:39:40.681667 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:40.681639 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88zv8" Jul 08 18:39:40.681667 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:40.681652 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9l9x" Jul 08 18:39:40.682003 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:40.681652 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dc669" Jul 08 18:39:40.682003 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:40.681738 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88zv8" podUID="205d4eb0-c6f3-4800-aa14-ece6eb2c362d" Jul 08 18:39:40.682003 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:40.681868 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9l9x" podUID="a0618904-8d08-404a-9a82-a0bb55351381" Jul 08 18:39:40.682003 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:40.681964 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dc669" podUID="14cb8ce5-3cb8-4827-8034-d74d7a8d75bf" Jul 08 18:39:40.771633 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:40.771595 2559 generic.go:356] "Generic (PLEG): container finished" podID="abf012dc-98c1-40d7-aca4-8e9b5ccccea6" containerID="1ce24c8569c8103f713b134721e5efc139bb4d06fe6f237ae061859058dc61ab" exitCode=0 Jul 08 18:39:40.772106 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:40.771690 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qw6cl" event={"ID":"abf012dc-98c1-40d7-aca4-8e9b5ccccea6","Type":"ContainerDied","Data":"1ce24c8569c8103f713b134721e5efc139bb4d06fe6f237ae061859058dc61ab"} Jul 08 18:39:40.773509 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:40.773490 2559 generic.go:356] "Generic (PLEG): container finished" podID="b82dabbd147d65c747169cc61141ecbe" containerID="3c764468375ad43d408601ac330fb06ffbdf1b27c185aa7d1fd0f6a74ee1b1e9" exitCode=0 Jul 08 18:39:40.773603 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:40.773579 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal" event={"ID":"b82dabbd147d65c747169cc61141ecbe","Type":"ContainerDied","Data":"3c764468375ad43d408601ac330fb06ffbdf1b27c185aa7d1fd0f6a74ee1b1e9"} Jul 08 18:39:40.775150 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:40.775123 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fqjq8" event={"ID":"45985403-baf5-4513-a47e-be19f11afd47","Type":"ContainerStarted","Data":"030ec954f06cd577d207f5ca07e96f3ef520764e9538e6aa478f220b9056061c"} Jul 08 18:39:40.832496 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:40.832437 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-fqjq8" podStartSLOduration=3.728434939 podStartE2EDuration="22.832420063s" totalImagesPullingTime="19.103985124s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:39:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:39:40.81492245 +0000 UTC m=+24.051220542" watchObservedRunningTime="2026-07-08 18:39:40.832420063 +0000 UTC m=+24.068718154" Jul 08 18:39:41.234266 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:41.234238 2559 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 08 18:39:41.335204 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:41.335165 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-qklqp" Jul 08 18:39:41.335853 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:41.335835 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-qklqp" Jul 08 18:39:41.697929 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:41.697818 2559 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T18:39:41.23426005Z","UUID":"8168e934-bcc9-4bfc-bd81-91125e206496","Handler":null,"Name":"","Endpoint":""} Jul 08 18:39:41.699394 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:41.699371 2559 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 08 18:39:41.699530 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:41.699428 2559 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 08 18:39:41.778989 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:41.778954 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4w8l" event={"ID":"0b0ede89-a5f3-4ce8-b2e1-45451e6c443d","Type":"ContainerStarted","Data":"7ab01a4e52c4e5dcbed92bc244e69c914403b3869bbbf63113f7138075b79a06"} Jul 08 18:39:41.780709 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:41.780682 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal" event={"ID":"b82dabbd147d65c747169cc61141ecbe","Type":"ContainerStarted","Data":"68e2360efa103814a8c322d8ed8e8c54b3a0f396c26c444bd38dcde0f03cc449"} Jul 08 18:39:41.797847 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:41.797804 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal" podStartSLOduration=23.797785219 podStartE2EDuration="23.797785219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:39:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:39:41.797682979 +0000 UTC m=+25.033981070" watchObservedRunningTime="2026-07-08 18:39:41.797785219 +0000 UTC m=+25.034083306" Jul 08 18:39:42.681399 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:42.681369 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88zv8" Jul 08 18:39:42.681560 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:42.681370 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dc669" Jul 08 18:39:42.681560 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:42.681479 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88zv8" podUID="205d4eb0-c6f3-4800-aa14-ece6eb2c362d" Jul 08 18:39:42.681560 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:42.681553 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dc669" podUID="14cb8ce5-3cb8-4827-8034-d74d7a8d75bf" Jul 08 18:39:42.681731 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:42.681580 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9l9x" Jul 08 18:39:42.681731 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:42.681670 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9l9x" podUID="a0618904-8d08-404a-9a82-a0bb55351381" Jul 08 18:39:42.784947 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:42.784852 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4w8l" event={"ID":"0b0ede89-a5f3-4ce8-b2e1-45451e6c443d","Type":"ContainerStarted","Data":"82cdd57b6a3f88a81c455df7919536712851f5bffe595e8b1cd282a0a75b5557"} Jul 08 18:39:42.788578 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:42.788554 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rj4xk_56014c3a-384a-4775-96e5-290968b196b1/ovn-acl-logging/0.log" Jul 08 18:39:42.789066 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:42.789012 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" event={"ID":"56014c3a-384a-4775-96e5-290968b196b1","Type":"ContainerStarted","Data":"68e0b77bd57dad252951d5dca54044f393ac3ca500047aa92b6e3ff526142efa"} Jul 08 18:39:42.789190 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:42.789076 2559 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 08 18:39:42.806438 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:42.806380 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4w8l" podStartSLOduration=3.047059813 podStartE2EDuration="24.806360517s" totalImagesPullingTime="21.759300704s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:39:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:39:42.805850346 +0000 UTC m=+26.042148437" watchObservedRunningTime="2026-07-08 18:39:42.806360517 +0000 UTC m=+26.042658609" Jul 08 18:39:44.681999 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:44.681799 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9l9x" Jul 08 18:39:44.682530 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:44.681836 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dc669" Jul 08 18:39:44.682530 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:44.682075 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9l9x" podUID="a0618904-8d08-404a-9a82-a0bb55351381" Jul 08 18:39:44.682530 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:44.681841 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88zv8" Jul 08 18:39:44.682530 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:44.682143 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dc669" podUID="14cb8ce5-3cb8-4827-8034-d74d7a8d75bf" Jul 08 18:39:44.682530 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:44.682220 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88zv8" podUID="205d4eb0-c6f3-4800-aa14-ece6eb2c362d" Jul 08 18:39:44.796500 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:44.796472 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rj4xk_56014c3a-384a-4775-96e5-290968b196b1/ovn-acl-logging/0.log" Jul 08 18:39:44.796847 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:44.796823 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" event={"ID":"56014c3a-384a-4775-96e5-290968b196b1","Type":"ContainerStarted","Data":"00ee3460e6b03cc86bb677059b26849e6d8addd105e09b83b96c237ff337c440"} Jul 08 18:39:44.797124 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:44.797098 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:44.797249 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:44.797164 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:44.797308 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:44.797259 2559 scope.go:117] "RemoveContainer" containerID="52613cf0715fc1a7457b48ff4230eadda6e725acade034593344256bb3782a1c" Jul 08 18:39:44.812635 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:44.812610 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:45.804346 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:45.804313 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rj4xk_56014c3a-384a-4775-96e5-290968b196b1/ovn-acl-logging/0.log" Jul 08 18:39:45.805072 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:45.804680 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" event={"ID":"56014c3a-384a-4775-96e5-290968b196b1","Type":"ContainerStarted","Data":"24cb0f7bcca669a9a3228f58bbc7e74526dc8fd67683a632bafc91ecd6f303bd"} Jul 08 18:39:45.805072 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:45.805037 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:45.821629 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:45.821567 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:39:45.887945 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:45.887642 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" podStartSLOduration=8.399341019 podStartE2EDuration="27.887624238s" totalImagesPullingTime="19.488283219s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:39:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:39:45.848743195 +0000 UTC m=+29.085041288" watchObservedRunningTime="2026-07-08 18:39:45.887624238 +0000 UTC m=+29.123922327" Jul 08 18:39:46.100797 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:46.100706 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-l9l9x"] Jul 08 18:39:46.100941 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:46.100866 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9l9x" Jul 08 18:39:46.101017 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:46.100991 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9l9x" podUID="a0618904-8d08-404a-9a82-a0bb55351381" Jul 08 18:39:46.104497 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:46.104470 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-88zv8"] Jul 08 18:39:46.104636 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:46.104570 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88zv8" Jul 08 18:39:46.104705 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:46.104653 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88zv8" podUID="205d4eb0-c6f3-4800-aa14-ece6eb2c362d" Jul 08 18:39:46.111108 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:46.111084 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dc669"] Jul 08 18:39:46.111236 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:46.111188 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dc669" Jul 08 18:39:46.111287 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:46.111274 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dc669" podUID="14cb8ce5-3cb8-4827-8034-d74d7a8d75bf" Jul 08 18:39:46.868026 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:46.867991 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-qklqp" Jul 08 18:39:46.868471 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:46.868115 2559 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 08 18:39:46.868679 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:46.868664 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qklqp" Jul 08 18:39:47.682579 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:47.682543 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dc669" Jul 08 18:39:47.682743 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:47.682627 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dc669" podUID="14cb8ce5-3cb8-4827-8034-d74d7a8d75bf" Jul 08 18:39:47.682743 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:47.682641 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88zv8" Jul 08 18:39:47.682872 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:47.682736 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88zv8" podUID="205d4eb0-c6f3-4800-aa14-ece6eb2c362d" Jul 08 18:39:47.682872 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:47.682789 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9l9x" Jul 08 18:39:47.682983 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:47.682867 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9l9x" podUID="a0618904-8d08-404a-9a82-a0bb55351381" Jul 08 18:39:48.812664 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:48.812473 2559 generic.go:356] "Generic (PLEG): container finished" podID="abf012dc-98c1-40d7-aca4-8e9b5ccccea6" containerID="07c6fbcfa9e1c209c0f2d3856a16952aa4bcc6244d26ab6814cfa0620c103581" exitCode=0 Jul 08 18:39:48.812664 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:48.812547 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qw6cl" event={"ID":"abf012dc-98c1-40d7-aca4-8e9b5ccccea6","Type":"ContainerDied","Data":"07c6fbcfa9e1c209c0f2d3856a16952aa4bcc6244d26ab6814cfa0620c103581"} Jul 08 18:39:49.681463 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:49.681445 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9l9x" Jul 08 18:39:49.681573 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:49.681446 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dc669" Jul 08 18:39:49.681573 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:49.681559 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9l9x" podUID="a0618904-8d08-404a-9a82-a0bb55351381" Jul 08 18:39:49.681689 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:49.681595 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88zv8" Jul 08 18:39:49.681751 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:49.681719 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dc669" podUID="14cb8ce5-3cb8-4827-8034-d74d7a8d75bf" Jul 08 18:39:49.681855 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:49.681832 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88zv8" podUID="205d4eb0-c6f3-4800-aa14-ece6eb2c362d" Jul 08 18:39:49.817381 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:49.817339 2559 generic.go:356] "Generic (PLEG): container finished" podID="abf012dc-98c1-40d7-aca4-8e9b5ccccea6" containerID="f97645584ad1799a992212b975ac4f1ecac956f3b4196f1ce48c0d83db983902" exitCode=0 Jul 08 18:39:49.817787 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:49.817423 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qw6cl" event={"ID":"abf012dc-98c1-40d7-aca4-8e9b5ccccea6","Type":"ContainerDied","Data":"f97645584ad1799a992212b975ac4f1ecac956f3b4196f1ce48c0d83db983902"} Jul 08 18:39:50.821373 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:50.821345 2559 generic.go:356] "Generic (PLEG): container finished" podID="abf012dc-98c1-40d7-aca4-8e9b5ccccea6" containerID="a3e6a5b7be3eb4a122f1a5ba7ad5386a87025dca7a78f43d79f3d75219fa23c7" exitCode=0 Jul 08 18:39:50.821670 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:50.821396 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qw6cl" event={"ID":"abf012dc-98c1-40d7-aca4-8e9b5ccccea6","Type":"ContainerDied","Data":"a3e6a5b7be3eb4a122f1a5ba7ad5386a87025dca7a78f43d79f3d75219fa23c7"} Jul 08 18:39:50.980236 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:50.980131 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14cb8ce5-3cb8-4827-8034-d74d7a8d75bf-metrics-certs\") pod \"network-metrics-daemon-dc669\" (UID: \"14cb8ce5-3cb8-4827-8034-d74d7a8d75bf\") " pod="openshift-multus/network-metrics-daemon-dc669" Jul 08 18:39:50.980375 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:50.980258 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:39:50.980375 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:50.980313 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14cb8ce5-3cb8-4827-8034-d74d7a8d75bf-metrics-certs podName:14cb8ce5-3cb8-4827-8034-d74d7a8d75bf nodeName:}" failed. No retries permitted until 2026-07-08 18:40:22.980298519 +0000 UTC m=+66.216596604 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/14cb8ce5-3cb8-4827-8034-d74d7a8d75bf-metrics-certs") pod "network-metrics-daemon-dc669" (UID: "14cb8ce5-3cb8-4827-8034-d74d7a8d75bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:39:51.080607 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:51.080572 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a0618904-8d08-404a-9a82-a0bb55351381-original-pull-secret\") pod \"global-pull-secret-syncer-l9l9x\" (UID: \"a0618904-8d08-404a-9a82-a0bb55351381\") " pod="kube-system/global-pull-secret-syncer-l9l9x" Jul 08 18:39:51.080793 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:51.080685 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 18:39:51.080793 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:51.080734 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0618904-8d08-404a-9a82-a0bb55351381-original-pull-secret podName:a0618904-8d08-404a-9a82-a0bb55351381 nodeName:}" failed. No retries permitted until 2026-07-08 18:40:23.080719148 +0000 UTC m=+66.317017216 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a0618904-8d08-404a-9a82-a0bb55351381-original-pull-secret") pod "global-pull-secret-syncer-l9l9x" (UID: "a0618904-8d08-404a-9a82-a0bb55351381") : object "kube-system"/"original-pull-secret" not registered Jul 08 18:39:51.181897 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:51.181866 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlzw9\" (UniqueName: \"kubernetes.io/projected/205d4eb0-c6f3-4800-aa14-ece6eb2c362d-kube-api-access-qlzw9\") pod \"network-check-target-88zv8\" (UID: \"205d4eb0-c6f3-4800-aa14-ece6eb2c362d\") " pod="openshift-network-diagnostics/network-check-target-88zv8" Jul 08 18:39:51.182027 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:51.181991 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 18:39:51.182027 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:51.182007 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 18:39:51.182027 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:51.182017 2559 projected.go:196] Error preparing data for projected volume kube-api-access-qlzw9 for pod openshift-network-diagnostics/network-check-target-88zv8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:39:51.182133 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:51.182061 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/205d4eb0-c6f3-4800-aa14-ece6eb2c362d-kube-api-access-qlzw9 podName:205d4eb0-c6f3-4800-aa14-ece6eb2c362d nodeName:}" failed. No retries permitted until 2026-07-08 18:40:23.182048977 +0000 UTC m=+66.418347046 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-qlzw9" (UniqueName: "kubernetes.io/projected/205d4eb0-c6f3-4800-aa14-ece6eb2c362d-kube-api-access-qlzw9") pod "network-check-target-88zv8" (UID: "205d4eb0-c6f3-4800-aa14-ece6eb2c362d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:39:51.681352 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:51.681315 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9l9x" Jul 08 18:39:51.681485 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:51.681358 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88zv8" Jul 08 18:39:51.681485 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:51.681318 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dc669" Jul 08 18:39:51.681485 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:51.681441 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l9l9x" podUID="a0618904-8d08-404a-9a82-a0bb55351381" Jul 08 18:39:51.681604 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:51.681504 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dc669" podUID="14cb8ce5-3cb8-4827-8034-d74d7a8d75bf" Jul 08 18:39:51.681604 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:51.681568 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88zv8" podUID="205d4eb0-c6f3-4800-aa14-ece6eb2c362d" Jul 08 18:39:52.431828 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.431640 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-154.ec2.internal" event="NodeReady" Jul 08 18:39:52.432283 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.431971 2559 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 08 18:39:52.482379 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.482347 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-8475db86b6-bj5jj"] Jul 08 18:39:52.484975 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.484957 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" Jul 08 18:39:52.489167 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.489129 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 08 18:39:52.492921 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.492900 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 08 18:39:52.493016 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.492923 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 08 18:39:52.493270 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.493176 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-n6k5m\"" Jul 08 18:39:52.506510 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.506480 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 08 18:39:52.515582 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.515564 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8475db86b6-bj5jj"] Jul 08 18:39:52.529140 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.529116 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-rdd7l"] Jul 08 18:39:52.532052 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.532035 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-9qx6k"] Jul 08 18:39:52.532194 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.532171 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rdd7l" Jul 08 18:39:52.534811 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.534787 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9qx6k" Jul 08 18:39:52.535558 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.535539 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 08 18:39:52.536230 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.535941 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-rmskx\"" Jul 08 18:39:52.536230 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.536138 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 08 18:39:52.538152 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.538131 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 08 18:39:52.538239 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.538131 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-nbwh9\"" Jul 08 18:39:52.538239 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.538170 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 08 18:39:52.538976 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.538957 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 08 18:39:52.546823 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.546805 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9qx6k"] Jul 08 18:39:52.549690 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.549672 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rdd7l"] Jul 08 18:39:52.594347 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.594268 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d317a3e7-440a-4808-8ce1-b748106a5614-image-registry-private-configuration\") pod \"image-registry-8475db86b6-bj5jj\" (UID: \"d317a3e7-440a-4808-8ce1-b748106a5614\") " pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" Jul 08 18:39:52.594347 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.594319 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d317a3e7-440a-4808-8ce1-b748106a5614-trusted-ca\") pod \"image-registry-8475db86b6-bj5jj\" (UID: \"d317a3e7-440a-4808-8ce1-b748106a5614\") " pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" Jul 08 18:39:52.594347 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.594345 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d317a3e7-440a-4808-8ce1-b748106a5614-bound-sa-token\") pod \"image-registry-8475db86b6-bj5jj\" (UID: \"d317a3e7-440a-4808-8ce1-b748106a5614\") " pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" Jul 08 18:39:52.594587 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.594367 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lh7z\" (UniqueName: \"kubernetes.io/projected/d317a3e7-440a-4808-8ce1-b748106a5614-kube-api-access-8lh7z\") pod \"image-registry-8475db86b6-bj5jj\" (UID: \"d317a3e7-440a-4808-8ce1-b748106a5614\") " pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" Jul 08 18:39:52.594587 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.594392 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71020478-23b3-4f75-a526-ed0da0f0ad8f-tmp-dir\") pod \"dns-default-rdd7l\" (UID: \"71020478-23b3-4f75-a526-ed0da0f0ad8f\") " pod="openshift-dns/dns-default-rdd7l" Jul 08 18:39:52.594587 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.594434 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d317a3e7-440a-4808-8ce1-b748106a5614-installation-pull-secrets\") pod \"image-registry-8475db86b6-bj5jj\" (UID: \"d317a3e7-440a-4808-8ce1-b748106a5614\") " pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" Jul 08 18:39:52.594587 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.594477 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc461fee-bcf4-4a16-92ec-50211c9314c6-cert\") pod \"ingress-canary-9qx6k\" (UID: \"cc461fee-bcf4-4a16-92ec-50211c9314c6\") " pod="openshift-ingress-canary/ingress-canary-9qx6k" Jul 08 18:39:52.594587 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.594511 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfnqg\" (UniqueName: \"kubernetes.io/projected/cc461fee-bcf4-4a16-92ec-50211c9314c6-kube-api-access-zfnqg\") pod \"ingress-canary-9qx6k\" (UID: \"cc461fee-bcf4-4a16-92ec-50211c9314c6\") " pod="openshift-ingress-canary/ingress-canary-9qx6k" Jul 08 18:39:52.594587 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.594540 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/71020478-23b3-4f75-a526-ed0da0f0ad8f-metrics-tls\") pod \"dns-default-rdd7l\" (UID: \"71020478-23b3-4f75-a526-ed0da0f0ad8f\") " pod="openshift-dns/dns-default-rdd7l" Jul 08 18:39:52.594587 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.594575 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mjs2\" (UniqueName: \"kubernetes.io/projected/71020478-23b3-4f75-a526-ed0da0f0ad8f-kube-api-access-7mjs2\") pod \"dns-default-rdd7l\" (UID: \"71020478-23b3-4f75-a526-ed0da0f0ad8f\") " pod="openshift-dns/dns-default-rdd7l" Jul 08 18:39:52.594957 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.594642 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/71020478-23b3-4f75-a526-ed0da0f0ad8f-config-volume\") pod \"dns-default-rdd7l\" (UID: \"71020478-23b3-4f75-a526-ed0da0f0ad8f\") " pod="openshift-dns/dns-default-rdd7l" Jul 08 18:39:52.594957 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.594692 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d317a3e7-440a-4808-8ce1-b748106a5614-registry-tls\") pod \"image-registry-8475db86b6-bj5jj\" (UID: \"d317a3e7-440a-4808-8ce1-b748106a5614\") " pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" Jul 08 18:39:52.594957 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.594711 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d317a3e7-440a-4808-8ce1-b748106a5614-ca-trust-extracted\") pod \"image-registry-8475db86b6-bj5jj\" (UID: \"d317a3e7-440a-4808-8ce1-b748106a5614\") " pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" Jul 08 18:39:52.594957 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.594734 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d317a3e7-440a-4808-8ce1-b748106a5614-registry-certificates\") pod \"image-registry-8475db86b6-bj5jj\" (UID: \"d317a3e7-440a-4808-8ce1-b748106a5614\") " pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" Jul 08 18:39:52.695395 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.695360 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d317a3e7-440a-4808-8ce1-b748106a5614-ca-trust-extracted\") pod \"image-registry-8475db86b6-bj5jj\" (UID: \"d317a3e7-440a-4808-8ce1-b748106a5614\") " pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" Jul 08 18:39:52.695583 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.695402 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d317a3e7-440a-4808-8ce1-b748106a5614-registry-certificates\") pod \"image-registry-8475db86b6-bj5jj\" (UID: \"d317a3e7-440a-4808-8ce1-b748106a5614\") " pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" Jul 08 18:39:52.695583 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.695439 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d317a3e7-440a-4808-8ce1-b748106a5614-image-registry-private-configuration\") pod \"image-registry-8475db86b6-bj5jj\" (UID: \"d317a3e7-440a-4808-8ce1-b748106a5614\") " pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" Jul 08 18:39:52.695583 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.695493 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d317a3e7-440a-4808-8ce1-b748106a5614-trusted-ca\") pod \"image-registry-8475db86b6-bj5jj\" (UID: \"d317a3e7-440a-4808-8ce1-b748106a5614\") " pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" Jul 08 18:39:52.695583 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.695516 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d317a3e7-440a-4808-8ce1-b748106a5614-bound-sa-token\") pod \"image-registry-8475db86b6-bj5jj\" (UID: \"d317a3e7-440a-4808-8ce1-b748106a5614\") " pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" Jul 08 18:39:52.695583 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.695540 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lh7z\" (UniqueName: \"kubernetes.io/projected/d317a3e7-440a-4808-8ce1-b748106a5614-kube-api-access-8lh7z\") pod \"image-registry-8475db86b6-bj5jj\" (UID: \"d317a3e7-440a-4808-8ce1-b748106a5614\") " pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" Jul 08 18:39:52.695583 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.695566 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71020478-23b3-4f75-a526-ed0da0f0ad8f-tmp-dir\") pod \"dns-default-rdd7l\" (UID: \"71020478-23b3-4f75-a526-ed0da0f0ad8f\") " pod="openshift-dns/dns-default-rdd7l" Jul 08 18:39:52.695954 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.695611 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d317a3e7-440a-4808-8ce1-b748106a5614-installation-pull-secrets\") pod \"image-registry-8475db86b6-bj5jj\" (UID: \"d317a3e7-440a-4808-8ce1-b748106a5614\") " pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" Jul 08 18:39:52.695954 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.695638 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc461fee-bcf4-4a16-92ec-50211c9314c6-cert\") pod \"ingress-canary-9qx6k\" (UID: \"cc461fee-bcf4-4a16-92ec-50211c9314c6\") " pod="openshift-ingress-canary/ingress-canary-9qx6k" Jul 08 18:39:52.695954 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.695660 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfnqg\" (UniqueName: \"kubernetes.io/projected/cc461fee-bcf4-4a16-92ec-50211c9314c6-kube-api-access-zfnqg\") pod \"ingress-canary-9qx6k\" (UID: \"cc461fee-bcf4-4a16-92ec-50211c9314c6\") " pod="openshift-ingress-canary/ingress-canary-9qx6k" Jul 08 18:39:52.695954 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.695684 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/71020478-23b3-4f75-a526-ed0da0f0ad8f-metrics-tls\") pod \"dns-default-rdd7l\" (UID: \"71020478-23b3-4f75-a526-ed0da0f0ad8f\") " pod="openshift-dns/dns-default-rdd7l" Jul 08 18:39:52.695954 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.695709 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mjs2\" (UniqueName: \"kubernetes.io/projected/71020478-23b3-4f75-a526-ed0da0f0ad8f-kube-api-access-7mjs2\") pod \"dns-default-rdd7l\" (UID: \"71020478-23b3-4f75-a526-ed0da0f0ad8f\") " pod="openshift-dns/dns-default-rdd7l" Jul 08 18:39:52.695954 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.695736 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/71020478-23b3-4f75-a526-ed0da0f0ad8f-config-volume\") pod \"dns-default-rdd7l\" (UID: \"71020478-23b3-4f75-a526-ed0da0f0ad8f\") " pod="openshift-dns/dns-default-rdd7l" Jul 08 18:39:52.695954 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.695797 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d317a3e7-440a-4808-8ce1-b748106a5614-registry-tls\") pod \"image-registry-8475db86b6-bj5jj\" (UID: \"d317a3e7-440a-4808-8ce1-b748106a5614\") " pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" Jul 08 18:39:52.695954 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.695838 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d317a3e7-440a-4808-8ce1-b748106a5614-ca-trust-extracted\") pod \"image-registry-8475db86b6-bj5jj\" (UID: \"d317a3e7-440a-4808-8ce1-b748106a5614\") " pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" Jul 08 18:39:52.695954 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:52.695895 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 18:39:52.695954 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:52.695910 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8475db86b6-bj5jj: secret "image-registry-tls" not found Jul 08 18:39:52.695954 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:52.695962 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d317a3e7-440a-4808-8ce1-b748106a5614-registry-tls podName:d317a3e7-440a-4808-8ce1-b748106a5614 nodeName:}" failed. No retries permitted until 2026-07-08 18:39:53.195944102 +0000 UTC m=+36.432242193 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d317a3e7-440a-4808-8ce1-b748106a5614-registry-tls") pod "image-registry-8475db86b6-bj5jj" (UID: "d317a3e7-440a-4808-8ce1-b748106a5614") : secret "image-registry-tls" not found Jul 08 18:39:52.696472 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:52.695969 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 18:39:52.696472 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:52.696027 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71020478-23b3-4f75-a526-ed0da0f0ad8f-metrics-tls podName:71020478-23b3-4f75-a526-ed0da0f0ad8f nodeName:}" failed. No retries permitted until 2026-07-08 18:39:53.196008924 +0000 UTC m=+36.432306995 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/71020478-23b3-4f75-a526-ed0da0f0ad8f-metrics-tls") pod "dns-default-rdd7l" (UID: "71020478-23b3-4f75-a526-ed0da0f0ad8f") : secret "dns-default-metrics-tls" not found Jul 08 18:39:52.696622 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.696575 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71020478-23b3-4f75-a526-ed0da0f0ad8f-tmp-dir\") pod \"dns-default-rdd7l\" (UID: \"71020478-23b3-4f75-a526-ed0da0f0ad8f\") " pod="openshift-dns/dns-default-rdd7l" Jul 08 18:39:52.696622 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.696610 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d317a3e7-440a-4808-8ce1-b748106a5614-trusted-ca\") pod \"image-registry-8475db86b6-bj5jj\" (UID: \"d317a3e7-440a-4808-8ce1-b748106a5614\") " pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" Jul 08 18:39:52.696821 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.696754 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/71020478-23b3-4f75-a526-ed0da0f0ad8f-config-volume\") pod \"dns-default-rdd7l\" (UID: \"71020478-23b3-4f75-a526-ed0da0f0ad8f\") " pod="openshift-dns/dns-default-rdd7l" Jul 08 18:39:52.696927 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.696905 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d317a3e7-440a-4808-8ce1-b748106a5614-registry-certificates\") pod \"image-registry-8475db86b6-bj5jj\" (UID: \"d317a3e7-440a-4808-8ce1-b748106a5614\") " pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" Jul 08 18:39:52.700716 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.700694 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d317a3e7-440a-4808-8ce1-b748106a5614-image-registry-private-configuration\") pod \"image-registry-8475db86b6-bj5jj\" (UID: \"d317a3e7-440a-4808-8ce1-b748106a5614\") " pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" Jul 08 18:39:52.700847 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.700738 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc461fee-bcf4-4a16-92ec-50211c9314c6-cert\") pod \"ingress-canary-9qx6k\" (UID: \"cc461fee-bcf4-4a16-92ec-50211c9314c6\") " pod="openshift-ingress-canary/ingress-canary-9qx6k" Jul 08 18:39:52.700999 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.700979 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d317a3e7-440a-4808-8ce1-b748106a5614-installation-pull-secrets\") pod \"image-registry-8475db86b6-bj5jj\" (UID: \"d317a3e7-440a-4808-8ce1-b748106a5614\") " pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" Jul 08 18:39:52.706593 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.706561 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d317a3e7-440a-4808-8ce1-b748106a5614-bound-sa-token\") pod \"image-registry-8475db86b6-bj5jj\" (UID: \"d317a3e7-440a-4808-8ce1-b748106a5614\") " pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" Jul 08 18:39:52.706848 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.706820 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfnqg\" (UniqueName: \"kubernetes.io/projected/cc461fee-bcf4-4a16-92ec-50211c9314c6-kube-api-access-zfnqg\") pod \"ingress-canary-9qx6k\" (UID: \"cc461fee-bcf4-4a16-92ec-50211c9314c6\") " pod="openshift-ingress-canary/ingress-canary-9qx6k" Jul 08 18:39:52.707040 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.706936 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mjs2\" (UniqueName: \"kubernetes.io/projected/71020478-23b3-4f75-a526-ed0da0f0ad8f-kube-api-access-7mjs2\") pod \"dns-default-rdd7l\" (UID: \"71020478-23b3-4f75-a526-ed0da0f0ad8f\") " pod="openshift-dns/dns-default-rdd7l" Jul 08 18:39:52.707425 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.707400 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lh7z\" (UniqueName: \"kubernetes.io/projected/d317a3e7-440a-4808-8ce1-b748106a5614-kube-api-access-8lh7z\") pod \"image-registry-8475db86b6-bj5jj\" (UID: \"d317a3e7-440a-4808-8ce1-b748106a5614\") " pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" Jul 08 18:39:52.849058 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:52.848985 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9qx6k" Jul 08 18:39:53.004616 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:53.004536 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9qx6k"] Jul 08 18:39:53.009450 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:39:53.009421 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc461fee_bcf4_4a16_92ec_50211c9314c6.slice/crio-d0b24ef093e1aa49c9c7430d6572a093e07e36d732aba4dd84ef9636b578e4b2 WatchSource:0}: Error finding container d0b24ef093e1aa49c9c7430d6572a093e07e36d732aba4dd84ef9636b578e4b2: Status 404 returned error can't find the container with id d0b24ef093e1aa49c9c7430d6572a093e07e36d732aba4dd84ef9636b578e4b2 Jul 08 18:39:53.200122 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:53.200091 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/71020478-23b3-4f75-a526-ed0da0f0ad8f-metrics-tls\") pod \"dns-default-rdd7l\" (UID: \"71020478-23b3-4f75-a526-ed0da0f0ad8f\") " pod="openshift-dns/dns-default-rdd7l" Jul 08 18:39:53.200279 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:53.200155 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d317a3e7-440a-4808-8ce1-b748106a5614-registry-tls\") pod \"image-registry-8475db86b6-bj5jj\" (UID: \"d317a3e7-440a-4808-8ce1-b748106a5614\") " pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" Jul 08 18:39:53.200279 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:53.200242 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 18:39:53.200279 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:53.200242 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 18:39:53.200375 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:53.200305 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71020478-23b3-4f75-a526-ed0da0f0ad8f-metrics-tls podName:71020478-23b3-4f75-a526-ed0da0f0ad8f nodeName:}" failed. No retries permitted until 2026-07-08 18:39:54.200289371 +0000 UTC m=+37.436587439 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/71020478-23b3-4f75-a526-ed0da0f0ad8f-metrics-tls") pod "dns-default-rdd7l" (UID: "71020478-23b3-4f75-a526-ed0da0f0ad8f") : secret "dns-default-metrics-tls" not found Jul 08 18:39:53.200375 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:53.200252 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8475db86b6-bj5jj: secret "image-registry-tls" not found Jul 08 18:39:53.200442 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:53.200379 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d317a3e7-440a-4808-8ce1-b748106a5614-registry-tls podName:d317a3e7-440a-4808-8ce1-b748106a5614 nodeName:}" failed. No retries permitted until 2026-07-08 18:39:54.200365586 +0000 UTC m=+37.436663655 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d317a3e7-440a-4808-8ce1-b748106a5614-registry-tls") pod "image-registry-8475db86b6-bj5jj" (UID: "d317a3e7-440a-4808-8ce1-b748106a5614") : secret "image-registry-tls" not found Jul 08 18:39:53.681969 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:53.681934 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88zv8" Jul 08 18:39:53.682798 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:53.682071 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dc669" Jul 08 18:39:53.682798 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:53.681939 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9l9x" Jul 08 18:39:53.685789 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:53.685753 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 18:39:53.685789 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:53.685761 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 18:39:53.687108 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:53.687090 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 18:39:53.687203 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:53.687137 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sc5vc\"" Jul 08 18:39:53.687203 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:53.687157 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 18:39:53.687203 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:53.687197 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-msx7w\"" Jul 08 18:39:53.830404 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:53.830367 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9qx6k" event={"ID":"cc461fee-bcf4-4a16-92ec-50211c9314c6","Type":"ContainerStarted","Data":"d0b24ef093e1aa49c9c7430d6572a093e07e36d732aba4dd84ef9636b578e4b2"} Jul 08 18:39:54.207217 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:54.207180 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/71020478-23b3-4f75-a526-ed0da0f0ad8f-metrics-tls\") pod \"dns-default-rdd7l\" (UID: \"71020478-23b3-4f75-a526-ed0da0f0ad8f\") " pod="openshift-dns/dns-default-rdd7l" Jul 08 18:39:54.207424 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:54.207237 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d317a3e7-440a-4808-8ce1-b748106a5614-registry-tls\") pod \"image-registry-8475db86b6-bj5jj\" (UID: \"d317a3e7-440a-4808-8ce1-b748106a5614\") " pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" Jul 08 18:39:54.207424 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:54.207354 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 18:39:54.207424 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:54.207365 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8475db86b6-bj5jj: secret "image-registry-tls" not found Jul 08 18:39:54.207424 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:54.207366 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 18:39:54.207424 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:54.207427 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d317a3e7-440a-4808-8ce1-b748106a5614-registry-tls podName:d317a3e7-440a-4808-8ce1-b748106a5614 nodeName:}" failed. No retries permitted until 2026-07-08 18:39:56.207408351 +0000 UTC m=+39.443706422 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d317a3e7-440a-4808-8ce1-b748106a5614-registry-tls") pod "image-registry-8475db86b6-bj5jj" (UID: "d317a3e7-440a-4808-8ce1-b748106a5614") : secret "image-registry-tls" not found Jul 08 18:39:54.207633 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:54.207441 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71020478-23b3-4f75-a526-ed0da0f0ad8f-metrics-tls podName:71020478-23b3-4f75-a526-ed0da0f0ad8f nodeName:}" failed. No retries permitted until 2026-07-08 18:39:56.207434792 +0000 UTC m=+39.443732860 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/71020478-23b3-4f75-a526-ed0da0f0ad8f-metrics-tls") pod "dns-default-rdd7l" (UID: "71020478-23b3-4f75-a526-ed0da0f0ad8f") : secret "dns-default-metrics-tls" not found Jul 08 18:39:56.225316 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:56.225095 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d317a3e7-440a-4808-8ce1-b748106a5614-registry-tls\") pod \"image-registry-8475db86b6-bj5jj\" (UID: \"d317a3e7-440a-4808-8ce1-b748106a5614\") " pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" Jul 08 18:39:56.225791 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:56.225256 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 18:39:56.225791 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:56.225390 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/71020478-23b3-4f75-a526-ed0da0f0ad8f-metrics-tls\") pod \"dns-default-rdd7l\" (UID: \"71020478-23b3-4f75-a526-ed0da0f0ad8f\") " pod="openshift-dns/dns-default-rdd7l" Jul 08 18:39:56.225791 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:56.225399 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8475db86b6-bj5jj: secret "image-registry-tls" not found Jul 08 18:39:56.225791 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:56.225477 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d317a3e7-440a-4808-8ce1-b748106a5614-registry-tls podName:d317a3e7-440a-4808-8ce1-b748106a5614 nodeName:}" failed. No retries permitted until 2026-07-08 18:40:00.225452394 +0000 UTC m=+43.461750487 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d317a3e7-440a-4808-8ce1-b748106a5614-registry-tls") pod "image-registry-8475db86b6-bj5jj" (UID: "d317a3e7-440a-4808-8ce1-b748106a5614") : secret "image-registry-tls" not found Jul 08 18:39:56.225791 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:56.225528 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 18:39:56.225791 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:39:56.225581 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71020478-23b3-4f75-a526-ed0da0f0ad8f-metrics-tls podName:71020478-23b3-4f75-a526-ed0da0f0ad8f nodeName:}" failed. No retries permitted until 2026-07-08 18:40:00.225570675 +0000 UTC m=+43.461868742 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/71020478-23b3-4f75-a526-ed0da0f0ad8f-metrics-tls") pod "dns-default-rdd7l" (UID: "71020478-23b3-4f75-a526-ed0da0f0ad8f") : secret "dns-default-metrics-tls" not found Jul 08 18:39:57.842842 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:57.842806 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9qx6k" event={"ID":"cc461fee-bcf4-4a16-92ec-50211c9314c6","Type":"ContainerStarted","Data":"e6ce2843d8d8de50733bd52d9a64a6a497868aa18cf32cbe71bb9867a0c7cd97"} Jul 08 18:39:57.846116 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:57.846090 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qw6cl" event={"ID":"abf012dc-98c1-40d7-aca4-8e9b5ccccea6","Type":"ContainerStarted","Data":"13e8e5e2af9a2bc7e9944696af5803b848c61a7056f87e6f82faca0f9452feda"} Jul 08 18:39:57.861653 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:57.861592 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-9qx6k" podStartSLOduration=1.206476699 podStartE2EDuration="5.861571839s" totalImagesPullingTime="4.65509514s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:39:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:39:57.860957651 +0000 UTC m=+41.097255744" watchObservedRunningTime="2026-07-08 18:39:57.861571839 +0000 UTC m=+41.097869931" Jul 08 18:39:58.850860 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:58.850823 2559 generic.go:356] "Generic (PLEG): container finished" podID="abf012dc-98c1-40d7-aca4-8e9b5ccccea6" containerID="13e8e5e2af9a2bc7e9944696af5803b848c61a7056f87e6f82faca0f9452feda" exitCode=0 Jul 08 18:39:58.851212 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:58.850908 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qw6cl" event={"ID":"abf012dc-98c1-40d7-aca4-8e9b5ccccea6","Type":"ContainerDied","Data":"13e8e5e2af9a2bc7e9944696af5803b848c61a7056f87e6f82faca0f9452feda"} Jul 08 18:39:59.855838 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:59.855802 2559 generic.go:356] "Generic (PLEG): container finished" podID="abf012dc-98c1-40d7-aca4-8e9b5ccccea6" containerID="fe018d0b09a5d957c2aad384fcb4cd505cf93c7cb325c5f24efe4c040a72b024" exitCode=0 Jul 08 18:39:59.856205 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:39:59.855862 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qw6cl" event={"ID":"abf012dc-98c1-40d7-aca4-8e9b5ccccea6","Type":"ContainerDied","Data":"fe018d0b09a5d957c2aad384fcb4cd505cf93c7cb325c5f24efe4c040a72b024"} Jul 08 18:40:00.258112 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:00.258076 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/71020478-23b3-4f75-a526-ed0da0f0ad8f-metrics-tls\") pod \"dns-default-rdd7l\" (UID: \"71020478-23b3-4f75-a526-ed0da0f0ad8f\") " pod="openshift-dns/dns-default-rdd7l" Jul 08 18:40:00.258270 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:00.258127 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d317a3e7-440a-4808-8ce1-b748106a5614-registry-tls\") pod \"image-registry-8475db86b6-bj5jj\" (UID: \"d317a3e7-440a-4808-8ce1-b748106a5614\") " pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" Jul 08 18:40:00.258270 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:40:00.258230 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 18:40:00.258337 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:40:00.258290 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71020478-23b3-4f75-a526-ed0da0f0ad8f-metrics-tls podName:71020478-23b3-4f75-a526-ed0da0f0ad8f nodeName:}" failed. No retries permitted until 2026-07-08 18:40:08.258275135 +0000 UTC m=+51.494573203 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/71020478-23b3-4f75-a526-ed0da0f0ad8f-metrics-tls") pod "dns-default-rdd7l" (UID: "71020478-23b3-4f75-a526-ed0da0f0ad8f") : secret "dns-default-metrics-tls" not found Jul 08 18:40:00.258337 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:40:00.258233 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 18:40:00.258337 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:40:00.258311 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8475db86b6-bj5jj: secret "image-registry-tls" not found Jul 08 18:40:00.258430 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:40:00.258342 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d317a3e7-440a-4808-8ce1-b748106a5614-registry-tls podName:d317a3e7-440a-4808-8ce1-b748106a5614 nodeName:}" failed. No retries permitted until 2026-07-08 18:40:08.25833297 +0000 UTC m=+51.494631038 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d317a3e7-440a-4808-8ce1-b748106a5614-registry-tls") pod "image-registry-8475db86b6-bj5jj" (UID: "d317a3e7-440a-4808-8ce1-b748106a5614") : secret "image-registry-tls" not found Jul 08 18:40:00.861819 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:00.861786 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qw6cl" event={"ID":"abf012dc-98c1-40d7-aca4-8e9b5ccccea6","Type":"ContainerStarted","Data":"fe06c4846ae120e736fb7749884c5ec652a53f66f7996115643f2f2734822ba9"} Jul 08 18:40:00.892636 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:00.892580 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qw6cl" podStartSLOduration=8.314657297 podStartE2EDuration="42.89256365s" totalImagesPullingTime="34.577906353s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:39:18 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:40:00.890628952 +0000 UTC m=+44.126927042" watchObservedRunningTime="2026-07-08 18:40:00.89256365 +0000 UTC m=+44.128861739" Jul 08 18:40:08.317142 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:08.317102 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/71020478-23b3-4f75-a526-ed0da0f0ad8f-metrics-tls\") pod \"dns-default-rdd7l\" (UID: \"71020478-23b3-4f75-a526-ed0da0f0ad8f\") " pod="openshift-dns/dns-default-rdd7l" Jul 08 18:40:08.317543 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:08.317160 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d317a3e7-440a-4808-8ce1-b748106a5614-registry-tls\") pod \"image-registry-8475db86b6-bj5jj\" (UID: \"d317a3e7-440a-4808-8ce1-b748106a5614\") " pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" Jul 08 18:40:08.317543 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:40:08.317259 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 18:40:08.317543 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:40:08.317281 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 18:40:08.317543 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:40:08.317292 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8475db86b6-bj5jj: secret "image-registry-tls" not found Jul 08 18:40:08.317543 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:40:08.317334 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71020478-23b3-4f75-a526-ed0da0f0ad8f-metrics-tls podName:71020478-23b3-4f75-a526-ed0da0f0ad8f nodeName:}" failed. No retries permitted until 2026-07-08 18:40:24.317316998 +0000 UTC m=+67.553615070 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/71020478-23b3-4f75-a526-ed0da0f0ad8f-metrics-tls") pod "dns-default-rdd7l" (UID: "71020478-23b3-4f75-a526-ed0da0f0ad8f") : secret "dns-default-metrics-tls" not found Jul 08 18:40:08.317543 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:40:08.317349 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d317a3e7-440a-4808-8ce1-b748106a5614-registry-tls podName:d317a3e7-440a-4808-8ce1-b748106a5614 nodeName:}" failed. No retries permitted until 2026-07-08 18:40:24.317342724 +0000 UTC m=+67.553640791 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d317a3e7-440a-4808-8ce1-b748106a5614-registry-tls") pod "image-registry-8475db86b6-bj5jj" (UID: "d317a3e7-440a-4808-8ce1-b748106a5614") : secret "image-registry-tls" not found Jul 08 18:40:17.828290 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:17.828260 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rj4xk" Jul 08 18:40:23.020443 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:23.020382 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14cb8ce5-3cb8-4827-8034-d74d7a8d75bf-metrics-certs\") pod \"network-metrics-daemon-dc669\" (UID: \"14cb8ce5-3cb8-4827-8034-d74d7a8d75bf\") " pod="openshift-multus/network-metrics-daemon-dc669" Jul 08 18:40:23.023890 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:23.023870 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 18:40:23.031564 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:40:23.031546 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 08 18:40:23.031612 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:40:23.031604 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14cb8ce5-3cb8-4827-8034-d74d7a8d75bf-metrics-certs podName:14cb8ce5-3cb8-4827-8034-d74d7a8d75bf nodeName:}" failed. No retries permitted until 2026-07-08 18:41:27.031586966 +0000 UTC m=+130.267885033 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/14cb8ce5-3cb8-4827-8034-d74d7a8d75bf-metrics-certs") pod "network-metrics-daemon-dc669" (UID: "14cb8ce5-3cb8-4827-8034-d74d7a8d75bf") : secret "metrics-daemon-secret" not found Jul 08 18:40:23.121386 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:23.121340 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a0618904-8d08-404a-9a82-a0bb55351381-original-pull-secret\") pod \"global-pull-secret-syncer-l9l9x\" (UID: \"a0618904-8d08-404a-9a82-a0bb55351381\") " pod="kube-system/global-pull-secret-syncer-l9l9x" Jul 08 18:40:23.124777 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:23.124751 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 18:40:23.134702 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:23.134683 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a0618904-8d08-404a-9a82-a0bb55351381-original-pull-secret\") pod \"global-pull-secret-syncer-l9l9x\" (UID: \"a0618904-8d08-404a-9a82-a0bb55351381\") " pod="kube-system/global-pull-secret-syncer-l9l9x" Jul 08 18:40:23.222103 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:23.222050 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlzw9\" (UniqueName: \"kubernetes.io/projected/205d4eb0-c6f3-4800-aa14-ece6eb2c362d-kube-api-access-qlzw9\") pod \"network-check-target-88zv8\" (UID: \"205d4eb0-c6f3-4800-aa14-ece6eb2c362d\") " pod="openshift-network-diagnostics/network-check-target-88zv8" Jul 08 18:40:23.225481 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:23.225464 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 18:40:23.236129 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:23.236110 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 18:40:23.246224 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:23.246199 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlzw9\" (UniqueName: \"kubernetes.io/projected/205d4eb0-c6f3-4800-aa14-ece6eb2c362d-kube-api-access-qlzw9\") pod \"network-check-target-88zv8\" (UID: \"205d4eb0-c6f3-4800-aa14-ece6eb2c362d\") " pod="openshift-network-diagnostics/network-check-target-88zv8" Jul 08 18:40:23.396046 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:23.396017 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-msx7w\"" Jul 08 18:40:23.403479 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:23.403458 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88zv8" Jul 08 18:40:23.404701 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:23.404684 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l9l9x" Jul 08 18:40:23.544604 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:23.544573 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-l9l9x"] Jul 08 18:40:23.548408 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:40:23.548381 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0618904_8d08_404a_9a82_a0bb55351381.slice/crio-4db1f4f9157391e9aec85c09211b872f8ec4c8cc578ad8a0807170e36d738d67 WatchSource:0}: Error finding container 4db1f4f9157391e9aec85c09211b872f8ec4c8cc578ad8a0807170e36d738d67: Status 404 returned error can't find the container with id 4db1f4f9157391e9aec85c09211b872f8ec4c8cc578ad8a0807170e36d738d67 Jul 08 18:40:23.577988 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:23.577806 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-88zv8"] Jul 08 18:40:23.582208 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:40:23.582185 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod205d4eb0_c6f3_4800_aa14_ece6eb2c362d.slice/crio-b79696b8d0285d0540fc5e20e16c9deec8771488189277c58eabf8023cac6560 WatchSource:0}: Error finding container b79696b8d0285d0540fc5e20e16c9deec8771488189277c58eabf8023cac6560: Status 404 returned error can't find the container with id b79696b8d0285d0540fc5e20e16c9deec8771488189277c58eabf8023cac6560 Jul 08 18:40:23.915640 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:23.915602 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-l9l9x" event={"ID":"a0618904-8d08-404a-9a82-a0bb55351381","Type":"ContainerStarted","Data":"4db1f4f9157391e9aec85c09211b872f8ec4c8cc578ad8a0807170e36d738d67"} Jul 08 18:40:23.916659 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:23.916634 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-88zv8" event={"ID":"205d4eb0-c6f3-4800-aa14-ece6eb2c362d","Type":"ContainerStarted","Data":"b79696b8d0285d0540fc5e20e16c9deec8771488189277c58eabf8023cac6560"} Jul 08 18:40:24.332187 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:24.332104 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/71020478-23b3-4f75-a526-ed0da0f0ad8f-metrics-tls\") pod \"dns-default-rdd7l\" (UID: \"71020478-23b3-4f75-a526-ed0da0f0ad8f\") " pod="openshift-dns/dns-default-rdd7l" Jul 08 18:40:24.332187 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:24.332182 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d317a3e7-440a-4808-8ce1-b748106a5614-registry-tls\") pod \"image-registry-8475db86b6-bj5jj\" (UID: \"d317a3e7-440a-4808-8ce1-b748106a5614\") " pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" Jul 08 18:40:24.332613 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:40:24.332286 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 18:40:24.332613 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:40:24.332378 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71020478-23b3-4f75-a526-ed0da0f0ad8f-metrics-tls podName:71020478-23b3-4f75-a526-ed0da0f0ad8f nodeName:}" failed. No retries permitted until 2026-07-08 18:40:56.332356772 +0000 UTC m=+99.568654843 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/71020478-23b3-4f75-a526-ed0da0f0ad8f-metrics-tls") pod "dns-default-rdd7l" (UID: "71020478-23b3-4f75-a526-ed0da0f0ad8f") : secret "dns-default-metrics-tls" not found Jul 08 18:40:24.332613 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:40:24.332296 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 18:40:24.332613 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:40:24.332408 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8475db86b6-bj5jj: secret "image-registry-tls" not found Jul 08 18:40:24.332613 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:40:24.332481 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d317a3e7-440a-4808-8ce1-b748106a5614-registry-tls podName:d317a3e7-440a-4808-8ce1-b748106a5614 nodeName:}" failed. No retries permitted until 2026-07-08 18:40:56.332450865 +0000 UTC m=+99.568748950 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d317a3e7-440a-4808-8ce1-b748106a5614-registry-tls") pod "image-registry-8475db86b6-bj5jj" (UID: "d317a3e7-440a-4808-8ce1-b748106a5614") : secret "image-registry-tls" not found Jul 08 18:40:29.933008 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:29.932969 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-l9l9x" event={"ID":"a0618904-8d08-404a-9a82-a0bb55351381","Type":"ContainerStarted","Data":"07169902f0b8a0dfed33fc01f083d64a32ab649101942f44e301a8092873baf9"} Jul 08 18:40:29.934475 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:29.934449 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-88zv8" event={"ID":"205d4eb0-c6f3-4800-aa14-ece6eb2c362d","Type":"ContainerStarted","Data":"8f11d61f5a766b152e247f6f88e4b27bf9610a0e462c5a53e9463607e23f0b94"} Jul 08 18:40:29.934595 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:29.934581 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-88zv8" Jul 08 18:40:29.954944 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:29.954888 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-l9l9x" podStartSLOduration=66.617781809 podStartE2EDuration="1m11.95487137s" totalImagesPullingTime="5.337089561s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:39:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:40:29.954628719 +0000 UTC m=+73.190926809" watchObservedRunningTime="2026-07-08 18:40:29.95487137 +0000 UTC m=+73.191169461" Jul 08 18:40:29.973521 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:29.973465 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-88zv8" podStartSLOduration=66.674176934 podStartE2EDuration="1m11.973447932s" totalImagesPullingTime="5.299270998s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:39:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:40:29.973399994 +0000 UTC m=+73.209698106" watchObservedRunningTime="2026-07-08 18:40:29.973447932 +0000 UTC m=+73.209746025" Jul 08 18:40:39.712945 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.712539 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-zzt4h"] Jul 08 18:40:39.716022 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.716006 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-g989c"] Jul 08 18:40:39.716151 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.716134 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-zzt4h" Jul 08 18:40:39.718363 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.718343 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-g989c" Jul 08 18:40:39.725931 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.725914 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 08 18:40:39.726320 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.726306 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-dmssv\"" Jul 08 18:40:39.726431 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.726414 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 08 18:40:39.751826 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.751802 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 08 18:40:39.752017 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.752000 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 08 18:40:39.753974 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.753952 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 08 18:40:39.754081 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.754029 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-bgv5d\"" Jul 08 18:40:39.754081 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.754060 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 08 18:40:39.754081 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.754070 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 08 18:40:39.754222 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.754070 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 08 18:40:39.761259 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.761238 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 08 18:40:39.770502 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.770436 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-g989c"] Jul 08 18:40:39.775712 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.772390 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-zzt4h"] Jul 08 18:40:39.776237 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.776213 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 08 18:40:39.848144 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.848104 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d94a3566-fcf3-47a1-b176-a33b71c3826d-config\") pod \"console-operator-68b757fb67-zzt4h\" (UID: \"d94a3566-fcf3-47a1-b176-a33b71c3826d\") " pod="openshift-console-operator/console-operator-68b757fb67-zzt4h" Jul 08 18:40:39.848337 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.848178 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d94a3566-fcf3-47a1-b176-a33b71c3826d-trusted-ca\") pod \"console-operator-68b757fb67-zzt4h\" (UID: \"d94a3566-fcf3-47a1-b176-a33b71c3826d\") " pod="openshift-console-operator/console-operator-68b757fb67-zzt4h" Jul 08 18:40:39.848337 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.848222 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86nfq\" (UniqueName: \"kubernetes.io/projected/61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e-kube-api-access-86nfq\") pod \"insights-operator-5c847dcfc4-g989c\" (UID: \"61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e\") " pod="openshift-insights/insights-operator-5c847dcfc4-g989c" Jul 08 18:40:39.848337 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.848245 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e-snapshots\") pod \"insights-operator-5c847dcfc4-g989c\" (UID: \"61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e\") " pod="openshift-insights/insights-operator-5c847dcfc4-g989c" Jul 08 18:40:39.848337 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.848267 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e-serving-cert\") pod \"insights-operator-5c847dcfc4-g989c\" (UID: \"61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e\") " pod="openshift-insights/insights-operator-5c847dcfc4-g989c" Jul 08 18:40:39.848468 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.848332 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d94a3566-fcf3-47a1-b176-a33b71c3826d-serving-cert\") pod \"console-operator-68b757fb67-zzt4h\" (UID: \"d94a3566-fcf3-47a1-b176-a33b71c3826d\") " pod="openshift-console-operator/console-operator-68b757fb67-zzt4h" Jul 08 18:40:39.848468 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.848356 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e-tmp\") pod \"insights-operator-5c847dcfc4-g989c\" (UID: \"61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e\") " pod="openshift-insights/insights-operator-5c847dcfc4-g989c" Jul 08 18:40:39.848468 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.848426 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-g989c\" (UID: \"61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e\") " pod="openshift-insights/insights-operator-5c847dcfc4-g989c" Jul 08 18:40:39.848468 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.848463 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-g989c\" (UID: \"61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e\") " pod="openshift-insights/insights-operator-5c847dcfc4-g989c" Jul 08 18:40:39.848588 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.848503 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6wbn\" (UniqueName: \"kubernetes.io/projected/d94a3566-fcf3-47a1-b176-a33b71c3826d-kube-api-access-l6wbn\") pod \"console-operator-68b757fb67-zzt4h\" (UID: \"d94a3566-fcf3-47a1-b176-a33b71c3826d\") " pod="openshift-console-operator/console-operator-68b757fb67-zzt4h" Jul 08 18:40:39.949380 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.949331 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d94a3566-fcf3-47a1-b176-a33b71c3826d-config\") pod \"console-operator-68b757fb67-zzt4h\" (UID: \"d94a3566-fcf3-47a1-b176-a33b71c3826d\") " pod="openshift-console-operator/console-operator-68b757fb67-zzt4h" Jul 08 18:40:39.949552 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.949505 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d94a3566-fcf3-47a1-b176-a33b71c3826d-trusted-ca\") pod \"console-operator-68b757fb67-zzt4h\" (UID: \"d94a3566-fcf3-47a1-b176-a33b71c3826d\") " pod="openshift-console-operator/console-operator-68b757fb67-zzt4h" Jul 08 18:40:39.949594 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.949560 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86nfq\" (UniqueName: \"kubernetes.io/projected/61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e-kube-api-access-86nfq\") pod \"insights-operator-5c847dcfc4-g989c\" (UID: \"61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e\") " pod="openshift-insights/insights-operator-5c847dcfc4-g989c" Jul 08 18:40:39.949628 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.949593 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e-snapshots\") pod \"insights-operator-5c847dcfc4-g989c\" (UID: \"61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e\") " pod="openshift-insights/insights-operator-5c847dcfc4-g989c" Jul 08 18:40:39.949628 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.949621 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e-serving-cert\") pod \"insights-operator-5c847dcfc4-g989c\" (UID: \"61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e\") " pod="openshift-insights/insights-operator-5c847dcfc4-g989c" Jul 08 18:40:39.949860 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.949836 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d94a3566-fcf3-47a1-b176-a33b71c3826d-serving-cert\") pod \"console-operator-68b757fb67-zzt4h\" (UID: \"d94a3566-fcf3-47a1-b176-a33b71c3826d\") " pod="openshift-console-operator/console-operator-68b757fb67-zzt4h" Jul 08 18:40:39.949939 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.949874 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e-tmp\") pod \"insights-operator-5c847dcfc4-g989c\" (UID: \"61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e\") " pod="openshift-insights/insights-operator-5c847dcfc4-g989c" Jul 08 18:40:39.949939 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.949926 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-g989c\" (UID: \"61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e\") " pod="openshift-insights/insights-operator-5c847dcfc4-g989c" Jul 08 18:40:39.950041 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.949952 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-g989c\" (UID: \"61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e\") " pod="openshift-insights/insights-operator-5c847dcfc4-g989c" Jul 08 18:40:39.950041 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.950006 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6wbn\" (UniqueName: \"kubernetes.io/projected/d94a3566-fcf3-47a1-b176-a33b71c3826d-kube-api-access-l6wbn\") pod \"console-operator-68b757fb67-zzt4h\" (UID: \"d94a3566-fcf3-47a1-b176-a33b71c3826d\") " pod="openshift-console-operator/console-operator-68b757fb67-zzt4h" Jul 08 18:40:39.950211 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.950187 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d94a3566-fcf3-47a1-b176-a33b71c3826d-config\") pod \"console-operator-68b757fb67-zzt4h\" (UID: \"d94a3566-fcf3-47a1-b176-a33b71c3826d\") " pod="openshift-console-operator/console-operator-68b757fb67-zzt4h" Jul 08 18:40:39.950366 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.950334 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e-snapshots\") pod \"insights-operator-5c847dcfc4-g989c\" (UID: \"61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e\") " pod="openshift-insights/insights-operator-5c847dcfc4-g989c" Jul 08 18:40:39.950366 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.950357 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d94a3566-fcf3-47a1-b176-a33b71c3826d-trusted-ca\") pod \"console-operator-68b757fb67-zzt4h\" (UID: \"d94a3566-fcf3-47a1-b176-a33b71c3826d\") " pod="openshift-console-operator/console-operator-68b757fb67-zzt4h" Jul 08 18:40:39.950519 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.950473 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e-tmp\") pod \"insights-operator-5c847dcfc4-g989c\" (UID: \"61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e\") " pod="openshift-insights/insights-operator-5c847dcfc4-g989c" Jul 08 18:40:39.950632 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.950609 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-g989c\" (UID: \"61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e\") " pod="openshift-insights/insights-operator-5c847dcfc4-g989c" Jul 08 18:40:39.951062 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.951039 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-g989c\" (UID: \"61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e\") " pod="openshift-insights/insights-operator-5c847dcfc4-g989c" Jul 08 18:40:39.952176 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.952155 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d94a3566-fcf3-47a1-b176-a33b71c3826d-serving-cert\") pod \"console-operator-68b757fb67-zzt4h\" (UID: \"d94a3566-fcf3-47a1-b176-a33b71c3826d\") " pod="openshift-console-operator/console-operator-68b757fb67-zzt4h" Jul 08 18:40:39.952261 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.952245 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e-serving-cert\") pod \"insights-operator-5c847dcfc4-g989c\" (UID: \"61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e\") " pod="openshift-insights/insights-operator-5c847dcfc4-g989c" Jul 08 18:40:39.958558 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.958533 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86nfq\" (UniqueName: \"kubernetes.io/projected/61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e-kube-api-access-86nfq\") pod \"insights-operator-5c847dcfc4-g989c\" (UID: \"61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e\") " pod="openshift-insights/insights-operator-5c847dcfc4-g989c" Jul 08 18:40:39.958640 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:39.958564 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6wbn\" (UniqueName: \"kubernetes.io/projected/d94a3566-fcf3-47a1-b176-a33b71c3826d-kube-api-access-l6wbn\") pod \"console-operator-68b757fb67-zzt4h\" (UID: \"d94a3566-fcf3-47a1-b176-a33b71c3826d\") " pod="openshift-console-operator/console-operator-68b757fb67-zzt4h" Jul 08 18:40:40.026195 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:40.026070 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-zzt4h" Jul 08 18:40:40.030778 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:40.030741 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-g989c" Jul 08 18:40:40.162129 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:40.162097 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-zzt4h"] Jul 08 18:40:40.165212 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:40:40.165182 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd94a3566_fcf3_47a1_b176_a33b71c3826d.slice/crio-71fd9cb139e3d642311e2a33f0786e995dd21c016323253d3d7fb743013a1ab2 WatchSource:0}: Error finding container 71fd9cb139e3d642311e2a33f0786e995dd21c016323253d3d7fb743013a1ab2: Status 404 returned error can't find the container with id 71fd9cb139e3d642311e2a33f0786e995dd21c016323253d3d7fb743013a1ab2 Jul 08 18:40:40.177920 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:40.177896 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-g989c"] Jul 08 18:40:40.181127 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:40:40.181103 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61cc5a69_f1a7_4a3a_8b33_427c95bd4a9e.slice/crio-39da16da59d651ee827ea67b031e36bdf54a8933dd699497b7d29f0cb689808b WatchSource:0}: Error finding container 39da16da59d651ee827ea67b031e36bdf54a8933dd699497b7d29f0cb689808b: Status 404 returned error can't find the container with id 39da16da59d651ee827ea67b031e36bdf54a8933dd699497b7d29f0cb689808b Jul 08 18:40:40.964320 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:40.964282 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-g989c" event={"ID":"61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e","Type":"ContainerStarted","Data":"39da16da59d651ee827ea67b031e36bdf54a8933dd699497b7d29f0cb689808b"} Jul 08 18:40:40.965587 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:40.965557 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-zzt4h" event={"ID":"d94a3566-fcf3-47a1-b176-a33b71c3826d","Type":"ContainerStarted","Data":"71fd9cb139e3d642311e2a33f0786e995dd21c016323253d3d7fb743013a1ab2"} Jul 08 18:40:43.974649 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:43.974610 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-g989c" event={"ID":"61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e","Type":"ContainerStarted","Data":"04e119e3f54ab287a51eaba575fef4ba6e8e7d083187586bde931441503f17e0"} Jul 08 18:40:43.976240 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:43.976218 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zzt4h_d94a3566-fcf3-47a1-b176-a33b71c3826d/console-operator/0.log" Jul 08 18:40:43.976360 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:43.976253 2559 generic.go:356] "Generic (PLEG): container finished" podID="d94a3566-fcf3-47a1-b176-a33b71c3826d" containerID="77e31c658f1e7b3d1ccccb365e041a76967dbd15b4e0ae60bfd86fe70a25726b" exitCode=255 Jul 08 18:40:43.976360 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:43.976289 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-zzt4h" event={"ID":"d94a3566-fcf3-47a1-b176-a33b71c3826d","Type":"ContainerDied","Data":"77e31c658f1e7b3d1ccccb365e041a76967dbd15b4e0ae60bfd86fe70a25726b"} Jul 08 18:40:43.976478 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:43.976465 2559 scope.go:117] "RemoveContainer" containerID="77e31c658f1e7b3d1ccccb365e041a76967dbd15b4e0ae60bfd86fe70a25726b" Jul 08 18:40:44.000236 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:44.000193 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-g989c" podStartSLOduration=2.051541327 podStartE2EDuration="5.000178434s" totalImagesPullingTime="2.948637107s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:40:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:40:43.999120578 +0000 UTC m=+87.235418668" watchObservedRunningTime="2026-07-08 18:40:44.000178434 +0000 UTC m=+87.236476535" Jul 08 18:40:44.503667 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:44.503630 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-56mxw"] Jul 08 18:40:44.507094 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:44.507071 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-56mxw" Jul 08 18:40:44.511242 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:44.511216 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 08 18:40:44.511242 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:44.511231 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jtszc\"" Jul 08 18:40:44.512496 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:44.512479 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 08 18:40:44.528019 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:44.527997 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-56mxw"] Jul 08 18:40:44.580087 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:44.580030 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/37037696-fffa-4456-aacb-6bc63f69254d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-56mxw\" (UID: \"37037696-fffa-4456-aacb-6bc63f69254d\") " pod="openshift-insights/insights-runtime-extractor-56mxw" Jul 08 18:40:44.580270 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:44.580101 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/37037696-fffa-4456-aacb-6bc63f69254d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-56mxw\" (UID: \"37037696-fffa-4456-aacb-6bc63f69254d\") " pod="openshift-insights/insights-runtime-extractor-56mxw" Jul 08 18:40:44.580270 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:44.580130 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvc4t\" (UniqueName: \"kubernetes.io/projected/37037696-fffa-4456-aacb-6bc63f69254d-kube-api-access-nvc4t\") pod \"insights-runtime-extractor-56mxw\" (UID: \"37037696-fffa-4456-aacb-6bc63f69254d\") " pod="openshift-insights/insights-runtime-extractor-56mxw" Jul 08 18:40:44.580270 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:44.580161 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/37037696-fffa-4456-aacb-6bc63f69254d-data-volume\") pod \"insights-runtime-extractor-56mxw\" (UID: \"37037696-fffa-4456-aacb-6bc63f69254d\") " pod="openshift-insights/insights-runtime-extractor-56mxw" Jul 08 18:40:44.580270 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:44.580240 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/37037696-fffa-4456-aacb-6bc63f69254d-crio-socket\") pod \"insights-runtime-extractor-56mxw\" (UID: \"37037696-fffa-4456-aacb-6bc63f69254d\") " pod="openshift-insights/insights-runtime-extractor-56mxw" Jul 08 18:40:44.681527 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:44.681483 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/37037696-fffa-4456-aacb-6bc63f69254d-crio-socket\") pod \"insights-runtime-extractor-56mxw\" (UID: \"37037696-fffa-4456-aacb-6bc63f69254d\") " pod="openshift-insights/insights-runtime-extractor-56mxw" Jul 08 18:40:44.681700 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:44.681560 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/37037696-fffa-4456-aacb-6bc63f69254d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-56mxw\" (UID: \"37037696-fffa-4456-aacb-6bc63f69254d\") " pod="openshift-insights/insights-runtime-extractor-56mxw" Jul 08 18:40:44.681700 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:44.681585 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/37037696-fffa-4456-aacb-6bc63f69254d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-56mxw\" (UID: \"37037696-fffa-4456-aacb-6bc63f69254d\") " pod="openshift-insights/insights-runtime-extractor-56mxw" Jul 08 18:40:44.681700 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:44.681604 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvc4t\" (UniqueName: \"kubernetes.io/projected/37037696-fffa-4456-aacb-6bc63f69254d-kube-api-access-nvc4t\") pod \"insights-runtime-extractor-56mxw\" (UID: \"37037696-fffa-4456-aacb-6bc63f69254d\") " pod="openshift-insights/insights-runtime-extractor-56mxw" Jul 08 18:40:44.681700 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:44.681629 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/37037696-fffa-4456-aacb-6bc63f69254d-data-volume\") pod \"insights-runtime-extractor-56mxw\" (UID: \"37037696-fffa-4456-aacb-6bc63f69254d\") " pod="openshift-insights/insights-runtime-extractor-56mxw" Jul 08 18:40:44.681700 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:44.681623 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/37037696-fffa-4456-aacb-6bc63f69254d-crio-socket\") pod \"insights-runtime-extractor-56mxw\" (UID: \"37037696-fffa-4456-aacb-6bc63f69254d\") " pod="openshift-insights/insights-runtime-extractor-56mxw" Jul 08 18:40:44.681700 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:40:44.681699 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 18:40:44.681993 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:40:44.681699 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 18:40:44.681993 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:40:44.681760 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37037696-fffa-4456-aacb-6bc63f69254d-insights-runtime-extractor-tls podName:37037696-fffa-4456-aacb-6bc63f69254d nodeName:}" failed. No retries permitted until 2026-07-08 18:40:45.181742958 +0000 UTC m=+88.418041035 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/37037696-fffa-4456-aacb-6bc63f69254d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-56mxw" (UID: "37037696-fffa-4456-aacb-6bc63f69254d") : secret "insights-runtime-extractor-tls" not found Jul 08 18:40:44.681993 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:40:44.681834 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/37037696-fffa-4456-aacb-6bc63f69254d-kube-rbac-proxy-cm podName:37037696-fffa-4456-aacb-6bc63f69254d nodeName:}" failed. No retries permitted until 2026-07-08 18:40:45.181815787 +0000 UTC m=+88.418113861 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/37037696-fffa-4456-aacb-6bc63f69254d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-56mxw" (UID: "37037696-fffa-4456-aacb-6bc63f69254d") : configmap "kube-rbac-proxy" not found Jul 08 18:40:44.681993 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:44.681987 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/37037696-fffa-4456-aacb-6bc63f69254d-data-volume\") pod \"insights-runtime-extractor-56mxw\" (UID: \"37037696-fffa-4456-aacb-6bc63f69254d\") " pod="openshift-insights/insights-runtime-extractor-56mxw" Jul 08 18:40:44.693965 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:44.693943 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvc4t\" (UniqueName: \"kubernetes.io/projected/37037696-fffa-4456-aacb-6bc63f69254d-kube-api-access-nvc4t\") pod \"insights-runtime-extractor-56mxw\" (UID: \"37037696-fffa-4456-aacb-6bc63f69254d\") " pod="openshift-insights/insights-runtime-extractor-56mxw" Jul 08 18:40:44.981679 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:44.981647 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zzt4h_d94a3566-fcf3-47a1-b176-a33b71c3826d/console-operator/1.log" Jul 08 18:40:44.982084 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:44.982054 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zzt4h_d94a3566-fcf3-47a1-b176-a33b71c3826d/console-operator/0.log" Jul 08 18:40:44.982137 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:44.982084 2559 generic.go:356] "Generic (PLEG): container finished" podID="d94a3566-fcf3-47a1-b176-a33b71c3826d" containerID="1d151f111ed037d27bf998c35fc74fb6c15350ffd8c583eb9570e0c76e052764" exitCode=255 Jul 08 18:40:44.982172 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:44.982162 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-zzt4h" event={"ID":"d94a3566-fcf3-47a1-b176-a33b71c3826d","Type":"ContainerDied","Data":"1d151f111ed037d27bf998c35fc74fb6c15350ffd8c583eb9570e0c76e052764"} Jul 08 18:40:44.982211 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:44.982201 2559 scope.go:117] "RemoveContainer" containerID="77e31c658f1e7b3d1ccccb365e041a76967dbd15b4e0ae60bfd86fe70a25726b" Jul 08 18:40:44.982399 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:44.982377 2559 scope.go:117] "RemoveContainer" containerID="1d151f111ed037d27bf998c35fc74fb6c15350ffd8c583eb9570e0c76e052764" Jul 08 18:40:44.982595 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:40:44.982575 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-zzt4h_openshift-console-operator(d94a3566-fcf3-47a1-b176-a33b71c3826d)\"" pod="openshift-console-operator/console-operator-68b757fb67-zzt4h" podUID="d94a3566-fcf3-47a1-b176-a33b71c3826d" Jul 08 18:40:45.185216 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:45.185181 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/37037696-fffa-4456-aacb-6bc63f69254d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-56mxw\" (UID: \"37037696-fffa-4456-aacb-6bc63f69254d\") " pod="openshift-insights/insights-runtime-extractor-56mxw" Jul 08 18:40:45.185216 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:45.185224 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/37037696-fffa-4456-aacb-6bc63f69254d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-56mxw\" (UID: \"37037696-fffa-4456-aacb-6bc63f69254d\") " pod="openshift-insights/insights-runtime-extractor-56mxw" Jul 08 18:40:45.185439 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:40:45.185317 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 18:40:45.185439 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:40:45.185326 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 18:40:45.185439 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:40:45.185368 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/37037696-fffa-4456-aacb-6bc63f69254d-kube-rbac-proxy-cm podName:37037696-fffa-4456-aacb-6bc63f69254d nodeName:}" failed. No retries permitted until 2026-07-08 18:40:46.185355518 +0000 UTC m=+89.421653586 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/37037696-fffa-4456-aacb-6bc63f69254d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-56mxw" (UID: "37037696-fffa-4456-aacb-6bc63f69254d") : configmap "kube-rbac-proxy" not found Jul 08 18:40:45.185439 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:40:45.185382 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37037696-fffa-4456-aacb-6bc63f69254d-insights-runtime-extractor-tls podName:37037696-fffa-4456-aacb-6bc63f69254d nodeName:}" failed. No retries permitted until 2026-07-08 18:40:46.185376431 +0000 UTC m=+89.421674499 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/37037696-fffa-4456-aacb-6bc63f69254d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-56mxw" (UID: "37037696-fffa-4456-aacb-6bc63f69254d") : secret "insights-runtime-extractor-tls" not found Jul 08 18:40:45.372290 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:45.372260 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-mbchd"] Jul 08 18:40:45.376421 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:45.376403 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-mbchd" Jul 08 18:40:45.379562 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:45.379537 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 08 18:40:45.379681 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:45.379608 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 08 18:40:45.379943 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:45.379923 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-v5qps\"" Jul 08 18:40:45.386873 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:45.386852 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bx5wp\" (UniqueName: \"kubernetes.io/projected/d2635c4f-2a5e-4eda-b367-77094e7f9573-kube-api-access-bx5wp\") pod \"migrator-d57d58cc9-mbchd\" (UID: \"d2635c4f-2a5e-4eda-b367-77094e7f9573\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-mbchd" Jul 08 18:40:45.390306 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:45.390279 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-mbchd"] Jul 08 18:40:45.487787 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:45.487734 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bx5wp\" (UniqueName: \"kubernetes.io/projected/d2635c4f-2a5e-4eda-b367-77094e7f9573-kube-api-access-bx5wp\") pod \"migrator-d57d58cc9-mbchd\" (UID: \"d2635c4f-2a5e-4eda-b367-77094e7f9573\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-mbchd" Jul 08 18:40:45.505666 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:45.505631 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bx5wp\" (UniqueName: \"kubernetes.io/projected/d2635c4f-2a5e-4eda-b367-77094e7f9573-kube-api-access-bx5wp\") pod \"migrator-d57d58cc9-mbchd\" (UID: \"d2635c4f-2a5e-4eda-b367-77094e7f9573\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-mbchd" Jul 08 18:40:45.685039 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:45.684964 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-mbchd" Jul 08 18:40:45.805003 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:45.804971 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-mbchd"] Jul 08 18:40:45.807931 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:40:45.807901 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2635c4f_2a5e_4eda_b367_77094e7f9573.slice/crio-6bf1ed71883980f6c59e01bb042f0dfdd3e0019478e577e99fa2c474278540cc WatchSource:0}: Error finding container 6bf1ed71883980f6c59e01bb042f0dfdd3e0019478e577e99fa2c474278540cc: Status 404 returned error can't find the container with id 6bf1ed71883980f6c59e01bb042f0dfdd3e0019478e577e99fa2c474278540cc Jul 08 18:40:45.987202 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:45.987125 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zzt4h_d94a3566-fcf3-47a1-b176-a33b71c3826d/console-operator/1.log" Jul 08 18:40:45.987661 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:45.987543 2559 scope.go:117] "RemoveContainer" containerID="1d151f111ed037d27bf998c35fc74fb6c15350ffd8c583eb9570e0c76e052764" Jul 08 18:40:45.987749 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:40:45.987724 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-zzt4h_openshift-console-operator(d94a3566-fcf3-47a1-b176-a33b71c3826d)\"" pod="openshift-console-operator/console-operator-68b757fb67-zzt4h" podUID="d94a3566-fcf3-47a1-b176-a33b71c3826d" Jul 08 18:40:45.988253 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:45.988230 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-mbchd" event={"ID":"d2635c4f-2a5e-4eda-b367-77094e7f9573","Type":"ContainerStarted","Data":"6bf1ed71883980f6c59e01bb042f0dfdd3e0019478e577e99fa2c474278540cc"} Jul 08 18:40:46.192228 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:46.192187 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/37037696-fffa-4456-aacb-6bc63f69254d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-56mxw\" (UID: \"37037696-fffa-4456-aacb-6bc63f69254d\") " pod="openshift-insights/insights-runtime-extractor-56mxw" Jul 08 18:40:46.192401 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:46.192257 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/37037696-fffa-4456-aacb-6bc63f69254d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-56mxw\" (UID: \"37037696-fffa-4456-aacb-6bc63f69254d\") " pod="openshift-insights/insights-runtime-extractor-56mxw" Jul 08 18:40:46.192401 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:40:46.192339 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 18:40:46.192401 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:40:46.192401 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37037696-fffa-4456-aacb-6bc63f69254d-insights-runtime-extractor-tls podName:37037696-fffa-4456-aacb-6bc63f69254d nodeName:}" failed. No retries permitted until 2026-07-08 18:40:48.19238547 +0000 UTC m=+91.428683541 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/37037696-fffa-4456-aacb-6bc63f69254d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-56mxw" (UID: "37037696-fffa-4456-aacb-6bc63f69254d") : secret "insights-runtime-extractor-tls" not found Jul 08 18:40:46.192509 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:40:46.192339 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 18:40:46.192509 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:40:46.192471 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/37037696-fffa-4456-aacb-6bc63f69254d-kube-rbac-proxy-cm podName:37037696-fffa-4456-aacb-6bc63f69254d nodeName:}" failed. No retries permitted until 2026-07-08 18:40:48.192459379 +0000 UTC m=+91.428757461 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/37037696-fffa-4456-aacb-6bc63f69254d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-56mxw" (UID: "37037696-fffa-4456-aacb-6bc63f69254d") : configmap "kube-rbac-proxy" not found Jul 08 18:40:46.693756 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:46.693726 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-x72mc_32185124-ea41-4309-9674-13230b4d7a6e/dns-node-resolver/0.log" Jul 08 18:40:47.694003 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:47.693973 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-q9hp4_ceb0a0a0-5ba1-4906-b214-5b1de0abbec3/node-ca/0.log" Jul 08 18:40:48.206648 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:48.206602 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/37037696-fffa-4456-aacb-6bc63f69254d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-56mxw\" (UID: \"37037696-fffa-4456-aacb-6bc63f69254d\") " pod="openshift-insights/insights-runtime-extractor-56mxw" Jul 08 18:40:48.206648 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:48.206654 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/37037696-fffa-4456-aacb-6bc63f69254d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-56mxw\" (UID: \"37037696-fffa-4456-aacb-6bc63f69254d\") " pod="openshift-insights/insights-runtime-extractor-56mxw" Jul 08 18:40:48.206867 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:40:48.206740 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 18:40:48.206867 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:40:48.206802 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 18:40:48.206867 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:40:48.206818 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37037696-fffa-4456-aacb-6bc63f69254d-insights-runtime-extractor-tls podName:37037696-fffa-4456-aacb-6bc63f69254d nodeName:}" failed. No retries permitted until 2026-07-08 18:40:52.206803741 +0000 UTC m=+95.443101808 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/37037696-fffa-4456-aacb-6bc63f69254d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-56mxw" (UID: "37037696-fffa-4456-aacb-6bc63f69254d") : secret "insights-runtime-extractor-tls" not found Jul 08 18:40:48.206867 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:40:48.206856 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/37037696-fffa-4456-aacb-6bc63f69254d-kube-rbac-proxy-cm podName:37037696-fffa-4456-aacb-6bc63f69254d nodeName:}" failed. No retries permitted until 2026-07-08 18:40:52.206844155 +0000 UTC m=+95.443142223 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/37037696-fffa-4456-aacb-6bc63f69254d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-56mxw" (UID: "37037696-fffa-4456-aacb-6bc63f69254d") : configmap "kube-rbac-proxy" not found Jul 08 18:40:48.295468 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:48.295385 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9qx6k_cc461fee-bcf4-4a16-92ec-50211c9314c6/serve-healthcheck-canary/0.log" Jul 08 18:40:48.999385 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:48.999343 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-mbchd" event={"ID":"d2635c4f-2a5e-4eda-b367-77094e7f9573","Type":"ContainerStarted","Data":"fe53916a46fcfb426fbd772b95778c16825f9815dd066abdad595becfd1a8bdd"} Jul 08 18:40:48.999757 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:48.999411 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-mbchd" event={"ID":"d2635c4f-2a5e-4eda-b367-77094e7f9573","Type":"ContainerStarted","Data":"847a7605738296258d038ceb6308a8f22ad7f0426a5b7524987ee220cdcc62f5"} Jul 08 18:40:49.021811 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:49.021741 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-mbchd" podStartSLOduration=1.796822331 podStartE2EDuration="4.021726041s" totalImagesPullingTime="2.22490371s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:40:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:40:49.02048989 +0000 UTC m=+92.256787982" watchObservedRunningTime="2026-07-08 18:40:49.021726041 +0000 UTC m=+92.258024130" Jul 08 18:40:50.026944 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:50.026913 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-zzt4h" Jul 08 18:40:50.027311 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:50.026971 2559 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-zzt4h" Jul 08 18:40:50.027311 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:50.027281 2559 scope.go:117] "RemoveContainer" containerID="1d151f111ed037d27bf998c35fc74fb6c15350ffd8c583eb9570e0c76e052764" Jul 08 18:40:50.027442 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:40:50.027415 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-zzt4h_openshift-console-operator(d94a3566-fcf3-47a1-b176-a33b71c3826d)\"" pod="openshift-console-operator/console-operator-68b757fb67-zzt4h" podUID="d94a3566-fcf3-47a1-b176-a33b71c3826d" Jul 08 18:40:52.239384 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:52.239325 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/37037696-fffa-4456-aacb-6bc63f69254d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-56mxw\" (UID: \"37037696-fffa-4456-aacb-6bc63f69254d\") " pod="openshift-insights/insights-runtime-extractor-56mxw" Jul 08 18:40:52.239384 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:52.239386 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/37037696-fffa-4456-aacb-6bc63f69254d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-56mxw\" (UID: \"37037696-fffa-4456-aacb-6bc63f69254d\") " pod="openshift-insights/insights-runtime-extractor-56mxw" Jul 08 18:40:52.239859 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:40:52.239482 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 18:40:52.239859 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:40:52.239484 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 18:40:52.239859 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:40:52.239536 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/37037696-fffa-4456-aacb-6bc63f69254d-kube-rbac-proxy-cm podName:37037696-fffa-4456-aacb-6bc63f69254d nodeName:}" failed. No retries permitted until 2026-07-08 18:41:00.239523808 +0000 UTC m=+103.475821876 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/37037696-fffa-4456-aacb-6bc63f69254d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-56mxw" (UID: "37037696-fffa-4456-aacb-6bc63f69254d") : configmap "kube-rbac-proxy" not found Jul 08 18:40:52.239859 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:40:52.239550 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37037696-fffa-4456-aacb-6bc63f69254d-insights-runtime-extractor-tls podName:37037696-fffa-4456-aacb-6bc63f69254d nodeName:}" failed. No retries permitted until 2026-07-08 18:41:00.239543608 +0000 UTC m=+103.475841676 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/37037696-fffa-4456-aacb-6bc63f69254d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-56mxw" (UID: "37037696-fffa-4456-aacb-6bc63f69254d") : secret "insights-runtime-extractor-tls" not found Jul 08 18:40:56.372698 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:56.372649 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/71020478-23b3-4f75-a526-ed0da0f0ad8f-metrics-tls\") pod \"dns-default-rdd7l\" (UID: \"71020478-23b3-4f75-a526-ed0da0f0ad8f\") " pod="openshift-dns/dns-default-rdd7l" Jul 08 18:40:56.373204 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:56.372750 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d317a3e7-440a-4808-8ce1-b748106a5614-registry-tls\") pod \"image-registry-8475db86b6-bj5jj\" (UID: \"d317a3e7-440a-4808-8ce1-b748106a5614\") " pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" Jul 08 18:40:56.375093 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:56.375060 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/71020478-23b3-4f75-a526-ed0da0f0ad8f-metrics-tls\") pod \"dns-default-rdd7l\" (UID: \"71020478-23b3-4f75-a526-ed0da0f0ad8f\") " pod="openshift-dns/dns-default-rdd7l" Jul 08 18:40:56.375093 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:56.375087 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d317a3e7-440a-4808-8ce1-b748106a5614-registry-tls\") pod \"image-registry-8475db86b6-bj5jj\" (UID: \"d317a3e7-440a-4808-8ce1-b748106a5614\") " pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" Jul 08 18:40:56.403723 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:56.400968 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-n6k5m\"" Jul 08 18:40:56.408071 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:56.408049 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" Jul 08 18:40:56.444960 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:56.444932 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-rmskx\"" Jul 08 18:40:56.453322 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:56.453290 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rdd7l" Jul 08 18:40:56.548355 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:56.548327 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8475db86b6-bj5jj"] Jul 08 18:40:56.550951 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:40:56.550927 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd317a3e7_440a_4808_8ce1_b748106a5614.slice/crio-c725c86be1bb19f5f2573aa4974039f166bf52e06579decc18d2c5b8947c720e WatchSource:0}: Error finding container c725c86be1bb19f5f2573aa4974039f166bf52e06579decc18d2c5b8947c720e: Status 404 returned error can't find the container with id c725c86be1bb19f5f2573aa4974039f166bf52e06579decc18d2c5b8947c720e Jul 08 18:40:56.589959 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:56.589932 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rdd7l"] Jul 08 18:40:56.592748 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:40:56.592721 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71020478_23b3_4f75_a526_ed0da0f0ad8f.slice/crio-964fa7dc433533a95e18ff2180d070730ae22ee2fe56c0bcda4413cd23a359c0 WatchSource:0}: Error finding container 964fa7dc433533a95e18ff2180d070730ae22ee2fe56c0bcda4413cd23a359c0: Status 404 returned error can't find the container with id 964fa7dc433533a95e18ff2180d070730ae22ee2fe56c0bcda4413cd23a359c0 Jul 08 18:40:57.024622 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:57.024586 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" event={"ID":"d317a3e7-440a-4808-8ce1-b748106a5614","Type":"ContainerStarted","Data":"df56974a818e07f0135b3d43d0d30aa03aed3fe9c28995cf503a7866b88ecaca"} Jul 08 18:40:57.024830 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:57.024644 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" event={"ID":"d317a3e7-440a-4808-8ce1-b748106a5614","Type":"ContainerStarted","Data":"c725c86be1bb19f5f2573aa4974039f166bf52e06579decc18d2c5b8947c720e"} Jul 08 18:40:57.024830 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:57.024671 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" Jul 08 18:40:57.025747 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:40:57.025717 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rdd7l" event={"ID":"71020478-23b3-4f75-a526-ed0da0f0ad8f","Type":"ContainerStarted","Data":"964fa7dc433533a95e18ff2180d070730ae22ee2fe56c0bcda4413cd23a359c0"} Jul 08 18:41:00.036074 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:00.036039 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rdd7l" event={"ID":"71020478-23b3-4f75-a526-ed0da0f0ad8f","Type":"ContainerStarted","Data":"471fb4b5c1dac350808af9cfaed7bd91ca6c8cb4d22b8d1dcc1001c046bc7285"} Jul 08 18:41:00.036471 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:00.036120 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rdd7l" event={"ID":"71020478-23b3-4f75-a526-ed0da0f0ad8f","Type":"ContainerStarted","Data":"8bab0cc79637ff36e9f0af6fc2501ca0e4398e611ba4af2c185cb0290aaef9a6"} Jul 08 18:41:00.036471 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:00.036191 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-rdd7l" Jul 08 18:41:00.056609 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:00.056558 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" podStartSLOduration=102.05654269 podStartE2EDuration="1m42.05654269s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:39:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:40:57.049256371 +0000 UTC m=+100.285554463" watchObservedRunningTime="2026-07-08 18:41:00.05654269 +0000 UTC m=+103.292840781" Jul 08 18:41:00.056965 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:00.056940 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-rdd7l" podStartSLOduration=64.975143887 podStartE2EDuration="1m8.056931638s" totalImagesPullingTime="3.081787751s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:39:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:41:00.056030382 +0000 UTC m=+103.292328472" watchObservedRunningTime="2026-07-08 18:41:00.056931638 +0000 UTC m=+103.293229727" Jul 08 18:41:00.304497 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:00.304402 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/37037696-fffa-4456-aacb-6bc63f69254d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-56mxw\" (UID: \"37037696-fffa-4456-aacb-6bc63f69254d\") " pod="openshift-insights/insights-runtime-extractor-56mxw" Jul 08 18:41:00.304497 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:00.304443 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/37037696-fffa-4456-aacb-6bc63f69254d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-56mxw\" (UID: \"37037696-fffa-4456-aacb-6bc63f69254d\") " pod="openshift-insights/insights-runtime-extractor-56mxw" Jul 08 18:41:00.305044 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:00.305021 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/37037696-fffa-4456-aacb-6bc63f69254d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-56mxw\" (UID: \"37037696-fffa-4456-aacb-6bc63f69254d\") " pod="openshift-insights/insights-runtime-extractor-56mxw" Jul 08 18:41:00.306719 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:00.306700 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/37037696-fffa-4456-aacb-6bc63f69254d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-56mxw\" (UID: \"37037696-fffa-4456-aacb-6bc63f69254d\") " pod="openshift-insights/insights-runtime-extractor-56mxw" Jul 08 18:41:00.415699 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:00.415660 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-56mxw" Jul 08 18:41:00.596414 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:00.596333 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-56mxw"] Jul 08 18:41:00.599128 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:41:00.599098 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37037696_fffa_4456_aacb_6bc63f69254d.slice/crio-d836eea9a1f1fef30c9a8aa78e0da4644a0b0f3f8c46313bc011037ab5d86b34 WatchSource:0}: Error finding container d836eea9a1f1fef30c9a8aa78e0da4644a0b0f3f8c46313bc011037ab5d86b34: Status 404 returned error can't find the container with id d836eea9a1f1fef30c9a8aa78e0da4644a0b0f3f8c46313bc011037ab5d86b34 Jul 08 18:41:00.941179 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:00.941145 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-88zv8" Jul 08 18:41:01.039826 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:01.039793 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-56mxw" event={"ID":"37037696-fffa-4456-aacb-6bc63f69254d","Type":"ContainerStarted","Data":"d836eea9a1f1fef30c9a8aa78e0da4644a0b0f3f8c46313bc011037ab5d86b34"} Jul 08 18:41:03.682245 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:03.682216 2559 scope.go:117] "RemoveContainer" containerID="1d151f111ed037d27bf998c35fc74fb6c15350ffd8c583eb9570e0c76e052764" Jul 08 18:41:05.056722 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:05.056689 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-56mxw" event={"ID":"37037696-fffa-4456-aacb-6bc63f69254d","Type":"ContainerStarted","Data":"141844b829647e7ad1f5e54d78994935cc24fba765359103507f211a6b020ad6"} Jul 08 18:41:05.058430 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:05.058407 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zzt4h_d94a3566-fcf3-47a1-b176-a33b71c3826d/console-operator/1.log" Jul 08 18:41:05.058542 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:05.058470 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-zzt4h" event={"ID":"d94a3566-fcf3-47a1-b176-a33b71c3826d","Type":"ContainerStarted","Data":"bfefe0bb96d14336d43d2889faf56f770c3dbd3889bc4676b0c733d1ef21d8d3"} Jul 08 18:41:05.058734 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:05.058718 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-zzt4h" Jul 08 18:41:05.080550 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:05.078591 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-zzt4h" podStartSLOduration=23.117428746 podStartE2EDuration="26.078564798s" totalImagesPullingTime="2.961136052s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:40:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:41:05.076147858 +0000 UTC m=+108.312445946" watchObservedRunningTime="2026-07-08 18:41:05.078564798 +0000 UTC m=+108.314862888" Jul 08 18:41:05.520805 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:05.520756 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-zzt4h" Jul 08 18:41:06.064718 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:06.064681 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-56mxw" event={"ID":"37037696-fffa-4456-aacb-6bc63f69254d","Type":"ContainerStarted","Data":"962196922a4fd25db57b0ce5cf7e1050f46192933ff926ca8bf090d729a7464e"} Jul 08 18:41:08.075553 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:08.075519 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-56mxw" event={"ID":"37037696-fffa-4456-aacb-6bc63f69254d","Type":"ContainerStarted","Data":"2542852250ac51e45b553d38dd0666c00d0bcde8c21c40dca4a05d7148c52ec5"} Jul 08 18:41:08.097216 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:08.097127 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-56mxw" podStartSLOduration=16.993177394 podStartE2EDuration="24.097112114s" totalImagesPullingTime="7.10393472s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:40:44 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:41:08.095431821 +0000 UTC m=+111.331729921" watchObservedRunningTime="2026-07-08 18:41:08.097112114 +0000 UTC m=+111.333410204" Jul 08 18:41:10.042564 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:10.042530 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-rdd7l" Jul 08 18:41:10.475170 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:10.475137 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-7dwmw"] Jul 08 18:41:10.478026 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:10.478009 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-7dwmw" Jul 08 18:41:10.483255 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:10.483216 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-lfz2x\"" Jul 08 18:41:10.483255 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:10.483234 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 08 18:41:10.483478 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:10.483273 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 08 18:41:10.496741 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:10.496715 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-7dwmw"] Jul 08 18:41:10.594184 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:10.594143 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggqp8\" (UniqueName: \"kubernetes.io/projected/59decd4d-8b27-4cc1-bb59-a9c9dc29f8e3-kube-api-access-ggqp8\") pod \"downloads-685bfb5f64-7dwmw\" (UID: \"59decd4d-8b27-4cc1-bb59-a9c9dc29f8e3\") " pod="openshift-console/downloads-685bfb5f64-7dwmw" Jul 08 18:41:10.695380 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:10.695340 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggqp8\" (UniqueName: \"kubernetes.io/projected/59decd4d-8b27-4cc1-bb59-a9c9dc29f8e3-kube-api-access-ggqp8\") pod \"downloads-685bfb5f64-7dwmw\" (UID: \"59decd4d-8b27-4cc1-bb59-a9c9dc29f8e3\") " pod="openshift-console/downloads-685bfb5f64-7dwmw" Jul 08 18:41:10.715514 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:10.715493 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggqp8\" (UniqueName: \"kubernetes.io/projected/59decd4d-8b27-4cc1-bb59-a9c9dc29f8e3-kube-api-access-ggqp8\") pod \"downloads-685bfb5f64-7dwmw\" (UID: \"59decd4d-8b27-4cc1-bb59-a9c9dc29f8e3\") " pod="openshift-console/downloads-685bfb5f64-7dwmw" Jul 08 18:41:10.786686 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:10.786572 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-7dwmw" Jul 08 18:41:10.927289 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:10.927266 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-7dwmw"] Jul 08 18:41:10.929189 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:41:10.929167 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59decd4d_8b27_4cc1_bb59_a9c9dc29f8e3.slice/crio-bb47ec5ab6415161230d04bac6f81c134356d467c9ead6aad58cacecd1e311a6 WatchSource:0}: Error finding container bb47ec5ab6415161230d04bac6f81c134356d467c9ead6aad58cacecd1e311a6: Status 404 returned error can't find the container with id bb47ec5ab6415161230d04bac6f81c134356d467c9ead6aad58cacecd1e311a6 Jul 08 18:41:11.086542 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:11.086459 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-7dwmw" event={"ID":"59decd4d-8b27-4cc1-bb59-a9c9dc29f8e3","Type":"ContainerStarted","Data":"bb47ec5ab6415161230d04bac6f81c134356d467c9ead6aad58cacecd1e311a6"} Jul 08 18:41:15.470324 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:15.470286 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-4cdpm"] Jul 08 18:41:15.474863 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:15.474843 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-4cdpm" Jul 08 18:41:15.477672 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:15.477644 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-dlzpg\"" Jul 08 18:41:15.477812 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:15.477650 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 08 18:41:15.483586 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:15.483561 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-4cdpm"] Jul 08 18:41:15.634191 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:15.634124 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6c5ec41b-bde9-47f9-963e-c454166a447e-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-4cdpm\" (UID: \"6c5ec41b-bde9-47f9-963e-c454166a447e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-4cdpm" Jul 08 18:41:15.735109 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:15.735011 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6c5ec41b-bde9-47f9-963e-c454166a447e-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-4cdpm\" (UID: \"6c5ec41b-bde9-47f9-963e-c454166a447e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-4cdpm" Jul 08 18:41:15.737817 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:15.737796 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6c5ec41b-bde9-47f9-963e-c454166a447e-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-4cdpm\" (UID: \"6c5ec41b-bde9-47f9-963e-c454166a447e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-4cdpm" Jul 08 18:41:15.786059 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:15.786019 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-4cdpm" Jul 08 18:41:15.823093 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:15.823063 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6bccf57766-gxkvr"] Jul 08 18:41:15.831463 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:15.831379 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bccf57766-gxkvr" Jul 08 18:41:15.838057 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:15.838028 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 08 18:41:15.838258 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:15.838037 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 08 18:41:15.838258 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:15.838117 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-mvf96\"" Jul 08 18:41:15.838258 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:15.838191 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 08 18:41:15.838468 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:15.838368 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 08 18:41:15.839484 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:15.838684 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 08 18:41:15.841572 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:15.841516 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bccf57766-gxkvr"] Jul 08 18:41:15.936401 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:15.936361 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3fa525fd-fa8e-4574-9203-eae015e8459f-console-config\") pod \"console-6bccf57766-gxkvr\" (UID: \"3fa525fd-fa8e-4574-9203-eae015e8459f\") " pod="openshift-console/console-6bccf57766-gxkvr" Jul 08 18:41:15.936401 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:15.936400 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3fa525fd-fa8e-4574-9203-eae015e8459f-console-oauth-config\") pod \"console-6bccf57766-gxkvr\" (UID: \"3fa525fd-fa8e-4574-9203-eae015e8459f\") " pod="openshift-console/console-6bccf57766-gxkvr" Jul 08 18:41:15.936616 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:15.936436 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3fa525fd-fa8e-4574-9203-eae015e8459f-oauth-serving-cert\") pod \"console-6bccf57766-gxkvr\" (UID: \"3fa525fd-fa8e-4574-9203-eae015e8459f\") " pod="openshift-console/console-6bccf57766-gxkvr" Jul 08 18:41:15.936616 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:15.936552 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3fa525fd-fa8e-4574-9203-eae015e8459f-service-ca\") pod \"console-6bccf57766-gxkvr\" (UID: \"3fa525fd-fa8e-4574-9203-eae015e8459f\") " pod="openshift-console/console-6bccf57766-gxkvr" Jul 08 18:41:15.936616 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:15.936595 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtlt4\" (UniqueName: \"kubernetes.io/projected/3fa525fd-fa8e-4574-9203-eae015e8459f-kube-api-access-qtlt4\") pod \"console-6bccf57766-gxkvr\" (UID: \"3fa525fd-fa8e-4574-9203-eae015e8459f\") " pod="openshift-console/console-6bccf57766-gxkvr" Jul 08 18:41:15.936749 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:15.936639 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3fa525fd-fa8e-4574-9203-eae015e8459f-console-serving-cert\") pod \"console-6bccf57766-gxkvr\" (UID: \"3fa525fd-fa8e-4574-9203-eae015e8459f\") " pod="openshift-console/console-6bccf57766-gxkvr" Jul 08 18:41:15.943738 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:15.943643 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-4cdpm"] Jul 08 18:41:15.946478 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:41:15.946450 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c5ec41b_bde9_47f9_963e_c454166a447e.slice/crio-5028227fb4e1a320fc7d4de2359033ba28c1100a33b89afaabbdb8b7939a9667 WatchSource:0}: Error finding container 5028227fb4e1a320fc7d4de2359033ba28c1100a33b89afaabbdb8b7939a9667: Status 404 returned error can't find the container with id 5028227fb4e1a320fc7d4de2359033ba28c1100a33b89afaabbdb8b7939a9667 Jul 08 18:41:16.038134 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:16.038039 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3fa525fd-fa8e-4574-9203-eae015e8459f-service-ca\") pod \"console-6bccf57766-gxkvr\" (UID: \"3fa525fd-fa8e-4574-9203-eae015e8459f\") " pod="openshift-console/console-6bccf57766-gxkvr" Jul 08 18:41:16.038134 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:16.038103 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtlt4\" (UniqueName: \"kubernetes.io/projected/3fa525fd-fa8e-4574-9203-eae015e8459f-kube-api-access-qtlt4\") pod \"console-6bccf57766-gxkvr\" (UID: \"3fa525fd-fa8e-4574-9203-eae015e8459f\") " pod="openshift-console/console-6bccf57766-gxkvr" Jul 08 18:41:16.038366 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:16.038158 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3fa525fd-fa8e-4574-9203-eae015e8459f-console-serving-cert\") pod \"console-6bccf57766-gxkvr\" (UID: \"3fa525fd-fa8e-4574-9203-eae015e8459f\") " pod="openshift-console/console-6bccf57766-gxkvr" Jul 08 18:41:16.038366 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:16.038190 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3fa525fd-fa8e-4574-9203-eae015e8459f-console-config\") pod \"console-6bccf57766-gxkvr\" (UID: \"3fa525fd-fa8e-4574-9203-eae015e8459f\") " pod="openshift-console/console-6bccf57766-gxkvr" Jul 08 18:41:16.038366 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:16.038222 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3fa525fd-fa8e-4574-9203-eae015e8459f-console-oauth-config\") pod \"console-6bccf57766-gxkvr\" (UID: \"3fa525fd-fa8e-4574-9203-eae015e8459f\") " pod="openshift-console/console-6bccf57766-gxkvr" Jul 08 18:41:16.038366 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:16.038263 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3fa525fd-fa8e-4574-9203-eae015e8459f-oauth-serving-cert\") pod \"console-6bccf57766-gxkvr\" (UID: \"3fa525fd-fa8e-4574-9203-eae015e8459f\") " pod="openshift-console/console-6bccf57766-gxkvr" Jul 08 18:41:16.038975 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:16.038932 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3fa525fd-fa8e-4574-9203-eae015e8459f-service-ca\") pod \"console-6bccf57766-gxkvr\" (UID: \"3fa525fd-fa8e-4574-9203-eae015e8459f\") " pod="openshift-console/console-6bccf57766-gxkvr" Jul 08 18:41:16.038975 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:16.038970 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3fa525fd-fa8e-4574-9203-eae015e8459f-console-config\") pod \"console-6bccf57766-gxkvr\" (UID: \"3fa525fd-fa8e-4574-9203-eae015e8459f\") " pod="openshift-console/console-6bccf57766-gxkvr" Jul 08 18:41:16.039225 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:16.039204 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3fa525fd-fa8e-4574-9203-eae015e8459f-oauth-serving-cert\") pod \"console-6bccf57766-gxkvr\" (UID: \"3fa525fd-fa8e-4574-9203-eae015e8459f\") " pod="openshift-console/console-6bccf57766-gxkvr" Jul 08 18:41:16.041069 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:16.041047 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3fa525fd-fa8e-4574-9203-eae015e8459f-console-oauth-config\") pod \"console-6bccf57766-gxkvr\" (UID: \"3fa525fd-fa8e-4574-9203-eae015e8459f\") " pod="openshift-console/console-6bccf57766-gxkvr" Jul 08 18:41:16.041147 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:16.041105 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3fa525fd-fa8e-4574-9203-eae015e8459f-console-serving-cert\") pod \"console-6bccf57766-gxkvr\" (UID: \"3fa525fd-fa8e-4574-9203-eae015e8459f\") " pod="openshift-console/console-6bccf57766-gxkvr" Jul 08 18:41:16.057827 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:16.057798 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtlt4\" (UniqueName: \"kubernetes.io/projected/3fa525fd-fa8e-4574-9203-eae015e8459f-kube-api-access-qtlt4\") pod \"console-6bccf57766-gxkvr\" (UID: \"3fa525fd-fa8e-4574-9203-eae015e8459f\") " pod="openshift-console/console-6bccf57766-gxkvr" Jul 08 18:41:16.104250 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:16.104203 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-4cdpm" event={"ID":"6c5ec41b-bde9-47f9-963e-c454166a447e","Type":"ContainerStarted","Data":"5028227fb4e1a320fc7d4de2359033ba28c1100a33b89afaabbdb8b7939a9667"} Jul 08 18:41:16.143475 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:16.143434 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bccf57766-gxkvr" Jul 08 18:41:16.279858 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:16.279828 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bccf57766-gxkvr"] Jul 08 18:41:16.284078 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:41:16.284051 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fa525fd_fa8e_4574_9203_eae015e8459f.slice/crio-e17b6a3f7c9036c02e21d91a3fbb68a340c61a229aebc22eefbeb291d9d4af87 WatchSource:0}: Error finding container e17b6a3f7c9036c02e21d91a3fbb68a340c61a229aebc22eefbeb291d9d4af87: Status 404 returned error can't find the container with id e17b6a3f7c9036c02e21d91a3fbb68a340c61a229aebc22eefbeb291d9d4af87 Jul 08 18:41:16.413817 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:16.413753 2559 patch_prober.go:28] interesting pod/image-registry-8475db86b6-bj5jj container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 08 18:41:16.413977 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:16.413836 2559 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" podUID="d317a3e7-440a-4808-8ce1-b748106a5614" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 18:41:17.108928 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:17.108894 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bccf57766-gxkvr" event={"ID":"3fa525fd-fa8e-4574-9203-eae015e8459f","Type":"ContainerStarted","Data":"e17b6a3f7c9036c02e21d91a3fbb68a340c61a229aebc22eefbeb291d9d4af87"} Jul 08 18:41:18.034035 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:18.034004 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" Jul 08 18:41:18.116236 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:18.115899 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-4cdpm" event={"ID":"6c5ec41b-bde9-47f9-963e-c454166a447e","Type":"ContainerStarted","Data":"5f7923fc24bfe66b98abdcae404857190c600a4f9cf9b8afaadb9b40d924c1d4"} Jul 08 18:41:18.116236 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:18.116105 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-4cdpm" Jul 08 18:41:18.121779 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:18.121738 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-4cdpm" Jul 08 18:41:18.132843 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:18.132764 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-4cdpm" podStartSLOduration=1.663469075 podStartE2EDuration="3.132742315s" totalImagesPullingTime="1.46927324s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:41:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:41:18.132422493 +0000 UTC m=+121.368720583" watchObservedRunningTime="2026-07-08 18:41:18.132742315 +0000 UTC m=+121.369040409" Jul 08 18:41:18.547651 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:18.547611 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-d58rk"] Jul 08 18:41:18.551448 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:18.551405 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-d58rk" Jul 08 18:41:18.554354 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:18.554326 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 08 18:41:18.554860 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:18.554656 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 08 18:41:18.554860 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:18.554682 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 08 18:41:18.555034 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:18.555011 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-qkhnr\"" Jul 08 18:41:18.555150 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:18.555089 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 08 18:41:18.555705 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:18.555687 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 08 18:41:18.561896 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:18.561873 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-d58rk"] Jul 08 18:41:18.659873 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:18.659836 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/40364492-d5f5-4bfb-a67a-3beec834dc23-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-d58rk\" (UID: \"40364492-d5f5-4bfb-a67a-3beec834dc23\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-d58rk" Jul 08 18:41:18.659873 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:18.659876 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40364492-d5f5-4bfb-a67a-3beec834dc23-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-d58rk\" (UID: \"40364492-d5f5-4bfb-a67a-3beec834dc23\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-d58rk" Jul 08 18:41:18.660162 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:18.659994 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8vqd\" (UniqueName: \"kubernetes.io/projected/40364492-d5f5-4bfb-a67a-3beec834dc23-kube-api-access-x8vqd\") pod \"prometheus-operator-567f7b8b7c-d58rk\" (UID: \"40364492-d5f5-4bfb-a67a-3beec834dc23\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-d58rk" Jul 08 18:41:18.660162 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:18.660026 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/40364492-d5f5-4bfb-a67a-3beec834dc23-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-d58rk\" (UID: \"40364492-d5f5-4bfb-a67a-3beec834dc23\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-d58rk" Jul 08 18:41:18.761729 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:18.761150 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8vqd\" (UniqueName: \"kubernetes.io/projected/40364492-d5f5-4bfb-a67a-3beec834dc23-kube-api-access-x8vqd\") pod \"prometheus-operator-567f7b8b7c-d58rk\" (UID: \"40364492-d5f5-4bfb-a67a-3beec834dc23\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-d58rk" Jul 08 18:41:18.761729 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:18.761200 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/40364492-d5f5-4bfb-a67a-3beec834dc23-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-d58rk\" (UID: \"40364492-d5f5-4bfb-a67a-3beec834dc23\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-d58rk" Jul 08 18:41:18.761729 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:18.761279 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/40364492-d5f5-4bfb-a67a-3beec834dc23-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-d58rk\" (UID: \"40364492-d5f5-4bfb-a67a-3beec834dc23\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-d58rk" Jul 08 18:41:18.761729 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:41:18.761399 2559 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 08 18:41:18.761729 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:18.761407 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40364492-d5f5-4bfb-a67a-3beec834dc23-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-d58rk\" (UID: \"40364492-d5f5-4bfb-a67a-3beec834dc23\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-d58rk" Jul 08 18:41:18.761729 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:41:18.761467 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40364492-d5f5-4bfb-a67a-3beec834dc23-prometheus-operator-tls podName:40364492-d5f5-4bfb-a67a-3beec834dc23 nodeName:}" failed. No retries permitted until 2026-07-08 18:41:19.26145031 +0000 UTC m=+122.497748378 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/40364492-d5f5-4bfb-a67a-3beec834dc23-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-d58rk" (UID: "40364492-d5f5-4bfb-a67a-3beec834dc23") : secret "prometheus-operator-tls" not found Jul 08 18:41:18.762198 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:18.762130 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40364492-d5f5-4bfb-a67a-3beec834dc23-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-d58rk\" (UID: \"40364492-d5f5-4bfb-a67a-3beec834dc23\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-d58rk" Jul 08 18:41:18.764003 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:18.763976 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/40364492-d5f5-4bfb-a67a-3beec834dc23-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-d58rk\" (UID: \"40364492-d5f5-4bfb-a67a-3beec834dc23\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-d58rk" Jul 08 18:41:18.771517 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:18.770728 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8vqd\" (UniqueName: \"kubernetes.io/projected/40364492-d5f5-4bfb-a67a-3beec834dc23-kube-api-access-x8vqd\") pod \"prometheus-operator-567f7b8b7c-d58rk\" (UID: \"40364492-d5f5-4bfb-a67a-3beec834dc23\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-d58rk" Jul 08 18:41:19.266481 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:19.265730 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/40364492-d5f5-4bfb-a67a-3beec834dc23-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-d58rk\" (UID: \"40364492-d5f5-4bfb-a67a-3beec834dc23\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-d58rk" Jul 08 18:41:19.269525 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:19.269474 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/40364492-d5f5-4bfb-a67a-3beec834dc23-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-d58rk\" (UID: \"40364492-d5f5-4bfb-a67a-3beec834dc23\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-d58rk" Jul 08 18:41:19.464398 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:19.464360 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-d58rk" Jul 08 18:41:19.990561 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:19.990529 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-d58rk"] Jul 08 18:41:19.993876 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:41:19.993847 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40364492_d5f5_4bfb_a67a_3beec834dc23.slice/crio-d3a35dc01539265c28b92b0694a27e3c82fb31074b16ef9a1f8795ec595da568 WatchSource:0}: Error finding container d3a35dc01539265c28b92b0694a27e3c82fb31074b16ef9a1f8795ec595da568: Status 404 returned error can't find the container with id d3a35dc01539265c28b92b0694a27e3c82fb31074b16ef9a1f8795ec595da568 Jul 08 18:41:20.131662 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:20.128067 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bccf57766-gxkvr" event={"ID":"3fa525fd-fa8e-4574-9203-eae015e8459f","Type":"ContainerStarted","Data":"204f331980b8f30111732bc493e350ce5c43709fe6760e8f746f91b72d68141a"} Jul 08 18:41:20.131662 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:20.130384 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-d58rk" event={"ID":"40364492-d5f5-4bfb-a67a-3beec834dc23","Type":"ContainerStarted","Data":"d3a35dc01539265c28b92b0694a27e3c82fb31074b16ef9a1f8795ec595da568"} Jul 08 18:41:20.149421 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:20.149351 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6bccf57766-gxkvr" podStartSLOduration=1.518717592 podStartE2EDuration="5.149331432s" totalImagesPullingTime="3.63061384s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:41:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:41:20.148107638 +0000 UTC m=+123.384405742" watchObservedRunningTime="2026-07-08 18:41:20.149331432 +0000 UTC m=+123.385629518" Jul 08 18:41:22.140137 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:22.140095 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-d58rk" event={"ID":"40364492-d5f5-4bfb-a67a-3beec834dc23","Type":"ContainerStarted","Data":"97d892eb0949f554df9b161f0c1221012737cb104cf117e602e8613f99ab1132"} Jul 08 18:41:22.140586 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:22.140160 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-d58rk" event={"ID":"40364492-d5f5-4bfb-a67a-3beec834dc23","Type":"ContainerStarted","Data":"169e7dae9ea8186f65edd205f5b71f32c999745e9f2db43cac3323a5afb62c42"} Jul 08 18:41:22.163385 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:22.163318 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-d58rk" podStartSLOduration=2.650930943 podStartE2EDuration="4.163294241s" totalImagesPullingTime="1.512363298s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:41:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:41:22.161332471 +0000 UTC m=+125.397630563" watchObservedRunningTime="2026-07-08 18:41:22.163294241 +0000 UTC m=+125.399592333" Jul 08 18:41:23.908418 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:23.908369 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-jlr4r"] Jul 08 18:41:23.912296 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:23.912273 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jlr4r" Jul 08 18:41:23.917318 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:23.917264 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-nmmql\"" Jul 08 18:41:23.917493 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:23.917474 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 08 18:41:23.917718 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:23.917702 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 08 18:41:23.931376 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:23.931343 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-jlr4r"] Jul 08 18:41:23.940813 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:23.940782 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fc9r7"] Jul 08 18:41:23.944319 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:23.944297 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fc9r7" Jul 08 18:41:23.948815 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:23.948787 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 08 18:41:23.948815 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:23.948807 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-pvj8z\"" Jul 08 18:41:23.948981 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:23.948820 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 08 18:41:23.950640 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:23.950623 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 08 18:41:23.959622 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:23.959591 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fc9r7"] Jul 08 18:41:23.962468 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:23.962436 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-nzczn"] Jul 08 18:41:23.967582 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:23.967553 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nzczn" Jul 08 18:41:23.974422 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:23.973267 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-846kr\"" Jul 08 18:41:23.974422 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:23.973743 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 08 18:41:23.974422 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:23.974404 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 08 18:41:23.975502 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:23.974958 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 08 18:41:24.007807 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.007750 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ce91fd5d-2c4c-4762-88b3-2d3cef665933-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-jlr4r\" (UID: \"ce91fd5d-2c4c-4762-88b3-2d3cef665933\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jlr4r" Jul 08 18:41:24.007987 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.007868 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce91fd5d-2c4c-4762-88b3-2d3cef665933-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-jlr4r\" (UID: \"ce91fd5d-2c4c-4762-88b3-2d3cef665933\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jlr4r" Jul 08 18:41:24.007987 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.007899 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce91fd5d-2c4c-4762-88b3-2d3cef665933-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-jlr4r\" (UID: \"ce91fd5d-2c4c-4762-88b3-2d3cef665933\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jlr4r" Jul 08 18:41:24.007987 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.007938 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnm5l\" (UniqueName: \"kubernetes.io/projected/ce91fd5d-2c4c-4762-88b3-2d3cef665933-kube-api-access-fnm5l\") pod \"openshift-state-metrics-54cf96dfcf-jlr4r\" (UID: \"ce91fd5d-2c4c-4762-88b3-2d3cef665933\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jlr4r" Jul 08 18:41:24.108629 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.108589 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1ee283a3-c9af-4aea-ab7e-448d83b4df89-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nzczn\" (UID: \"1ee283a3-c9af-4aea-ab7e-448d83b4df89\") " pod="openshift-monitoring/node-exporter-nzczn" Jul 08 18:41:24.108629 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.108638 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1ee283a3-c9af-4aea-ab7e-448d83b4df89-root\") pod \"node-exporter-nzczn\" (UID: \"1ee283a3-c9af-4aea-ab7e-448d83b4df89\") " pod="openshift-monitoring/node-exporter-nzczn" Jul 08 18:41:24.108887 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.108658 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1ee283a3-c9af-4aea-ab7e-448d83b4df89-node-exporter-wtmp\") pod \"node-exporter-nzczn\" (UID: \"1ee283a3-c9af-4aea-ab7e-448d83b4df89\") " pod="openshift-monitoring/node-exporter-nzczn" Jul 08 18:41:24.108887 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.108718 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1ee283a3-c9af-4aea-ab7e-448d83b4df89-node-exporter-textfile\") pod \"node-exporter-nzczn\" (UID: \"1ee283a3-c9af-4aea-ab7e-448d83b4df89\") " pod="openshift-monitoring/node-exporter-nzczn" Jul 08 18:41:24.108887 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.108754 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/437edb92-3ae1-4a09-8f2f-cc1d583a6753-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-fc9r7\" (UID: \"437edb92-3ae1-4a09-8f2f-cc1d583a6753\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fc9r7" Jul 08 18:41:24.108887 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.108844 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/437edb92-3ae1-4a09-8f2f-cc1d583a6753-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-fc9r7\" (UID: \"437edb92-3ae1-4a09-8f2f-cc1d583a6753\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fc9r7" Jul 08 18:41:24.108887 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.108879 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ce91fd5d-2c4c-4762-88b3-2d3cef665933-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-jlr4r\" (UID: \"ce91fd5d-2c4c-4762-88b3-2d3cef665933\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jlr4r" Jul 08 18:41:24.109094 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.108908 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1ee283a3-c9af-4aea-ab7e-448d83b4df89-node-exporter-tls\") pod \"node-exporter-nzczn\" (UID: \"1ee283a3-c9af-4aea-ab7e-448d83b4df89\") " pod="openshift-monitoring/node-exporter-nzczn" Jul 08 18:41:24.109094 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.108957 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce91fd5d-2c4c-4762-88b3-2d3cef665933-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-jlr4r\" (UID: \"ce91fd5d-2c4c-4762-88b3-2d3cef665933\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jlr4r" Jul 08 18:41:24.109094 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.108975 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/437edb92-3ae1-4a09-8f2f-cc1d583a6753-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-fc9r7\" (UID: \"437edb92-3ae1-4a09-8f2f-cc1d583a6753\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fc9r7" Jul 08 18:41:24.109094 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.108991 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/437edb92-3ae1-4a09-8f2f-cc1d583a6753-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-fc9r7\" (UID: \"437edb92-3ae1-4a09-8f2f-cc1d583a6753\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fc9r7" Jul 08 18:41:24.109094 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.109010 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce91fd5d-2c4c-4762-88b3-2d3cef665933-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-jlr4r\" (UID: \"ce91fd5d-2c4c-4762-88b3-2d3cef665933\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jlr4r" Jul 08 18:41:24.109094 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.109026 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1ee283a3-c9af-4aea-ab7e-448d83b4df89-node-exporter-accelerators-collector-config\") pod \"node-exporter-nzczn\" (UID: \"1ee283a3-c9af-4aea-ab7e-448d83b4df89\") " pod="openshift-monitoring/node-exporter-nzczn" Jul 08 18:41:24.109094 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.109051 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/437edb92-3ae1-4a09-8f2f-cc1d583a6753-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-fc9r7\" (UID: \"437edb92-3ae1-4a09-8f2f-cc1d583a6753\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fc9r7" Jul 08 18:41:24.109469 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:41:24.109104 2559 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 08 18:41:24.109469 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.109147 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzcb4\" (UniqueName: \"kubernetes.io/projected/1ee283a3-c9af-4aea-ab7e-448d83b4df89-kube-api-access-rzcb4\") pod \"node-exporter-nzczn\" (UID: \"1ee283a3-c9af-4aea-ab7e-448d83b4df89\") " pod="openshift-monitoring/node-exporter-nzczn" Jul 08 18:41:24.109469 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:41:24.109173 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce91fd5d-2c4c-4762-88b3-2d3cef665933-openshift-state-metrics-tls podName:ce91fd5d-2c4c-4762-88b3-2d3cef665933 nodeName:}" failed. No retries permitted until 2026-07-08 18:41:24.609152556 +0000 UTC m=+127.845450639 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/ce91fd5d-2c4c-4762-88b3-2d3cef665933-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-jlr4r" (UID: "ce91fd5d-2c4c-4762-88b3-2d3cef665933") : secret "openshift-state-metrics-tls" not found Jul 08 18:41:24.109469 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.109223 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnm5l\" (UniqueName: \"kubernetes.io/projected/ce91fd5d-2c4c-4762-88b3-2d3cef665933-kube-api-access-fnm5l\") pod \"openshift-state-metrics-54cf96dfcf-jlr4r\" (UID: \"ce91fd5d-2c4c-4762-88b3-2d3cef665933\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jlr4r" Jul 08 18:41:24.109469 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.109252 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ee283a3-c9af-4aea-ab7e-448d83b4df89-metrics-client-ca\") pod \"node-exporter-nzczn\" (UID: \"1ee283a3-c9af-4aea-ab7e-448d83b4df89\") " pod="openshift-monitoring/node-exporter-nzczn" Jul 08 18:41:24.109469 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.109288 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1ee283a3-c9af-4aea-ab7e-448d83b4df89-sys\") pod \"node-exporter-nzczn\" (UID: \"1ee283a3-c9af-4aea-ab7e-448d83b4df89\") " pod="openshift-monitoring/node-exporter-nzczn" Jul 08 18:41:24.109469 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.109317 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4sww\" (UniqueName: \"kubernetes.io/projected/437edb92-3ae1-4a09-8f2f-cc1d583a6753-kube-api-access-h4sww\") pod \"kube-state-metrics-7c9dbb8b7d-fc9r7\" (UID: \"437edb92-3ae1-4a09-8f2f-cc1d583a6753\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fc9r7" Jul 08 18:41:24.109944 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.109922 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce91fd5d-2c4c-4762-88b3-2d3cef665933-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-jlr4r\" (UID: \"ce91fd5d-2c4c-4762-88b3-2d3cef665933\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jlr4r" Jul 08 18:41:24.111581 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.111555 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ce91fd5d-2c4c-4762-88b3-2d3cef665933-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-jlr4r\" (UID: \"ce91fd5d-2c4c-4762-88b3-2d3cef665933\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jlr4r" Jul 08 18:41:24.121669 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.121642 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnm5l\" (UniqueName: \"kubernetes.io/projected/ce91fd5d-2c4c-4762-88b3-2d3cef665933-kube-api-access-fnm5l\") pod \"openshift-state-metrics-54cf96dfcf-jlr4r\" (UID: \"ce91fd5d-2c4c-4762-88b3-2d3cef665933\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jlr4r" Jul 08 18:41:24.210031 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.209946 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1ee283a3-c9af-4aea-ab7e-448d83b4df89-sys\") pod \"node-exporter-nzczn\" (UID: \"1ee283a3-c9af-4aea-ab7e-448d83b4df89\") " pod="openshift-monitoring/node-exporter-nzczn" Jul 08 18:41:24.210031 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.209988 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4sww\" (UniqueName: \"kubernetes.io/projected/437edb92-3ae1-4a09-8f2f-cc1d583a6753-kube-api-access-h4sww\") pod \"kube-state-metrics-7c9dbb8b7d-fc9r7\" (UID: \"437edb92-3ae1-4a09-8f2f-cc1d583a6753\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fc9r7" Jul 08 18:41:24.210031 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.210008 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1ee283a3-c9af-4aea-ab7e-448d83b4df89-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nzczn\" (UID: \"1ee283a3-c9af-4aea-ab7e-448d83b4df89\") " pod="openshift-monitoring/node-exporter-nzczn" Jul 08 18:41:24.210286 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.210034 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1ee283a3-c9af-4aea-ab7e-448d83b4df89-root\") pod \"node-exporter-nzczn\" (UID: \"1ee283a3-c9af-4aea-ab7e-448d83b4df89\") " pod="openshift-monitoring/node-exporter-nzczn" Jul 08 18:41:24.210286 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.210063 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1ee283a3-c9af-4aea-ab7e-448d83b4df89-sys\") pod \"node-exporter-nzczn\" (UID: \"1ee283a3-c9af-4aea-ab7e-448d83b4df89\") " pod="openshift-monitoring/node-exporter-nzczn" Jul 08 18:41:24.210286 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.210086 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1ee283a3-c9af-4aea-ab7e-448d83b4df89-root\") pod \"node-exporter-nzczn\" (UID: \"1ee283a3-c9af-4aea-ab7e-448d83b4df89\") " pod="openshift-monitoring/node-exporter-nzczn" Jul 08 18:41:24.210286 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.210143 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1ee283a3-c9af-4aea-ab7e-448d83b4df89-node-exporter-wtmp\") pod \"node-exporter-nzczn\" (UID: \"1ee283a3-c9af-4aea-ab7e-448d83b4df89\") " pod="openshift-monitoring/node-exporter-nzczn" Jul 08 18:41:24.210286 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.210188 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1ee283a3-c9af-4aea-ab7e-448d83b4df89-node-exporter-textfile\") pod \"node-exporter-nzczn\" (UID: \"1ee283a3-c9af-4aea-ab7e-448d83b4df89\") " pod="openshift-monitoring/node-exporter-nzczn" Jul 08 18:41:24.210286 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.210211 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/437edb92-3ae1-4a09-8f2f-cc1d583a6753-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-fc9r7\" (UID: \"437edb92-3ae1-4a09-8f2f-cc1d583a6753\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fc9r7" Jul 08 18:41:24.210286 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.210255 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/437edb92-3ae1-4a09-8f2f-cc1d583a6753-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-fc9r7\" (UID: \"437edb92-3ae1-4a09-8f2f-cc1d583a6753\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fc9r7" Jul 08 18:41:24.210611 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.210293 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1ee283a3-c9af-4aea-ab7e-448d83b4df89-node-exporter-tls\") pod \"node-exporter-nzczn\" (UID: \"1ee283a3-c9af-4aea-ab7e-448d83b4df89\") " pod="openshift-monitoring/node-exporter-nzczn" Jul 08 18:41:24.210611 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.210343 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1ee283a3-c9af-4aea-ab7e-448d83b4df89-node-exporter-wtmp\") pod \"node-exporter-nzczn\" (UID: \"1ee283a3-c9af-4aea-ab7e-448d83b4df89\") " pod="openshift-monitoring/node-exporter-nzczn" Jul 08 18:41:24.210611 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.210386 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/437edb92-3ae1-4a09-8f2f-cc1d583a6753-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-fc9r7\" (UID: \"437edb92-3ae1-4a09-8f2f-cc1d583a6753\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fc9r7" Jul 08 18:41:24.210611 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.210415 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/437edb92-3ae1-4a09-8f2f-cc1d583a6753-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-fc9r7\" (UID: \"437edb92-3ae1-4a09-8f2f-cc1d583a6753\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fc9r7" Jul 08 18:41:24.210611 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:41:24.210440 2559 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 08 18:41:24.210611 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.210449 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1ee283a3-c9af-4aea-ab7e-448d83b4df89-node-exporter-accelerators-collector-config\") pod \"node-exporter-nzczn\" (UID: \"1ee283a3-c9af-4aea-ab7e-448d83b4df89\") " pod="openshift-monitoring/node-exporter-nzczn" Jul 08 18:41:24.210611 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.210471 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/437edb92-3ae1-4a09-8f2f-cc1d583a6753-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-fc9r7\" (UID: \"437edb92-3ae1-4a09-8f2f-cc1d583a6753\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fc9r7" Jul 08 18:41:24.210611 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:41:24.210494 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ee283a3-c9af-4aea-ab7e-448d83b4df89-node-exporter-tls podName:1ee283a3-c9af-4aea-ab7e-448d83b4df89 nodeName:}" failed. No retries permitted until 2026-07-08 18:41:24.710477501 +0000 UTC m=+127.946775586 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/1ee283a3-c9af-4aea-ab7e-448d83b4df89-node-exporter-tls") pod "node-exporter-nzczn" (UID: "1ee283a3-c9af-4aea-ab7e-448d83b4df89") : secret "node-exporter-tls" not found Jul 08 18:41:24.210611 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.210538 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzcb4\" (UniqueName: \"kubernetes.io/projected/1ee283a3-c9af-4aea-ab7e-448d83b4df89-kube-api-access-rzcb4\") pod \"node-exporter-nzczn\" (UID: \"1ee283a3-c9af-4aea-ab7e-448d83b4df89\") " pod="openshift-monitoring/node-exporter-nzczn" Jul 08 18:41:24.210611 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.210563 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1ee283a3-c9af-4aea-ab7e-448d83b4df89-node-exporter-textfile\") pod \"node-exporter-nzczn\" (UID: \"1ee283a3-c9af-4aea-ab7e-448d83b4df89\") " pod="openshift-monitoring/node-exporter-nzczn" Jul 08 18:41:24.210611 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.210577 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ee283a3-c9af-4aea-ab7e-448d83b4df89-metrics-client-ca\") pod \"node-exporter-nzczn\" (UID: \"1ee283a3-c9af-4aea-ab7e-448d83b4df89\") " pod="openshift-monitoring/node-exporter-nzczn" Jul 08 18:41:24.211171 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.210739 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/437edb92-3ae1-4a09-8f2f-cc1d583a6753-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-fc9r7\" (UID: \"437edb92-3ae1-4a09-8f2f-cc1d583a6753\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fc9r7" Jul 08 18:41:24.211171 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.211068 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1ee283a3-c9af-4aea-ab7e-448d83b4df89-node-exporter-accelerators-collector-config\") pod \"node-exporter-nzczn\" (UID: \"1ee283a3-c9af-4aea-ab7e-448d83b4df89\") " pod="openshift-monitoring/node-exporter-nzczn" Jul 08 18:41:24.211330 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.211210 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/437edb92-3ae1-4a09-8f2f-cc1d583a6753-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-fc9r7\" (UID: \"437edb92-3ae1-4a09-8f2f-cc1d583a6753\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fc9r7" Jul 08 18:41:24.211330 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.211235 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ee283a3-c9af-4aea-ab7e-448d83b4df89-metrics-client-ca\") pod \"node-exporter-nzczn\" (UID: \"1ee283a3-c9af-4aea-ab7e-448d83b4df89\") " pod="openshift-monitoring/node-exporter-nzczn" Jul 08 18:41:24.211579 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.211557 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/437edb92-3ae1-4a09-8f2f-cc1d583a6753-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-fc9r7\" (UID: \"437edb92-3ae1-4a09-8f2f-cc1d583a6753\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fc9r7" Jul 08 18:41:24.212966 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.212919 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1ee283a3-c9af-4aea-ab7e-448d83b4df89-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nzczn\" (UID: \"1ee283a3-c9af-4aea-ab7e-448d83b4df89\") " pod="openshift-monitoring/node-exporter-nzczn" Jul 08 18:41:24.213275 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.213247 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/437edb92-3ae1-4a09-8f2f-cc1d583a6753-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-fc9r7\" (UID: \"437edb92-3ae1-4a09-8f2f-cc1d583a6753\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fc9r7" Jul 08 18:41:24.213425 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.213396 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/437edb92-3ae1-4a09-8f2f-cc1d583a6753-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-fc9r7\" (UID: \"437edb92-3ae1-4a09-8f2f-cc1d583a6753\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fc9r7" Jul 08 18:41:24.220313 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.220278 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzcb4\" (UniqueName: \"kubernetes.io/projected/1ee283a3-c9af-4aea-ab7e-448d83b4df89-kube-api-access-rzcb4\") pod \"node-exporter-nzczn\" (UID: \"1ee283a3-c9af-4aea-ab7e-448d83b4df89\") " pod="openshift-monitoring/node-exporter-nzczn" Jul 08 18:41:24.220432 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.220386 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4sww\" (UniqueName: \"kubernetes.io/projected/437edb92-3ae1-4a09-8f2f-cc1d583a6753-kube-api-access-h4sww\") pod \"kube-state-metrics-7c9dbb8b7d-fc9r7\" (UID: \"437edb92-3ae1-4a09-8f2f-cc1d583a6753\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fc9r7" Jul 08 18:41:24.255566 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.255516 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fc9r7" Jul 08 18:41:24.614300 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.614205 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce91fd5d-2c4c-4762-88b3-2d3cef665933-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-jlr4r\" (UID: \"ce91fd5d-2c4c-4762-88b3-2d3cef665933\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jlr4r" Jul 08 18:41:24.614457 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:41:24.614391 2559 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 08 18:41:24.614457 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:41:24.614455 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce91fd5d-2c4c-4762-88b3-2d3cef665933-openshift-state-metrics-tls podName:ce91fd5d-2c4c-4762-88b3-2d3cef665933 nodeName:}" failed. No retries permitted until 2026-07-08 18:41:25.614439977 +0000 UTC m=+128.850738046 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/ce91fd5d-2c4c-4762-88b3-2d3cef665933-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-jlr4r" (UID: "ce91fd5d-2c4c-4762-88b3-2d3cef665933") : secret "openshift-state-metrics-tls" not found Jul 08 18:41:24.715360 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.715323 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1ee283a3-c9af-4aea-ab7e-448d83b4df89-node-exporter-tls\") pod \"node-exporter-nzczn\" (UID: \"1ee283a3-c9af-4aea-ab7e-448d83b4df89\") " pod="openshift-monitoring/node-exporter-nzczn" Jul 08 18:41:24.718077 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.718051 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1ee283a3-c9af-4aea-ab7e-448d83b4df89-node-exporter-tls\") pod \"node-exporter-nzczn\" (UID: \"1ee283a3-c9af-4aea-ab7e-448d83b4df89\") " pod="openshift-monitoring/node-exporter-nzczn" Jul 08 18:41:24.878235 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:24.878201 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nzczn" Jul 08 18:41:25.017266 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.017230 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-c64665fcd-gqmhp"] Jul 08 18:41:25.020604 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.020576 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c64665fcd-gqmhp" Jul 08 18:41:25.029814 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.029707 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 08 18:41:25.032637 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.032610 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c64665fcd-gqmhp"] Jul 08 18:41:25.048345 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.046570 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 18:41:25.050852 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.050830 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:41:25.053721 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.053701 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 08 18:41:25.054077 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.053701 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 08 18:41:25.054174 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.054131 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 08 18:41:25.054279 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.053890 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 08 18:41:25.054374 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.053937 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 08 18:41:25.054431 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.054358 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 08 18:41:25.054519 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.053957 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-ncmwv\"" Jul 08 18:41:25.054606 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.054586 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 08 18:41:25.054712 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.054630 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 08 18:41:25.054712 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.054675 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 08 18:41:25.064757 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.064717 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 18:41:25.118385 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.118354 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2299a3ad-27c4-4235-a567-83b92bd3e18f-oauth-serving-cert\") pod \"console-c64665fcd-gqmhp\" (UID: \"2299a3ad-27c4-4235-a567-83b92bd3e18f\") " pod="openshift-console/console-c64665fcd-gqmhp" Jul 08 18:41:25.118549 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.118424 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2299a3ad-27c4-4235-a567-83b92bd3e18f-console-serving-cert\") pod \"console-c64665fcd-gqmhp\" (UID: \"2299a3ad-27c4-4235-a567-83b92bd3e18f\") " pod="openshift-console/console-c64665fcd-gqmhp" Jul 08 18:41:25.118549 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.118455 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2299a3ad-27c4-4235-a567-83b92bd3e18f-console-config\") pod \"console-c64665fcd-gqmhp\" (UID: \"2299a3ad-27c4-4235-a567-83b92bd3e18f\") " pod="openshift-console/console-c64665fcd-gqmhp" Jul 08 18:41:25.118549 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.118471 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2299a3ad-27c4-4235-a567-83b92bd3e18f-console-oauth-config\") pod \"console-c64665fcd-gqmhp\" (UID: \"2299a3ad-27c4-4235-a567-83b92bd3e18f\") " pod="openshift-console/console-c64665fcd-gqmhp" Jul 08 18:41:25.118549 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.118507 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2299a3ad-27c4-4235-a567-83b92bd3e18f-service-ca\") pod \"console-c64665fcd-gqmhp\" (UID: \"2299a3ad-27c4-4235-a567-83b92bd3e18f\") " pod="openshift-console/console-c64665fcd-gqmhp" Jul 08 18:41:25.118732 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.118640 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpqv6\" (UniqueName: \"kubernetes.io/projected/2299a3ad-27c4-4235-a567-83b92bd3e18f-kube-api-access-gpqv6\") pod \"console-c64665fcd-gqmhp\" (UID: \"2299a3ad-27c4-4235-a567-83b92bd3e18f\") " pod="openshift-console/console-c64665fcd-gqmhp" Jul 08 18:41:25.118732 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.118678 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2299a3ad-27c4-4235-a567-83b92bd3e18f-trusted-ca-bundle\") pod \"console-c64665fcd-gqmhp\" (UID: \"2299a3ad-27c4-4235-a567-83b92bd3e18f\") " pod="openshift-console/console-c64665fcd-gqmhp" Jul 08 18:41:25.219658 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.219549 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2299a3ad-27c4-4235-a567-83b92bd3e18f-console-oauth-config\") pod \"console-c64665fcd-gqmhp\" (UID: \"2299a3ad-27c4-4235-a567-83b92bd3e18f\") " pod="openshift-console/console-c64665fcd-gqmhp" Jul 08 18:41:25.219658 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.219620 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/df8759f2-760b-4f41-8579-af54892eeda8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:41:25.219658 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.219643 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/df8759f2-760b-4f41-8579-af54892eeda8-config-out\") pod \"alertmanager-main-0\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:41:25.219983 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.219670 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2299a3ad-27c4-4235-a567-83b92bd3e18f-service-ca\") pod \"console-c64665fcd-gqmhp\" (UID: \"2299a3ad-27c4-4235-a567-83b92bd3e18f\") " pod="openshift-console/console-c64665fcd-gqmhp" Jul 08 18:41:25.219983 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.219707 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:41:25.219983 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.219732 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/df8759f2-760b-4f41-8579-af54892eeda8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:41:25.219983 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.219756 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txtn5\" (UniqueName: \"kubernetes.io/projected/df8759f2-760b-4f41-8579-af54892eeda8-kube-api-access-txtn5\") pod \"alertmanager-main-0\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:41:25.219983 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.219797 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:41:25.219983 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.219822 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df8759f2-760b-4f41-8579-af54892eeda8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:41:25.219983 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.219862 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpqv6\" (UniqueName: \"kubernetes.io/projected/2299a3ad-27c4-4235-a567-83b92bd3e18f-kube-api-access-gpqv6\") pod \"console-c64665fcd-gqmhp\" (UID: \"2299a3ad-27c4-4235-a567-83b92bd3e18f\") " pod="openshift-console/console-c64665fcd-gqmhp" Jul 08 18:41:25.219983 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.219889 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2299a3ad-27c4-4235-a567-83b92bd3e18f-trusted-ca-bundle\") pod \"console-c64665fcd-gqmhp\" (UID: \"2299a3ad-27c4-4235-a567-83b92bd3e18f\") " pod="openshift-console/console-c64665fcd-gqmhp" Jul 08 18:41:25.219983 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.219918 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-web-config\") pod \"alertmanager-main-0\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:41:25.219983 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.219957 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:41:25.220461 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.219994 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/df8759f2-760b-4f41-8579-af54892eeda8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:41:25.220461 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.220023 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2299a3ad-27c4-4235-a567-83b92bd3e18f-oauth-serving-cert\") pod \"console-c64665fcd-gqmhp\" (UID: \"2299a3ad-27c4-4235-a567-83b92bd3e18f\") " pod="openshift-console/console-c64665fcd-gqmhp" Jul 08 18:41:25.220461 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.220049 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-config-volume\") pod \"alertmanager-main-0\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:41:25.220461 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.220072 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:41:25.220461 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.220121 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2299a3ad-27c4-4235-a567-83b92bd3e18f-console-serving-cert\") pod \"console-c64665fcd-gqmhp\" (UID: \"2299a3ad-27c4-4235-a567-83b92bd3e18f\") " pod="openshift-console/console-c64665fcd-gqmhp" Jul 08 18:41:25.220461 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.220143 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:41:25.220461 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.220177 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2299a3ad-27c4-4235-a567-83b92bd3e18f-console-config\") pod \"console-c64665fcd-gqmhp\" (UID: \"2299a3ad-27c4-4235-a567-83b92bd3e18f\") " pod="openshift-console/console-c64665fcd-gqmhp" Jul 08 18:41:25.220792 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.220457 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2299a3ad-27c4-4235-a567-83b92bd3e18f-service-ca\") pod \"console-c64665fcd-gqmhp\" (UID: \"2299a3ad-27c4-4235-a567-83b92bd3e18f\") " pod="openshift-console/console-c64665fcd-gqmhp" Jul 08 18:41:25.220858 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.220837 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2299a3ad-27c4-4235-a567-83b92bd3e18f-oauth-serving-cert\") pod \"console-c64665fcd-gqmhp\" (UID: \"2299a3ad-27c4-4235-a567-83b92bd3e18f\") " pod="openshift-console/console-c64665fcd-gqmhp" Jul 08 18:41:25.221060 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.221018 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2299a3ad-27c4-4235-a567-83b92bd3e18f-console-config\") pod \"console-c64665fcd-gqmhp\" (UID: \"2299a3ad-27c4-4235-a567-83b92bd3e18f\") " pod="openshift-console/console-c64665fcd-gqmhp" Jul 08 18:41:25.221254 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.221230 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2299a3ad-27c4-4235-a567-83b92bd3e18f-trusted-ca-bundle\") pod \"console-c64665fcd-gqmhp\" (UID: \"2299a3ad-27c4-4235-a567-83b92bd3e18f\") " pod="openshift-console/console-c64665fcd-gqmhp" Jul 08 18:41:25.222535 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.222509 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2299a3ad-27c4-4235-a567-83b92bd3e18f-console-oauth-config\") pod \"console-c64665fcd-gqmhp\" (UID: \"2299a3ad-27c4-4235-a567-83b92bd3e18f\") " pod="openshift-console/console-c64665fcd-gqmhp" Jul 08 18:41:25.222839 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.222821 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2299a3ad-27c4-4235-a567-83b92bd3e18f-console-serving-cert\") pod \"console-c64665fcd-gqmhp\" (UID: \"2299a3ad-27c4-4235-a567-83b92bd3e18f\") " pod="openshift-console/console-c64665fcd-gqmhp" Jul 08 18:41:25.232496 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.232468 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpqv6\" (UniqueName: \"kubernetes.io/projected/2299a3ad-27c4-4235-a567-83b92bd3e18f-kube-api-access-gpqv6\") pod \"console-c64665fcd-gqmhp\" (UID: \"2299a3ad-27c4-4235-a567-83b92bd3e18f\") " pod="openshift-console/console-c64665fcd-gqmhp" Jul 08 18:41:25.320911 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.320871 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:41:25.320911 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.320929 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/df8759f2-760b-4f41-8579-af54892eeda8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:41:25.321163 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.320964 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-config-volume\") pod \"alertmanager-main-0\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:41:25.321163 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.320982 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:41:25.321163 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.321038 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:41:25.321163 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.321090 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/df8759f2-760b-4f41-8579-af54892eeda8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:41:25.321163 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.321111 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/df8759f2-760b-4f41-8579-af54892eeda8-config-out\") pod \"alertmanager-main-0\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:41:25.321163 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.321164 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:41:25.321428 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.321189 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/df8759f2-760b-4f41-8579-af54892eeda8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:41:25.321428 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.321212 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txtn5\" (UniqueName: \"kubernetes.io/projected/df8759f2-760b-4f41-8579-af54892eeda8-kube-api-access-txtn5\") pod \"alertmanager-main-0\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:41:25.321428 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:41:25.321229 2559 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 08 18:41:25.321428 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:41:25.321317 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-secret-alertmanager-main-tls podName:df8759f2-760b-4f41-8579-af54892eeda8 nodeName:}" failed. No retries permitted until 2026-07-08 18:41:25.821294864 +0000 UTC m=+129.057592936 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "df8759f2-760b-4f41-8579-af54892eeda8") : secret "alertmanager-main-tls" not found Jul 08 18:41:25.321428 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.321238 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:41:25.321691 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.321432 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df8759f2-760b-4f41-8579-af54892eeda8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:41:25.321691 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.321516 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-web-config\") pod \"alertmanager-main-0\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:41:25.321691 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.321529 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/df8759f2-760b-4f41-8579-af54892eeda8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:41:25.322732 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.322305 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/df8759f2-760b-4f41-8579-af54892eeda8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:41:25.322732 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.322687 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df8759f2-760b-4f41-8579-af54892eeda8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:41:25.324125 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.324080 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/df8759f2-760b-4f41-8579-af54892eeda8-config-out\") pod \"alertmanager-main-0\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:41:25.324877 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.324679 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:41:25.324877 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.324797 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:41:25.325239 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.325212 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:41:25.325565 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.325541 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-config-volume\") pod \"alertmanager-main-0\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:41:25.325634 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.325590 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:41:25.325634 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.325623 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-web-config\") pod \"alertmanager-main-0\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:41:25.326477 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.326456 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/df8759f2-760b-4f41-8579-af54892eeda8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:41:25.331217 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.331197 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-txtn5\" (UniqueName: \"kubernetes.io/projected/df8759f2-760b-4f41-8579-af54892eeda8-kube-api-access-txtn5\") pod \"alertmanager-main-0\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:41:25.334019 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.334000 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c64665fcd-gqmhp" Jul 08 18:41:25.625151 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.625107 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce91fd5d-2c4c-4762-88b3-2d3cef665933-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-jlr4r\" (UID: \"ce91fd5d-2c4c-4762-88b3-2d3cef665933\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jlr4r" Jul 08 18:41:25.627838 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.627809 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce91fd5d-2c4c-4762-88b3-2d3cef665933-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-jlr4r\" (UID: \"ce91fd5d-2c4c-4762-88b3-2d3cef665933\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jlr4r" Jul 08 18:41:25.723216 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.723164 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jlr4r" Jul 08 18:41:25.827374 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.827346 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:41:25.829860 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.829830 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:41:25.963277 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.963193 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:41:25.964324 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.964289 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-579fd557b5-qzlxn"] Jul 08 18:41:25.968423 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.968402 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-579fd557b5-qzlxn" Jul 08 18:41:25.982147 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.982118 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 08 18:41:25.982252 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.982203 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 08 18:41:25.982492 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.982474 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-572fsl1mvdnno\"" Jul 08 18:41:25.982595 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.982570 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-qlnzn\"" Jul 08 18:41:25.982869 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.982851 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 08 18:41:25.982945 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.982933 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 08 18:41:25.987122 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:25.987101 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 08 18:41:26.013486 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:26.013451 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-579fd557b5-qzlxn"] Jul 08 18:41:26.130896 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:26.130856 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/357c8e19-72d5-48e2-bcb6-7f222b54b0e6-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-579fd557b5-qzlxn\" (UID: \"357c8e19-72d5-48e2-bcb6-7f222b54b0e6\") " pod="openshift-monitoring/thanos-querier-579fd557b5-qzlxn" Jul 08 18:41:26.131364 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:26.130914 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/357c8e19-72d5-48e2-bcb6-7f222b54b0e6-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-579fd557b5-qzlxn\" (UID: \"357c8e19-72d5-48e2-bcb6-7f222b54b0e6\") " pod="openshift-monitoring/thanos-querier-579fd557b5-qzlxn" Jul 08 18:41:26.131364 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:26.130947 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5shdw\" (UniqueName: \"kubernetes.io/projected/357c8e19-72d5-48e2-bcb6-7f222b54b0e6-kube-api-access-5shdw\") pod \"thanos-querier-579fd557b5-qzlxn\" (UID: \"357c8e19-72d5-48e2-bcb6-7f222b54b0e6\") " pod="openshift-monitoring/thanos-querier-579fd557b5-qzlxn" Jul 08 18:41:26.131364 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:26.131003 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/357c8e19-72d5-48e2-bcb6-7f222b54b0e6-secret-thanos-querier-tls\") pod \"thanos-querier-579fd557b5-qzlxn\" (UID: \"357c8e19-72d5-48e2-bcb6-7f222b54b0e6\") " pod="openshift-monitoring/thanos-querier-579fd557b5-qzlxn" Jul 08 18:41:26.131364 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:26.131027 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/357c8e19-72d5-48e2-bcb6-7f222b54b0e6-secret-grpc-tls\") pod \"thanos-querier-579fd557b5-qzlxn\" (UID: \"357c8e19-72d5-48e2-bcb6-7f222b54b0e6\") " pod="openshift-monitoring/thanos-querier-579fd557b5-qzlxn" Jul 08 18:41:26.131364 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:26.131045 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/357c8e19-72d5-48e2-bcb6-7f222b54b0e6-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-579fd557b5-qzlxn\" (UID: \"357c8e19-72d5-48e2-bcb6-7f222b54b0e6\") " pod="openshift-monitoring/thanos-querier-579fd557b5-qzlxn" Jul 08 18:41:26.131364 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:26.131170 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/357c8e19-72d5-48e2-bcb6-7f222b54b0e6-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-579fd557b5-qzlxn\" (UID: \"357c8e19-72d5-48e2-bcb6-7f222b54b0e6\") " pod="openshift-monitoring/thanos-querier-579fd557b5-qzlxn" Jul 08 18:41:26.131364 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:26.131210 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/357c8e19-72d5-48e2-bcb6-7f222b54b0e6-metrics-client-ca\") pod \"thanos-querier-579fd557b5-qzlxn\" (UID: \"357c8e19-72d5-48e2-bcb6-7f222b54b0e6\") " pod="openshift-monitoring/thanos-querier-579fd557b5-qzlxn" Jul 08 18:41:26.143883 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:26.143848 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6bccf57766-gxkvr" Jul 08 18:41:26.144034 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:26.143921 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6bccf57766-gxkvr" Jul 08 18:41:26.149251 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:26.149226 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6bccf57766-gxkvr" Jul 08 18:41:26.162538 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:26.162513 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6bccf57766-gxkvr" Jul 08 18:41:26.232458 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:26.232387 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/357c8e19-72d5-48e2-bcb6-7f222b54b0e6-secret-thanos-querier-tls\") pod \"thanos-querier-579fd557b5-qzlxn\" (UID: \"357c8e19-72d5-48e2-bcb6-7f222b54b0e6\") " pod="openshift-monitoring/thanos-querier-579fd557b5-qzlxn" Jul 08 18:41:26.232458 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:26.232435 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/357c8e19-72d5-48e2-bcb6-7f222b54b0e6-secret-grpc-tls\") pod \"thanos-querier-579fd557b5-qzlxn\" (UID: \"357c8e19-72d5-48e2-bcb6-7f222b54b0e6\") " pod="openshift-monitoring/thanos-querier-579fd557b5-qzlxn" Jul 08 18:41:26.232666 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:26.232565 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/357c8e19-72d5-48e2-bcb6-7f222b54b0e6-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-579fd557b5-qzlxn\" (UID: \"357c8e19-72d5-48e2-bcb6-7f222b54b0e6\") " pod="openshift-monitoring/thanos-querier-579fd557b5-qzlxn" Jul 08 18:41:26.232666 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:26.232634 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/357c8e19-72d5-48e2-bcb6-7f222b54b0e6-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-579fd557b5-qzlxn\" (UID: \"357c8e19-72d5-48e2-bcb6-7f222b54b0e6\") " pod="openshift-monitoring/thanos-querier-579fd557b5-qzlxn" Jul 08 18:41:26.232666 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:26.232664 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/357c8e19-72d5-48e2-bcb6-7f222b54b0e6-metrics-client-ca\") pod \"thanos-querier-579fd557b5-qzlxn\" (UID: \"357c8e19-72d5-48e2-bcb6-7f222b54b0e6\") " pod="openshift-monitoring/thanos-querier-579fd557b5-qzlxn" Jul 08 18:41:26.232847 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:26.232713 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/357c8e19-72d5-48e2-bcb6-7f222b54b0e6-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-579fd557b5-qzlxn\" (UID: \"357c8e19-72d5-48e2-bcb6-7f222b54b0e6\") " pod="openshift-monitoring/thanos-querier-579fd557b5-qzlxn" Jul 08 18:41:26.232847 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:26.232754 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/357c8e19-72d5-48e2-bcb6-7f222b54b0e6-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-579fd557b5-qzlxn\" (UID: \"357c8e19-72d5-48e2-bcb6-7f222b54b0e6\") " pod="openshift-monitoring/thanos-querier-579fd557b5-qzlxn" Jul 08 18:41:26.232847 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:26.232800 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5shdw\" (UniqueName: \"kubernetes.io/projected/357c8e19-72d5-48e2-bcb6-7f222b54b0e6-kube-api-access-5shdw\") pod \"thanos-querier-579fd557b5-qzlxn\" (UID: \"357c8e19-72d5-48e2-bcb6-7f222b54b0e6\") " pod="openshift-monitoring/thanos-querier-579fd557b5-qzlxn" Jul 08 18:41:26.233800 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:26.233721 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/357c8e19-72d5-48e2-bcb6-7f222b54b0e6-metrics-client-ca\") pod \"thanos-querier-579fd557b5-qzlxn\" (UID: \"357c8e19-72d5-48e2-bcb6-7f222b54b0e6\") " pod="openshift-monitoring/thanos-querier-579fd557b5-qzlxn" Jul 08 18:41:26.235824 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:26.235781 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/357c8e19-72d5-48e2-bcb6-7f222b54b0e6-secret-thanos-querier-tls\") pod \"thanos-querier-579fd557b5-qzlxn\" (UID: \"357c8e19-72d5-48e2-bcb6-7f222b54b0e6\") " pod="openshift-monitoring/thanos-querier-579fd557b5-qzlxn" Jul 08 18:41:26.235936 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:26.235867 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/357c8e19-72d5-48e2-bcb6-7f222b54b0e6-secret-grpc-tls\") pod \"thanos-querier-579fd557b5-qzlxn\" (UID: \"357c8e19-72d5-48e2-bcb6-7f222b54b0e6\") " pod="openshift-monitoring/thanos-querier-579fd557b5-qzlxn" Jul 08 18:41:26.235936 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:26.235881 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/357c8e19-72d5-48e2-bcb6-7f222b54b0e6-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-579fd557b5-qzlxn\" (UID: \"357c8e19-72d5-48e2-bcb6-7f222b54b0e6\") " pod="openshift-monitoring/thanos-querier-579fd557b5-qzlxn" Jul 08 18:41:26.236752 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:26.236722 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/357c8e19-72d5-48e2-bcb6-7f222b54b0e6-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-579fd557b5-qzlxn\" (UID: \"357c8e19-72d5-48e2-bcb6-7f222b54b0e6\") " pod="openshift-monitoring/thanos-querier-579fd557b5-qzlxn" Jul 08 18:41:26.236902 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:26.236874 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/357c8e19-72d5-48e2-bcb6-7f222b54b0e6-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-579fd557b5-qzlxn\" (UID: \"357c8e19-72d5-48e2-bcb6-7f222b54b0e6\") " pod="openshift-monitoring/thanos-querier-579fd557b5-qzlxn" Jul 08 18:41:26.237843 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:26.237803 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/357c8e19-72d5-48e2-bcb6-7f222b54b0e6-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-579fd557b5-qzlxn\" (UID: \"357c8e19-72d5-48e2-bcb6-7f222b54b0e6\") " pod="openshift-monitoring/thanos-querier-579fd557b5-qzlxn" Jul 08 18:41:26.264866 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:26.264835 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5shdw\" (UniqueName: \"kubernetes.io/projected/357c8e19-72d5-48e2-bcb6-7f222b54b0e6-kube-api-access-5shdw\") pod \"thanos-querier-579fd557b5-qzlxn\" (UID: \"357c8e19-72d5-48e2-bcb6-7f222b54b0e6\") " pod="openshift-monitoring/thanos-querier-579fd557b5-qzlxn" Jul 08 18:41:26.280662 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:26.280625 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-579fd557b5-qzlxn" Jul 08 18:41:27.038997 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:27.038950 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14cb8ce5-3cb8-4827-8034-d74d7a8d75bf-metrics-certs\") pod \"network-metrics-daemon-dc669\" (UID: \"14cb8ce5-3cb8-4827-8034-d74d7a8d75bf\") " pod="openshift-multus/network-metrics-daemon-dc669" Jul 08 18:41:27.041608 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:27.041580 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14cb8ce5-3cb8-4827-8034-d74d7a8d75bf-metrics-certs\") pod \"network-metrics-daemon-dc669\" (UID: \"14cb8ce5-3cb8-4827-8034-d74d7a8d75bf\") " pod="openshift-multus/network-metrics-daemon-dc669" Jul 08 18:41:27.301288 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:27.301204 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sc5vc\"" Jul 08 18:41:27.309249 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:27.309219 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dc669" Jul 08 18:41:28.349782 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:28.349729 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6dcfd8bc94-jlv5w"] Jul 08 18:41:28.353630 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:28.353608 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6dcfd8bc94-jlv5w" Jul 08 18:41:28.356872 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:28.356837 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-f3her9m45ed7l\"" Jul 08 18:41:28.357511 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:28.357487 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 08 18:41:28.358049 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:28.358029 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-4wj4j\"" Jul 08 18:41:28.358380 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:28.358362 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 08 18:41:28.358819 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:28.358641 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 08 18:41:28.358913 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:28.358839 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 08 18:41:28.368483 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:28.368462 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6dcfd8bc94-jlv5w"] Jul 08 18:41:28.550042 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:28.550002 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b4d384fb-fe74-4395-bc16-cffdc9cf9789-metrics-server-audit-profiles\") pod \"metrics-server-6dcfd8bc94-jlv5w\" (UID: \"b4d384fb-fe74-4395-bc16-cffdc9cf9789\") " pod="openshift-monitoring/metrics-server-6dcfd8bc94-jlv5w" Jul 08 18:41:28.550237 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:28.550065 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b4d384fb-fe74-4395-bc16-cffdc9cf9789-audit-log\") pod \"metrics-server-6dcfd8bc94-jlv5w\" (UID: \"b4d384fb-fe74-4395-bc16-cffdc9cf9789\") " pod="openshift-monitoring/metrics-server-6dcfd8bc94-jlv5w" Jul 08 18:41:28.550237 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:28.550091 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b4d384fb-fe74-4395-bc16-cffdc9cf9789-secret-metrics-server-tls\") pod \"metrics-server-6dcfd8bc94-jlv5w\" (UID: \"b4d384fb-fe74-4395-bc16-cffdc9cf9789\") " pod="openshift-monitoring/metrics-server-6dcfd8bc94-jlv5w" Jul 08 18:41:28.550237 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:28.550184 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqc2l\" (UniqueName: \"kubernetes.io/projected/b4d384fb-fe74-4395-bc16-cffdc9cf9789-kube-api-access-gqc2l\") pod \"metrics-server-6dcfd8bc94-jlv5w\" (UID: \"b4d384fb-fe74-4395-bc16-cffdc9cf9789\") " pod="openshift-monitoring/metrics-server-6dcfd8bc94-jlv5w" Jul 08 18:41:28.550237 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:28.550222 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b4d384fb-fe74-4395-bc16-cffdc9cf9789-secret-metrics-server-client-certs\") pod \"metrics-server-6dcfd8bc94-jlv5w\" (UID: \"b4d384fb-fe74-4395-bc16-cffdc9cf9789\") " pod="openshift-monitoring/metrics-server-6dcfd8bc94-jlv5w" Jul 08 18:41:28.550452 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:28.550254 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4d384fb-fe74-4395-bc16-cffdc9cf9789-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6dcfd8bc94-jlv5w\" (UID: \"b4d384fb-fe74-4395-bc16-cffdc9cf9789\") " pod="openshift-monitoring/metrics-server-6dcfd8bc94-jlv5w" Jul 08 18:41:28.550452 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:28.550385 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b4d384fb-fe74-4395-bc16-cffdc9cf9789-client-ca-bundle\") pod \"metrics-server-6dcfd8bc94-jlv5w\" (UID: \"b4d384fb-fe74-4395-bc16-cffdc9cf9789\") " pod="openshift-monitoring/metrics-server-6dcfd8bc94-jlv5w" Jul 08 18:41:28.651642 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:28.651607 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqc2l\" (UniqueName: \"kubernetes.io/projected/b4d384fb-fe74-4395-bc16-cffdc9cf9789-kube-api-access-gqc2l\") pod \"metrics-server-6dcfd8bc94-jlv5w\" (UID: \"b4d384fb-fe74-4395-bc16-cffdc9cf9789\") " pod="openshift-monitoring/metrics-server-6dcfd8bc94-jlv5w" Jul 08 18:41:28.651847 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:28.651666 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b4d384fb-fe74-4395-bc16-cffdc9cf9789-secret-metrics-server-client-certs\") pod \"metrics-server-6dcfd8bc94-jlv5w\" (UID: \"b4d384fb-fe74-4395-bc16-cffdc9cf9789\") " pod="openshift-monitoring/metrics-server-6dcfd8bc94-jlv5w" Jul 08 18:41:28.651847 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:28.651700 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4d384fb-fe74-4395-bc16-cffdc9cf9789-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6dcfd8bc94-jlv5w\" (UID: \"b4d384fb-fe74-4395-bc16-cffdc9cf9789\") " pod="openshift-monitoring/metrics-server-6dcfd8bc94-jlv5w" Jul 08 18:41:28.651847 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:28.651745 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b4d384fb-fe74-4395-bc16-cffdc9cf9789-client-ca-bundle\") pod \"metrics-server-6dcfd8bc94-jlv5w\" (UID: \"b4d384fb-fe74-4395-bc16-cffdc9cf9789\") " pod="openshift-monitoring/metrics-server-6dcfd8bc94-jlv5w" Jul 08 18:41:28.651847 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:28.651822 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b4d384fb-fe74-4395-bc16-cffdc9cf9789-metrics-server-audit-profiles\") pod \"metrics-server-6dcfd8bc94-jlv5w\" (UID: \"b4d384fb-fe74-4395-bc16-cffdc9cf9789\") " pod="openshift-monitoring/metrics-server-6dcfd8bc94-jlv5w" Jul 08 18:41:28.652064 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:28.651868 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b4d384fb-fe74-4395-bc16-cffdc9cf9789-audit-log\") pod \"metrics-server-6dcfd8bc94-jlv5w\" (UID: \"b4d384fb-fe74-4395-bc16-cffdc9cf9789\") " pod="openshift-monitoring/metrics-server-6dcfd8bc94-jlv5w" Jul 08 18:41:28.652064 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:28.651898 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b4d384fb-fe74-4395-bc16-cffdc9cf9789-secret-metrics-server-tls\") pod \"metrics-server-6dcfd8bc94-jlv5w\" (UID: \"b4d384fb-fe74-4395-bc16-cffdc9cf9789\") " pod="openshift-monitoring/metrics-server-6dcfd8bc94-jlv5w" Jul 08 18:41:28.652688 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:28.652616 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4d384fb-fe74-4395-bc16-cffdc9cf9789-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6dcfd8bc94-jlv5w\" (UID: \"b4d384fb-fe74-4395-bc16-cffdc9cf9789\") " pod="openshift-monitoring/metrics-server-6dcfd8bc94-jlv5w" Jul 08 18:41:28.653163 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:28.652943 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b4d384fb-fe74-4395-bc16-cffdc9cf9789-audit-log\") pod \"metrics-server-6dcfd8bc94-jlv5w\" (UID: \"b4d384fb-fe74-4395-bc16-cffdc9cf9789\") " pod="openshift-monitoring/metrics-server-6dcfd8bc94-jlv5w" Jul 08 18:41:28.653379 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:28.653355 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b4d384fb-fe74-4395-bc16-cffdc9cf9789-metrics-server-audit-profiles\") pod \"metrics-server-6dcfd8bc94-jlv5w\" (UID: \"b4d384fb-fe74-4395-bc16-cffdc9cf9789\") " pod="openshift-monitoring/metrics-server-6dcfd8bc94-jlv5w" Jul 08 18:41:28.654881 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:28.654858 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b4d384fb-fe74-4395-bc16-cffdc9cf9789-secret-metrics-server-client-certs\") pod \"metrics-server-6dcfd8bc94-jlv5w\" (UID: \"b4d384fb-fe74-4395-bc16-cffdc9cf9789\") " pod="openshift-monitoring/metrics-server-6dcfd8bc94-jlv5w" Jul 08 18:41:28.656001 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:28.655978 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b4d384fb-fe74-4395-bc16-cffdc9cf9789-secret-metrics-server-tls\") pod \"metrics-server-6dcfd8bc94-jlv5w\" (UID: \"b4d384fb-fe74-4395-bc16-cffdc9cf9789\") " pod="openshift-monitoring/metrics-server-6dcfd8bc94-jlv5w" Jul 08 18:41:28.656335 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:28.656308 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b4d384fb-fe74-4395-bc16-cffdc9cf9789-client-ca-bundle\") pod \"metrics-server-6dcfd8bc94-jlv5w\" (UID: \"b4d384fb-fe74-4395-bc16-cffdc9cf9789\") " pod="openshift-monitoring/metrics-server-6dcfd8bc94-jlv5w" Jul 08 18:41:28.663220 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:28.663200 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqc2l\" (UniqueName: \"kubernetes.io/projected/b4d384fb-fe74-4395-bc16-cffdc9cf9789-kube-api-access-gqc2l\") pod \"metrics-server-6dcfd8bc94-jlv5w\" (UID: \"b4d384fb-fe74-4395-bc16-cffdc9cf9789\") " pod="openshift-monitoring/metrics-server-6dcfd8bc94-jlv5w" Jul 08 18:41:28.666092 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:28.666073 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6dcfd8bc94-jlv5w" Jul 08 18:41:30.656556 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:30.656523 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c64665fcd-gqmhp"] Jul 08 18:41:30.705980 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:30.705943 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-f8c787784-rfdrg"] Jul 08 18:41:30.711127 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:30.711097 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f8c787784-rfdrg" Jul 08 18:41:30.729932 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:30.729899 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f8c787784-rfdrg"] Jul 08 18:41:30.770111 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:30.770079 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b36f17f1-a05f-44b2-9304-291e12d3edb8-console-config\") pod \"console-f8c787784-rfdrg\" (UID: \"b36f17f1-a05f-44b2-9304-291e12d3edb8\") " pod="openshift-console/console-f8c787784-rfdrg" Jul 08 18:41:30.770111 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:30.770116 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b36f17f1-a05f-44b2-9304-291e12d3edb8-oauth-serving-cert\") pod \"console-f8c787784-rfdrg\" (UID: \"b36f17f1-a05f-44b2-9304-291e12d3edb8\") " pod="openshift-console/console-f8c787784-rfdrg" Jul 08 18:41:30.770333 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:30.770145 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b36f17f1-a05f-44b2-9304-291e12d3edb8-console-oauth-config\") pod \"console-f8c787784-rfdrg\" (UID: \"b36f17f1-a05f-44b2-9304-291e12d3edb8\") " pod="openshift-console/console-f8c787784-rfdrg" Jul 08 18:41:30.770333 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:30.770251 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b36f17f1-a05f-44b2-9304-291e12d3edb8-console-serving-cert\") pod \"console-f8c787784-rfdrg\" (UID: \"b36f17f1-a05f-44b2-9304-291e12d3edb8\") " pod="openshift-console/console-f8c787784-rfdrg" Jul 08 18:41:30.770333 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:30.770277 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b36f17f1-a05f-44b2-9304-291e12d3edb8-trusted-ca-bundle\") pod \"console-f8c787784-rfdrg\" (UID: \"b36f17f1-a05f-44b2-9304-291e12d3edb8\") " pod="openshift-console/console-f8c787784-rfdrg" Jul 08 18:41:30.770333 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:30.770298 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b36f17f1-a05f-44b2-9304-291e12d3edb8-service-ca\") pod \"console-f8c787784-rfdrg\" (UID: \"b36f17f1-a05f-44b2-9304-291e12d3edb8\") " pod="openshift-console/console-f8c787784-rfdrg" Jul 08 18:41:30.770531 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:30.770398 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzclk\" (UniqueName: \"kubernetes.io/projected/b36f17f1-a05f-44b2-9304-291e12d3edb8-kube-api-access-vzclk\") pod \"console-f8c787784-rfdrg\" (UID: \"b36f17f1-a05f-44b2-9304-291e12d3edb8\") " pod="openshift-console/console-f8c787784-rfdrg" Jul 08 18:41:30.871361 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:30.871309 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzclk\" (UniqueName: \"kubernetes.io/projected/b36f17f1-a05f-44b2-9304-291e12d3edb8-kube-api-access-vzclk\") pod \"console-f8c787784-rfdrg\" (UID: \"b36f17f1-a05f-44b2-9304-291e12d3edb8\") " pod="openshift-console/console-f8c787784-rfdrg" Jul 08 18:41:30.871361 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:30.871366 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b36f17f1-a05f-44b2-9304-291e12d3edb8-console-config\") pod \"console-f8c787784-rfdrg\" (UID: \"b36f17f1-a05f-44b2-9304-291e12d3edb8\") " pod="openshift-console/console-f8c787784-rfdrg" Jul 08 18:41:30.871610 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:30.871484 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b36f17f1-a05f-44b2-9304-291e12d3edb8-oauth-serving-cert\") pod \"console-f8c787784-rfdrg\" (UID: \"b36f17f1-a05f-44b2-9304-291e12d3edb8\") " pod="openshift-console/console-f8c787784-rfdrg" Jul 08 18:41:30.871610 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:30.871530 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b36f17f1-a05f-44b2-9304-291e12d3edb8-console-oauth-config\") pod \"console-f8c787784-rfdrg\" (UID: \"b36f17f1-a05f-44b2-9304-291e12d3edb8\") " pod="openshift-console/console-f8c787784-rfdrg" Jul 08 18:41:30.871610 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:30.871586 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b36f17f1-a05f-44b2-9304-291e12d3edb8-console-serving-cert\") pod \"console-f8c787784-rfdrg\" (UID: \"b36f17f1-a05f-44b2-9304-291e12d3edb8\") " pod="openshift-console/console-f8c787784-rfdrg" Jul 08 18:41:30.871763 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:30.871618 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b36f17f1-a05f-44b2-9304-291e12d3edb8-trusted-ca-bundle\") pod \"console-f8c787784-rfdrg\" (UID: \"b36f17f1-a05f-44b2-9304-291e12d3edb8\") " pod="openshift-console/console-f8c787784-rfdrg" Jul 08 18:41:30.871763 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:30.871651 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b36f17f1-a05f-44b2-9304-291e12d3edb8-service-ca\") pod \"console-f8c787784-rfdrg\" (UID: \"b36f17f1-a05f-44b2-9304-291e12d3edb8\") " pod="openshift-console/console-f8c787784-rfdrg" Jul 08 18:41:30.872291 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:30.872232 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b36f17f1-a05f-44b2-9304-291e12d3edb8-console-config\") pod \"console-f8c787784-rfdrg\" (UID: \"b36f17f1-a05f-44b2-9304-291e12d3edb8\") " pod="openshift-console/console-f8c787784-rfdrg" Jul 08 18:41:30.872698 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:30.872651 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b36f17f1-a05f-44b2-9304-291e12d3edb8-oauth-serving-cert\") pod \"console-f8c787784-rfdrg\" (UID: \"b36f17f1-a05f-44b2-9304-291e12d3edb8\") " pod="openshift-console/console-f8c787784-rfdrg" Jul 08 18:41:30.872855 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:30.872792 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b36f17f1-a05f-44b2-9304-291e12d3edb8-service-ca\") pod \"console-f8c787784-rfdrg\" (UID: \"b36f17f1-a05f-44b2-9304-291e12d3edb8\") " pod="openshift-console/console-f8c787784-rfdrg" Jul 08 18:41:30.872934 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:30.872918 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b36f17f1-a05f-44b2-9304-291e12d3edb8-trusted-ca-bundle\") pod \"console-f8c787784-rfdrg\" (UID: \"b36f17f1-a05f-44b2-9304-291e12d3edb8\") " pod="openshift-console/console-f8c787784-rfdrg" Jul 08 18:41:30.874886 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:30.874859 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b36f17f1-a05f-44b2-9304-291e12d3edb8-console-oauth-config\") pod \"console-f8c787784-rfdrg\" (UID: \"b36f17f1-a05f-44b2-9304-291e12d3edb8\") " pod="openshift-console/console-f8c787784-rfdrg" Jul 08 18:41:30.875445 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:30.875421 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b36f17f1-a05f-44b2-9304-291e12d3edb8-console-serving-cert\") pod \"console-f8c787784-rfdrg\" (UID: \"b36f17f1-a05f-44b2-9304-291e12d3edb8\") " pod="openshift-console/console-f8c787784-rfdrg" Jul 08 18:41:30.881103 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:30.881079 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzclk\" (UniqueName: \"kubernetes.io/projected/b36f17f1-a05f-44b2-9304-291e12d3edb8-kube-api-access-vzclk\") pod \"console-f8c787784-rfdrg\" (UID: \"b36f17f1-a05f-44b2-9304-291e12d3edb8\") " pod="openshift-console/console-f8c787784-rfdrg" Jul 08 18:41:31.022823 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:31.022721 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f8c787784-rfdrg" Jul 08 18:41:31.097680 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:41:31.097611 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ee283a3_c9af_4aea_ab7e_448d83b4df89.slice/crio-dae49653d4d792761cd7856e4559d86d4c2f9cd2b300c437c29025f2fe25addb WatchSource:0}: Error finding container dae49653d4d792761cd7856e4559d86d4c2f9cd2b300c437c29025f2fe25addb: Status 404 returned error can't find the container with id dae49653d4d792761cd7856e4559d86d4c2f9cd2b300c437c29025f2fe25addb Jul 08 18:41:31.183469 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:31.183341 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nzczn" event={"ID":"1ee283a3-c9af-4aea-ab7e-448d83b4df89","Type":"ContainerStarted","Data":"dae49653d4d792761cd7856e4559d86d4c2f9cd2b300c437c29025f2fe25addb"} Jul 08 18:41:31.326103 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:31.325519 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-jlr4r"] Jul 08 18:41:31.331634 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:41:31.331549 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce91fd5d_2c4c_4762_88b3_2d3cef665933.slice/crio-699715f41c644410adaba23234a04c8dd236a28adc060a7521e6a74580882eed WatchSource:0}: Error finding container 699715f41c644410adaba23234a04c8dd236a28adc060a7521e6a74580882eed: Status 404 returned error can't find the container with id 699715f41c644410adaba23234a04c8dd236a28adc060a7521e6a74580882eed Jul 08 18:41:31.409829 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:31.409695 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 18:41:31.412622 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:31.412400 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fc9r7"] Jul 08 18:41:31.413754 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:41:31.413730 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod437edb92_3ae1_4a09_8f2f_cc1d583a6753.slice/crio-7b864a017998010cae10834b4c9b097bee87b8393d006d24da98380ca56e8ee9 WatchSource:0}: Error finding container 7b864a017998010cae10834b4c9b097bee87b8393d006d24da98380ca56e8ee9: Status 404 returned error can't find the container with id 7b864a017998010cae10834b4c9b097bee87b8393d006d24da98380ca56e8ee9 Jul 08 18:41:31.415699 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:41:31.415677 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf8759f2_760b_4f41_8579_af54892eeda8.slice/crio-47b4626130bff1bc7b8f5ff2ac48df97763bb9a7e3e3dd4503bc75ac61e6c8b1 WatchSource:0}: Error finding container 47b4626130bff1bc7b8f5ff2ac48df97763bb9a7e3e3dd4503bc75ac61e6c8b1: Status 404 returned error can't find the container with id 47b4626130bff1bc7b8f5ff2ac48df97763bb9a7e3e3dd4503bc75ac61e6c8b1 Jul 08 18:41:31.426836 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:31.426375 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c64665fcd-gqmhp"] Jul 08 18:41:31.430548 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:41:31.430521 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2299a3ad_27c4_4235_a567_83b92bd3e18f.slice/crio-260ee65a6ebccd54cf042526960aed4760065f299009ae97ff3d2432553db0fe WatchSource:0}: Error finding container 260ee65a6ebccd54cf042526960aed4760065f299009ae97ff3d2432553db0fe: Status 404 returned error can't find the container with id 260ee65a6ebccd54cf042526960aed4760065f299009ae97ff3d2432553db0fe Jul 08 18:41:31.518602 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:31.518498 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-579fd557b5-qzlxn"] Jul 08 18:41:31.522637 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:41:31.522608 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod357c8e19_72d5_48e2_bcb6_7f222b54b0e6.slice/crio-62fc57ea2f378c171eec52de565b9cb05484b1f79932b636bd8c8108bda15dd0 WatchSource:0}: Error finding container 62fc57ea2f378c171eec52de565b9cb05484b1f79932b636bd8c8108bda15dd0: Status 404 returned error can't find the container with id 62fc57ea2f378c171eec52de565b9cb05484b1f79932b636bd8c8108bda15dd0 Jul 08 18:41:31.663941 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:31.663729 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6dcfd8bc94-jlv5w"] Jul 08 18:41:31.667949 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:41:31.667915 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4d384fb_fe74_4395_bc16_cffdc9cf9789.slice/crio-00e0aaed1e81e6051a3e21031f83d9a2930de89ebb5146bdfc38ecc09d9a1802 WatchSource:0}: Error finding container 00e0aaed1e81e6051a3e21031f83d9a2930de89ebb5146bdfc38ecc09d9a1802: Status 404 returned error can't find the container with id 00e0aaed1e81e6051a3e21031f83d9a2930de89ebb5146bdfc38ecc09d9a1802 Jul 08 18:41:31.668465 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:31.668397 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dc669"] Jul 08 18:41:31.671835 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:31.671813 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f8c787784-rfdrg"] Jul 08 18:41:31.679450 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:41:31.679364 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb36f17f1_a05f_44b2_9304_291e12d3edb8.slice/crio-ed62431a9e45499cf9aeac2d206ccee72a30cf88cab06e043539780c64cbf78f WatchSource:0}: Error finding container ed62431a9e45499cf9aeac2d206ccee72a30cf88cab06e043539780c64cbf78f: Status 404 returned error can't find the container with id ed62431a9e45499cf9aeac2d206ccee72a30cf88cab06e043539780c64cbf78f Jul 08 18:41:32.192481 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:32.192442 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8475db86b6-bj5jj"] Jul 08 18:41:32.197889 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:32.197853 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-579fd557b5-qzlxn" event={"ID":"357c8e19-72d5-48e2-bcb6-7f222b54b0e6","Type":"ContainerStarted","Data":"62fc57ea2f378c171eec52de565b9cb05484b1f79932b636bd8c8108bda15dd0"} Jul 08 18:41:32.204865 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:32.204792 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fc9r7" event={"ID":"437edb92-3ae1-4a09-8f2f-cc1d583a6753","Type":"ContainerStarted","Data":"7b864a017998010cae10834b4c9b097bee87b8393d006d24da98380ca56e8ee9"} Jul 08 18:41:32.215407 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:32.215248 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-7dwmw" event={"ID":"59decd4d-8b27-4cc1-bb59-a9c9dc29f8e3","Type":"ContainerStarted","Data":"3551ffe27406f9cfc89982f428339250f483580cc70591aa0abc229b4e20c077"} Jul 08 18:41:32.221008 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:32.219445 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-7dwmw" Jul 08 18:41:32.225530 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:32.225472 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-7dwmw" Jul 08 18:41:32.226441 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:32.226368 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dc669" event={"ID":"14cb8ce5-3cb8-4827-8034-d74d7a8d75bf","Type":"ContainerStarted","Data":"b7207c9c7df7016249231675267e733f68f80bc5b83801de4966a95076faa285"} Jul 08 18:41:32.229441 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:32.229348 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"df8759f2-760b-4f41-8579-af54892eeda8","Type":"ContainerStarted","Data":"47b4626130bff1bc7b8f5ff2ac48df97763bb9a7e3e3dd4503bc75ac61e6c8b1"} Jul 08 18:41:32.232461 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:32.232398 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6dcfd8bc94-jlv5w" event={"ID":"b4d384fb-fe74-4395-bc16-cffdc9cf9789","Type":"ContainerStarted","Data":"00e0aaed1e81e6051a3e21031f83d9a2930de89ebb5146bdfc38ecc09d9a1802"} Jul 08 18:41:32.237739 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:32.237164 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c64665fcd-gqmhp" event={"ID":"2299a3ad-27c4-4235-a567-83b92bd3e18f","Type":"ContainerStarted","Data":"20680d5c8d00ceccb5a369c8a5dfc1d87bc7619de624a3a93a5cba263c98a120"} Jul 08 18:41:32.237866 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:32.237788 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c64665fcd-gqmhp" event={"ID":"2299a3ad-27c4-4235-a567-83b92bd3e18f","Type":"ContainerStarted","Data":"260ee65a6ebccd54cf042526960aed4760065f299009ae97ff3d2432553db0fe"} Jul 08 18:41:32.242797 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:32.242610 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f8c787784-rfdrg" event={"ID":"b36f17f1-a05f-44b2-9304-291e12d3edb8","Type":"ContainerStarted","Data":"7b6e26ea26dccb21544ae058ad7d4c566ddc6aaca2a1cd239ba0e2c70818b005"} Jul 08 18:41:32.242797 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:32.242657 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f8c787784-rfdrg" event={"ID":"b36f17f1-a05f-44b2-9304-291e12d3edb8","Type":"ContainerStarted","Data":"ed62431a9e45499cf9aeac2d206ccee72a30cf88cab06e043539780c64cbf78f"} Jul 08 18:41:32.250420 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:32.250284 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jlr4r" event={"ID":"ce91fd5d-2c4c-4762-88b3-2d3cef665933","Type":"ContainerStarted","Data":"163ec22797c7b1365eb60ae6e40679e07be8080ef65509286087f45283e4e9c9"} Jul 08 18:41:32.250420 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:32.250345 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jlr4r" event={"ID":"ce91fd5d-2c4c-4762-88b3-2d3cef665933","Type":"ContainerStarted","Data":"2ae488ace32b11d5e58a8841b802c20580a4af0fc79141d0e785328c71668099"} Jul 08 18:41:32.250420 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:32.250375 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jlr4r" event={"ID":"ce91fd5d-2c4c-4762-88b3-2d3cef665933","Type":"ContainerStarted","Data":"699715f41c644410adaba23234a04c8dd236a28adc060a7521e6a74580882eed"} Jul 08 18:41:32.263333 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:32.263208 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-7dwmw" podStartSLOduration=1.858454753 podStartE2EDuration="22.263188839s" totalImagesPullingTime="20.404734086s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:41:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:41:32.240341056 +0000 UTC m=+135.476639146" watchObservedRunningTime="2026-07-08 18:41:32.263188839 +0000 UTC m=+135.499486922" Jul 08 18:41:32.292445 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:32.290165 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-c64665fcd-gqmhp" podStartSLOduration=8.290144057 podStartE2EDuration="8.290144057s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:41:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:41:32.288062379 +0000 UTC m=+135.524360469" watchObservedRunningTime="2026-07-08 18:41:32.290144057 +0000 UTC m=+135.526442149" Jul 08 18:41:35.334719 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:35.334639 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-c64665fcd-gqmhp" Jul 08 18:41:37.305304 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:37.303758 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dc669" event={"ID":"14cb8ce5-3cb8-4827-8034-d74d7a8d75bf","Type":"ContainerStarted","Data":"3e3583919bc11d78a8d82b67b3d176c4e1833b12e655f3c9920a8d04e7475f65"} Jul 08 18:41:37.306448 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:37.306161 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"df8759f2-760b-4f41-8579-af54892eeda8","Type":"ContainerStarted","Data":"98283d730fd7d0bf93237202b82984dba46fded80da3bb4eef97af3c59b59d3a"} Jul 08 18:41:37.317700 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:37.312236 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jlr4r" event={"ID":"ce91fd5d-2c4c-4762-88b3-2d3cef665933","Type":"ContainerStarted","Data":"305b05b8e32dc6eccc930b31c45868120f2e45c1f30b31bbd29eb24dc4813afa"} Jul 08 18:41:37.320416 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:37.320380 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nzczn" event={"ID":"1ee283a3-c9af-4aea-ab7e-448d83b4df89","Type":"ContainerStarted","Data":"263d0fdd05d50fd886f1bdc68c6b5183226a751efaa7c817489432b2876ccd17"} Jul 08 18:41:37.323142 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:37.323037 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-579fd557b5-qzlxn" event={"ID":"357c8e19-72d5-48e2-bcb6-7f222b54b0e6","Type":"ContainerStarted","Data":"ef15010a74a071dccae823d0ad7cf8357c3c2df08cdc0033f3f8220d37ebbef6"} Jul 08 18:41:37.349048 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:37.349012 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fc9r7" event={"ID":"437edb92-3ae1-4a09-8f2f-cc1d583a6753","Type":"ContainerStarted","Data":"96758949de6c990a0613fc14a8a8e705431878a45de24a51e698ea57c9c9b1b8"} Jul 08 18:41:37.388823 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:37.383593 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-f8c787784-rfdrg" podStartSLOduration=7.383573448 podStartE2EDuration="7.383573448s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:41:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:41:32.315493411 +0000 UTC m=+135.551791504" watchObservedRunningTime="2026-07-08 18:41:37.383573448 +0000 UTC m=+140.619871539" Jul 08 18:41:38.356659 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:38.356615 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-579fd557b5-qzlxn" event={"ID":"357c8e19-72d5-48e2-bcb6-7f222b54b0e6","Type":"ContainerStarted","Data":"e5fb3810aab199f2a1ec50a4da8f313e7824d4097fe7d11b3b379addebf2c32d"} Jul 08 18:41:38.357247 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:38.356710 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-579fd557b5-qzlxn" event={"ID":"357c8e19-72d5-48e2-bcb6-7f222b54b0e6","Type":"ContainerStarted","Data":"347892ce6d06b360768e3d9a4a5957b60ac3f14d0c5d82b89bc663bccfd3260e"} Jul 08 18:41:38.359161 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:38.359131 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fc9r7" event={"ID":"437edb92-3ae1-4a09-8f2f-cc1d583a6753","Type":"ContainerStarted","Data":"d6e78c65f62454843aef26391a9ef0ffd341005dbcd6f75421265ab36916a740"} Jul 08 18:41:38.359285 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:38.359198 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fc9r7" event={"ID":"437edb92-3ae1-4a09-8f2f-cc1d583a6753","Type":"ContainerStarted","Data":"43e7cf9846724229b119f0ced01544c1e89cf6db3bcc11b480bc416e6c2ffd7b"} Jul 08 18:41:38.361504 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:38.361290 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dc669" event={"ID":"14cb8ce5-3cb8-4827-8034-d74d7a8d75bf","Type":"ContainerStarted","Data":"3bb19e0008cba02ba421411fbce81cb79328f75b0fa8e820c618c8a2a82228be"} Jul 08 18:41:38.363079 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:38.363045 2559 generic.go:356] "Generic (PLEG): container finished" podID="df8759f2-760b-4f41-8579-af54892eeda8" containerID="98283d730fd7d0bf93237202b82984dba46fded80da3bb4eef97af3c59b59d3a" exitCode=0 Jul 08 18:41:38.363185 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:38.363139 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"df8759f2-760b-4f41-8579-af54892eeda8","Type":"ContainerDied","Data":"98283d730fd7d0bf93237202b82984dba46fded80da3bb4eef97af3c59b59d3a"} Jul 08 18:41:38.364941 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:38.364917 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6dcfd8bc94-jlv5w" event={"ID":"b4d384fb-fe74-4395-bc16-cffdc9cf9789","Type":"ContainerStarted","Data":"64002aab5e1ac98059d3301bffa62533b72d91f890e788ca903394ea26a92fd5"} Jul 08 18:41:38.366703 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:38.366662 2559 generic.go:356] "Generic (PLEG): container finished" podID="1ee283a3-c9af-4aea-ab7e-448d83b4df89" containerID="263d0fdd05d50fd886f1bdc68c6b5183226a751efaa7c817489432b2876ccd17" exitCode=0 Jul 08 18:41:38.366792 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:38.366725 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nzczn" event={"ID":"1ee283a3-c9af-4aea-ab7e-448d83b4df89","Type":"ContainerDied","Data":"263d0fdd05d50fd886f1bdc68c6b5183226a751efaa7c817489432b2876ccd17"} Jul 08 18:41:38.380174 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:38.380115 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jlr4r" podStartSLOduration=9.769034598 podStartE2EDuration="15.380093713s" totalImagesPullingTime="5.611059115s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:41:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:41:37.448701428 +0000 UTC m=+140.684999531" watchObservedRunningTime="2026-07-08 18:41:38.380093713 +0000 UTC m=+141.616391804" Jul 08 18:41:38.380665 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:38.380621 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fc9r7" podStartSLOduration=10.025706332 podStartE2EDuration="15.380609273s" totalImagesPullingTime="5.354902941s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:41:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:41:38.378386648 +0000 UTC m=+141.614684739" watchObservedRunningTime="2026-07-08 18:41:38.380609273 +0000 UTC m=+141.616907363" Jul 08 18:41:38.399645 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:38.398933 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6dcfd8bc94-jlv5w" podStartSLOduration=4.959578187 podStartE2EDuration="10.398913209s" totalImagesPullingTime="5.439335022s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:41:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:41:38.398798815 +0000 UTC m=+141.635096899" watchObservedRunningTime="2026-07-08 18:41:38.398913209 +0000 UTC m=+141.635211303" Jul 08 18:41:38.419561 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:38.419505 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-dc669" podStartSLOduration=135.328353174 podStartE2EDuration="2m20.41947601s" totalImagesPullingTime="5.091122836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:39:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:41:38.418352277 +0000 UTC m=+141.654650367" watchObservedRunningTime="2026-07-08 18:41:38.41947601 +0000 UTC m=+141.655774101" Jul 08 18:41:39.376041 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:39.375998 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-579fd557b5-qzlxn" event={"ID":"357c8e19-72d5-48e2-bcb6-7f222b54b0e6","Type":"ContainerStarted","Data":"0c201418e68f8aba0933512509c4d6955325d3789b211d69bffc4508f74150e2"} Jul 08 18:41:39.382113 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:39.381278 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nzczn" event={"ID":"1ee283a3-c9af-4aea-ab7e-448d83b4df89","Type":"ContainerStarted","Data":"f06e942936c3cbfb41d39664435c4bd53f876b36a24a55974c17f052a524b709"} Jul 08 18:41:39.382113 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:39.381337 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nzczn" event={"ID":"1ee283a3-c9af-4aea-ab7e-448d83b4df89","Type":"ContainerStarted","Data":"3517a3119205ff3ec8efabb5ce92721f831a88115b630a80554256a57ae879ae"} Jul 08 18:41:39.404863 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:39.403092 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-nzczn" podStartSLOduration=11.702805344 podStartE2EDuration="16.403068623s" totalImagesPullingTime="4.46371554s" totalInitContainerRuntime="236.547739ms" isStatefulPod=true podCreationTimestamp="2026-07-08 18:41:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:41:39.401480364 +0000 UTC m=+142.637778473" watchObservedRunningTime="2026-07-08 18:41:39.403068623 +0000 UTC m=+142.639366719" Jul 08 18:41:40.388284 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:40.388238 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-579fd557b5-qzlxn" event={"ID":"357c8e19-72d5-48e2-bcb6-7f222b54b0e6","Type":"ContainerStarted","Data":"3d3ffc6827958410ba2168654f86e21d08cdabf640dc70255a6bec887cf58991"} Jul 08 18:41:40.388791 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:40.388342 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-579fd557b5-qzlxn" event={"ID":"357c8e19-72d5-48e2-bcb6-7f222b54b0e6","Type":"ContainerStarted","Data":"37a5fa7978ab50d57bc909da69da3436aceeb518630f2fb9981df610d578adb7"} Jul 08 18:41:40.388791 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:40.388553 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-579fd557b5-qzlxn" Jul 08 18:41:40.420463 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:40.420408 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-579fd557b5-qzlxn" podStartSLOduration=8.324822695 podStartE2EDuration="15.420389638s" totalImagesPullingTime="7.095566943s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:41:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:41:40.41854834 +0000 UTC m=+143.654846432" watchObservedRunningTime="2026-07-08 18:41:40.420389638 +0000 UTC m=+143.656687727" Jul 08 18:41:41.023466 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:41.023421 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-f8c787784-rfdrg" Jul 08 18:41:41.023656 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:41.023497 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-f8c787784-rfdrg" Jul 08 18:41:41.028263 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:41.028238 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-f8c787784-rfdrg" Jul 08 18:41:41.396752 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:41.396723 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-f8c787784-rfdrg" Jul 08 18:41:41.446671 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:41.446647 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6bccf57766-gxkvr"] Jul 08 18:41:42.400000 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:42.399964 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"df8759f2-760b-4f41-8579-af54892eeda8","Type":"ContainerStarted","Data":"9370261e94d3f2a96fb802ea7a7adc05e9a50fe65a4e12feabc44affcc8d6393"} Jul 08 18:41:42.400451 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:42.400041 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"df8759f2-760b-4f41-8579-af54892eeda8","Type":"ContainerStarted","Data":"d487bb83bd00b6d4846ff72453693c7fa4dd6a1b74c9e50d220406ba1cd7ca7a"} Jul 08 18:41:42.400451 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:42.400071 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"df8759f2-760b-4f41-8579-af54892eeda8","Type":"ContainerStarted","Data":"71cbe39b06da045bc9b79179052ed976322ab3a4812afa69883f50686ae51b1f"} Jul 08 18:41:42.400451 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:42.400105 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"df8759f2-760b-4f41-8579-af54892eeda8","Type":"ContainerStarted","Data":"1dd94a789bbc2462926658af6b73197cc0b8570408634ea5cb9595fdcdab68a6"} Jul 08 18:41:42.400451 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:42.400130 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"df8759f2-760b-4f41-8579-af54892eeda8","Type":"ContainerStarted","Data":"2657c44bde85c9c53ad52db26c0603b537c420a7cf49e271d6931ab5c1950d4a"} Jul 08 18:41:42.400451 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:42.400152 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"df8759f2-760b-4f41-8579-af54892eeda8","Type":"ContainerStarted","Data":"d241845cfbb28260bb9b4b2dece65e32e98eeb33ed83226f1ae36d5ae63a89c4"} Jul 08 18:41:42.428687 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:42.428619 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=8.971844814 podStartE2EDuration="17.428601524s" totalImagesPullingTime="8.413207931s" totalInitContainerRuntime="43.548779ms" isStatefulPod=false podCreationTimestamp="2026-07-08 18:41:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:41:42.427331264 +0000 UTC m=+145.663629354" watchObservedRunningTime="2026-07-08 18:41:42.428601524 +0000 UTC m=+145.664899605" Jul 08 18:41:46.398786 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:46.398738 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-579fd557b5-qzlxn" Jul 08 18:41:48.666611 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:48.666576 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6dcfd8bc94-jlv5w" Jul 08 18:41:48.667022 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:48.666640 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6dcfd8bc94-jlv5w" Jul 08 18:41:54.454101 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:54.454018 2559 generic.go:356] "Generic (PLEG): container finished" podID="61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e" containerID="04e119e3f54ab287a51eaba575fef4ba6e8e7d083187586bde931441503f17e0" exitCode=0 Jul 08 18:41:54.454101 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:54.454077 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-g989c" event={"ID":"61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e","Type":"ContainerDied","Data":"04e119e3f54ab287a51eaba575fef4ba6e8e7d083187586bde931441503f17e0"} Jul 08 18:41:54.454512 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:54.454402 2559 scope.go:117] "RemoveContainer" containerID="04e119e3f54ab287a51eaba575fef4ba6e8e7d083187586bde931441503f17e0" Jul 08 18:41:55.463063 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:55.463029 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-g989c" event={"ID":"61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e","Type":"ContainerStarted","Data":"215e51ccfc6f21572d79b1b8a89c7fab4dc21ec8bd95fd6e9e21347b2676f016"} Jul 08 18:41:57.222980 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.222935 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" podUID="d317a3e7-440a-4808-8ce1-b748106a5614" containerName="registry" containerID="cri-o://df56974a818e07f0135b3d43d0d30aa03aed3fe9c28995cf503a7866b88ecaca" gracePeriod=30 Jul 08 18:41:57.269746 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.269703 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-c64665fcd-gqmhp" podUID="2299a3ad-27c4-4235-a567-83b92bd3e18f" containerName="console" containerID="cri-o://20680d5c8d00ceccb5a369c8a5dfc1d87bc7619de624a3a93a5cba263c98a120" gracePeriod=15 Jul 08 18:41:57.473339 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.473254 2559 generic.go:356] "Generic (PLEG): container finished" podID="2299a3ad-27c4-4235-a567-83b92bd3e18f" containerID="20680d5c8d00ceccb5a369c8a5dfc1d87bc7619de624a3a93a5cba263c98a120" exitCode=0 Jul 08 18:41:57.473500 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.473328 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c64665fcd-gqmhp" event={"ID":"2299a3ad-27c4-4235-a567-83b92bd3e18f","Type":"ContainerDied","Data":"20680d5c8d00ceccb5a369c8a5dfc1d87bc7619de624a3a93a5cba263c98a120"} Jul 08 18:41:57.474913 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.474888 2559 generic.go:356] "Generic (PLEG): container finished" podID="d317a3e7-440a-4808-8ce1-b748106a5614" containerID="df56974a818e07f0135b3d43d0d30aa03aed3fe9c28995cf503a7866b88ecaca" exitCode=0 Jul 08 18:41:57.475022 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.474942 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" event={"ID":"d317a3e7-440a-4808-8ce1-b748106a5614","Type":"ContainerDied","Data":"df56974a818e07f0135b3d43d0d30aa03aed3fe9c28995cf503a7866b88ecaca"} Jul 08 18:41:57.566998 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.566962 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" Jul 08 18:41:57.583069 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.583041 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c64665fcd-gqmhp" Jul 08 18:41:57.644435 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.644401 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d317a3e7-440a-4808-8ce1-b748106a5614-registry-certificates\") pod \"d317a3e7-440a-4808-8ce1-b748106a5614\" (UID: \"d317a3e7-440a-4808-8ce1-b748106a5614\") " Jul 08 18:41:57.644604 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.644459 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d317a3e7-440a-4808-8ce1-b748106a5614-bound-sa-token\") pod \"d317a3e7-440a-4808-8ce1-b748106a5614\" (UID: \"d317a3e7-440a-4808-8ce1-b748106a5614\") " Jul 08 18:41:57.644604 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.644488 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d317a3e7-440a-4808-8ce1-b748106a5614-image-registry-private-configuration\") pod \"d317a3e7-440a-4808-8ce1-b748106a5614\" (UID: \"d317a3e7-440a-4808-8ce1-b748106a5614\") " Jul 08 18:41:57.644604 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.644511 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d317a3e7-440a-4808-8ce1-b748106a5614-registry-tls\") pod \"d317a3e7-440a-4808-8ce1-b748106a5614\" (UID: \"d317a3e7-440a-4808-8ce1-b748106a5614\") " Jul 08 18:41:57.644604 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.644544 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d317a3e7-440a-4808-8ce1-b748106a5614-trusted-ca\") pod \"d317a3e7-440a-4808-8ce1-b748106a5614\" (UID: \"d317a3e7-440a-4808-8ce1-b748106a5614\") " Jul 08 18:41:57.644604 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.644564 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d317a3e7-440a-4808-8ce1-b748106a5614-ca-trust-extracted\") pod \"d317a3e7-440a-4808-8ce1-b748106a5614\" (UID: \"d317a3e7-440a-4808-8ce1-b748106a5614\") " Jul 08 18:41:57.644913 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.644611 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d317a3e7-440a-4808-8ce1-b748106a5614-installation-pull-secrets\") pod \"d317a3e7-440a-4808-8ce1-b748106a5614\" (UID: \"d317a3e7-440a-4808-8ce1-b748106a5614\") " Jul 08 18:41:57.644913 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.644659 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8lh7z\" (UniqueName: \"kubernetes.io/projected/d317a3e7-440a-4808-8ce1-b748106a5614-kube-api-access-8lh7z\") pod \"d317a3e7-440a-4808-8ce1-b748106a5614\" (UID: \"d317a3e7-440a-4808-8ce1-b748106a5614\") " Jul 08 18:41:57.645012 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.644945 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d317a3e7-440a-4808-8ce1-b748106a5614-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "d317a3e7-440a-4808-8ce1-b748106a5614" (UID: "d317a3e7-440a-4808-8ce1-b748106a5614"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:41:57.645093 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.645055 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d317a3e7-440a-4808-8ce1-b748106a5614-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "d317a3e7-440a-4808-8ce1-b748106a5614" (UID: "d317a3e7-440a-4808-8ce1-b748106a5614"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:41:57.647163 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.647134 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d317a3e7-440a-4808-8ce1-b748106a5614-kube-api-access-8lh7z" (OuterVolumeSpecName: "kube-api-access-8lh7z") pod "d317a3e7-440a-4808-8ce1-b748106a5614" (UID: "d317a3e7-440a-4808-8ce1-b748106a5614"). InnerVolumeSpecName "kube-api-access-8lh7z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:41:57.647287 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.647167 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d317a3e7-440a-4808-8ce1-b748106a5614-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d317a3e7-440a-4808-8ce1-b748106a5614" (UID: "d317a3e7-440a-4808-8ce1-b748106a5614"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:41:57.647352 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.647319 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d317a3e7-440a-4808-8ce1-b748106a5614-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "d317a3e7-440a-4808-8ce1-b748106a5614" (UID: "d317a3e7-440a-4808-8ce1-b748106a5614"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:41:57.647442 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.647424 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d317a3e7-440a-4808-8ce1-b748106a5614-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "d317a3e7-440a-4808-8ce1-b748106a5614" (UID: "d317a3e7-440a-4808-8ce1-b748106a5614"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:41:57.647494 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.647446 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d317a3e7-440a-4808-8ce1-b748106a5614-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "d317a3e7-440a-4808-8ce1-b748106a5614" (UID: "d317a3e7-440a-4808-8ce1-b748106a5614"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:41:57.653359 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.653333 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d317a3e7-440a-4808-8ce1-b748106a5614-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "d317a3e7-440a-4808-8ce1-b748106a5614" (UID: "d317a3e7-440a-4808-8ce1-b748106a5614"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:41:57.745582 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.745484 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2299a3ad-27c4-4235-a567-83b92bd3e18f-trusted-ca-bundle\") pod \"2299a3ad-27c4-4235-a567-83b92bd3e18f\" (UID: \"2299a3ad-27c4-4235-a567-83b92bd3e18f\") " Jul 08 18:41:57.745582 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.745535 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2299a3ad-27c4-4235-a567-83b92bd3e18f-console-serving-cert\") pod \"2299a3ad-27c4-4235-a567-83b92bd3e18f\" (UID: \"2299a3ad-27c4-4235-a567-83b92bd3e18f\") " Jul 08 18:41:57.745582 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.745571 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2299a3ad-27c4-4235-a567-83b92bd3e18f-console-config\") pod \"2299a3ad-27c4-4235-a567-83b92bd3e18f\" (UID: \"2299a3ad-27c4-4235-a567-83b92bd3e18f\") " Jul 08 18:41:57.745881 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.745629 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2299a3ad-27c4-4235-a567-83b92bd3e18f-console-oauth-config\") pod \"2299a3ad-27c4-4235-a567-83b92bd3e18f\" (UID: \"2299a3ad-27c4-4235-a567-83b92bd3e18f\") " Jul 08 18:41:57.745881 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.745660 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2299a3ad-27c4-4235-a567-83b92bd3e18f-oauth-serving-cert\") pod \"2299a3ad-27c4-4235-a567-83b92bd3e18f\" (UID: \"2299a3ad-27c4-4235-a567-83b92bd3e18f\") " Jul 08 18:41:57.745881 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.745684 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2299a3ad-27c4-4235-a567-83b92bd3e18f-service-ca\") pod \"2299a3ad-27c4-4235-a567-83b92bd3e18f\" (UID: \"2299a3ad-27c4-4235-a567-83b92bd3e18f\") " Jul 08 18:41:57.745881 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.745707 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gpqv6\" (UniqueName: \"kubernetes.io/projected/2299a3ad-27c4-4235-a567-83b92bd3e18f-kube-api-access-gpqv6\") pod \"2299a3ad-27c4-4235-a567-83b92bd3e18f\" (UID: \"2299a3ad-27c4-4235-a567-83b92bd3e18f\") " Jul 08 18:41:57.746058 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.746038 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8lh7z\" (UniqueName: \"kubernetes.io/projected/d317a3e7-440a-4808-8ce1-b748106a5614-kube-api-access-8lh7z\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:41:57.746058 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.746031 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2299a3ad-27c4-4235-a567-83b92bd3e18f-console-config" (OuterVolumeSpecName: "console-config") pod "2299a3ad-27c4-4235-a567-83b92bd3e18f" (UID: "2299a3ad-27c4-4235-a567-83b92bd3e18f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:41:57.746159 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.746058 2559 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d317a3e7-440a-4808-8ce1-b748106a5614-registry-certificates\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:41:57.746159 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.746077 2559 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d317a3e7-440a-4808-8ce1-b748106a5614-bound-sa-token\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:41:57.746159 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.746093 2559 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d317a3e7-440a-4808-8ce1-b748106a5614-image-registry-private-configuration\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:41:57.746159 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.746106 2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d317a3e7-440a-4808-8ce1-b748106a5614-registry-tls\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:41:57.746159 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.746119 2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d317a3e7-440a-4808-8ce1-b748106a5614-trusted-ca\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:41:57.746159 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.746132 2559 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d317a3e7-440a-4808-8ce1-b748106a5614-ca-trust-extracted\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:41:57.746159 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.746145 2559 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d317a3e7-440a-4808-8ce1-b748106a5614-installation-pull-secrets\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:41:57.746375 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.746158 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2299a3ad-27c4-4235-a567-83b92bd3e18f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2299a3ad-27c4-4235-a567-83b92bd3e18f" (UID: "2299a3ad-27c4-4235-a567-83b92bd3e18f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:41:57.746375 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.745886 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2299a3ad-27c4-4235-a567-83b92bd3e18f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "2299a3ad-27c4-4235-a567-83b92bd3e18f" (UID: "2299a3ad-27c4-4235-a567-83b92bd3e18f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:41:57.746375 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.746200 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2299a3ad-27c4-4235-a567-83b92bd3e18f-service-ca" (OuterVolumeSpecName: "service-ca") pod "2299a3ad-27c4-4235-a567-83b92bd3e18f" (UID: "2299a3ad-27c4-4235-a567-83b92bd3e18f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:41:57.747974 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.747949 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2299a3ad-27c4-4235-a567-83b92bd3e18f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2299a3ad-27c4-4235-a567-83b92bd3e18f" (UID: "2299a3ad-27c4-4235-a567-83b92bd3e18f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:41:57.748081 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.747972 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2299a3ad-27c4-4235-a567-83b92bd3e18f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2299a3ad-27c4-4235-a567-83b92bd3e18f" (UID: "2299a3ad-27c4-4235-a567-83b92bd3e18f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:41:57.748081 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.748030 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2299a3ad-27c4-4235-a567-83b92bd3e18f-kube-api-access-gpqv6" (OuterVolumeSpecName: "kube-api-access-gpqv6") pod "2299a3ad-27c4-4235-a567-83b92bd3e18f" (UID: "2299a3ad-27c4-4235-a567-83b92bd3e18f"). InnerVolumeSpecName "kube-api-access-gpqv6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:41:57.847206 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.847172 2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2299a3ad-27c4-4235-a567-83b92bd3e18f-trusted-ca-bundle\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:41:57.847206 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.847199 2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2299a3ad-27c4-4235-a567-83b92bd3e18f-console-serving-cert\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:41:57.847206 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.847210 2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2299a3ad-27c4-4235-a567-83b92bd3e18f-console-config\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:41:57.847424 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.847219 2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2299a3ad-27c4-4235-a567-83b92bd3e18f-console-oauth-config\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:41:57.847424 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.847227 2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2299a3ad-27c4-4235-a567-83b92bd3e18f-oauth-serving-cert\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:41:57.847424 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.847237 2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2299a3ad-27c4-4235-a567-83b92bd3e18f-service-ca\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:41:57.847424 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:57.847246 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gpqv6\" (UniqueName: \"kubernetes.io/projected/2299a3ad-27c4-4235-a567-83b92bd3e18f-kube-api-access-gpqv6\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:41:58.480258 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:58.480224 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c64665fcd-gqmhp" Jul 08 18:41:58.480718 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:58.480226 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c64665fcd-gqmhp" event={"ID":"2299a3ad-27c4-4235-a567-83b92bd3e18f","Type":"ContainerDied","Data":"260ee65a6ebccd54cf042526960aed4760065f299009ae97ff3d2432553db0fe"} Jul 08 18:41:58.480718 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:58.480406 2559 scope.go:117] "RemoveContainer" containerID="20680d5c8d00ceccb5a369c8a5dfc1d87bc7619de624a3a93a5cba263c98a120" Jul 08 18:41:58.481692 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:58.481655 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" Jul 08 18:41:58.481753 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:58.481694 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8475db86b6-bj5jj" event={"ID":"d317a3e7-440a-4808-8ce1-b748106a5614","Type":"ContainerDied","Data":"c725c86be1bb19f5f2573aa4974039f166bf52e06579decc18d2c5b8947c720e"} Jul 08 18:41:58.490824 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:58.490808 2559 scope.go:117] "RemoveContainer" containerID="df56974a818e07f0135b3d43d0d30aa03aed3fe9c28995cf503a7866b88ecaca" Jul 08 18:41:58.500497 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:58.500430 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8475db86b6-bj5jj"] Jul 08 18:41:58.504645 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:58.504623 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-8475db86b6-bj5jj"] Jul 08 18:41:58.516206 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:58.516183 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c64665fcd-gqmhp"] Jul 08 18:41:58.518736 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:58.518712 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-c64665fcd-gqmhp"] Jul 08 18:41:59.687190 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:59.687153 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2299a3ad-27c4-4235-a567-83b92bd3e18f" path="/var/lib/kubelet/pods/2299a3ad-27c4-4235-a567-83b92bd3e18f/volumes" Jul 08 18:41:59.687593 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:41:59.687579 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d317a3e7-440a-4808-8ce1-b748106a5614" path="/var/lib/kubelet/pods/d317a3e7-440a-4808-8ce1-b748106a5614/volumes" Jul 08 18:42:06.474101 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:06.474061 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6bccf57766-gxkvr" podUID="3fa525fd-fa8e-4574-9203-eae015e8459f" containerName="console" containerID="cri-o://204f331980b8f30111732bc493e350ce5c43709fe6760e8f746f91b72d68141a" gracePeriod=15 Jul 08 18:42:06.743795 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:06.743710 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bccf57766-gxkvr" Jul 08 18:42:06.923660 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:06.923621 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3fa525fd-fa8e-4574-9203-eae015e8459f-console-oauth-config\") pod \"3fa525fd-fa8e-4574-9203-eae015e8459f\" (UID: \"3fa525fd-fa8e-4574-9203-eae015e8459f\") " Jul 08 18:42:06.923890 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:06.923708 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qtlt4\" (UniqueName: \"kubernetes.io/projected/3fa525fd-fa8e-4574-9203-eae015e8459f-kube-api-access-qtlt4\") pod \"3fa525fd-fa8e-4574-9203-eae015e8459f\" (UID: \"3fa525fd-fa8e-4574-9203-eae015e8459f\") " Jul 08 18:42:06.923890 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:06.923737 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3fa525fd-fa8e-4574-9203-eae015e8459f-service-ca\") pod \"3fa525fd-fa8e-4574-9203-eae015e8459f\" (UID: \"3fa525fd-fa8e-4574-9203-eae015e8459f\") " Jul 08 18:42:06.923890 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:06.923849 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3fa525fd-fa8e-4574-9203-eae015e8459f-oauth-serving-cert\") pod \"3fa525fd-fa8e-4574-9203-eae015e8459f\" (UID: \"3fa525fd-fa8e-4574-9203-eae015e8459f\") " Jul 08 18:42:06.923890 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:06.923882 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3fa525fd-fa8e-4574-9203-eae015e8459f-console-serving-cert\") pod \"3fa525fd-fa8e-4574-9203-eae015e8459f\" (UID: \"3fa525fd-fa8e-4574-9203-eae015e8459f\") " Jul 08 18:42:06.924111 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:06.924030 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3fa525fd-fa8e-4574-9203-eae015e8459f-console-config\") pod \"3fa525fd-fa8e-4574-9203-eae015e8459f\" (UID: \"3fa525fd-fa8e-4574-9203-eae015e8459f\") " Jul 08 18:42:06.924166 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:06.924147 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fa525fd-fa8e-4574-9203-eae015e8459f-service-ca" (OuterVolumeSpecName: "service-ca") pod "3fa525fd-fa8e-4574-9203-eae015e8459f" (UID: "3fa525fd-fa8e-4574-9203-eae015e8459f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:42:06.924332 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:06.924314 2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3fa525fd-fa8e-4574-9203-eae015e8459f-service-ca\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:42:06.924332 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:06.924313 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fa525fd-fa8e-4574-9203-eae015e8459f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3fa525fd-fa8e-4574-9203-eae015e8459f" (UID: "3fa525fd-fa8e-4574-9203-eae015e8459f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:42:06.924450 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:06.924383 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fa525fd-fa8e-4574-9203-eae015e8459f-console-config" (OuterVolumeSpecName: "console-config") pod "3fa525fd-fa8e-4574-9203-eae015e8459f" (UID: "3fa525fd-fa8e-4574-9203-eae015e8459f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:42:06.926010 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:06.925985 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fa525fd-fa8e-4574-9203-eae015e8459f-kube-api-access-qtlt4" (OuterVolumeSpecName: "kube-api-access-qtlt4") pod "3fa525fd-fa8e-4574-9203-eae015e8459f" (UID: "3fa525fd-fa8e-4574-9203-eae015e8459f"). InnerVolumeSpecName "kube-api-access-qtlt4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:42:06.926113 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:06.926015 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fa525fd-fa8e-4574-9203-eae015e8459f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3fa525fd-fa8e-4574-9203-eae015e8459f" (UID: "3fa525fd-fa8e-4574-9203-eae015e8459f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:42:06.926113 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:06.926050 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fa525fd-fa8e-4574-9203-eae015e8459f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3fa525fd-fa8e-4574-9203-eae015e8459f" (UID: "3fa525fd-fa8e-4574-9203-eae015e8459f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:42:07.025067 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:07.024970 2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3fa525fd-fa8e-4574-9203-eae015e8459f-oauth-serving-cert\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:42:07.025067 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:07.025005 2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3fa525fd-fa8e-4574-9203-eae015e8459f-console-serving-cert\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:42:07.025067 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:07.025018 2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3fa525fd-fa8e-4574-9203-eae015e8459f-console-config\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:42:07.025067 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:07.025032 2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3fa525fd-fa8e-4574-9203-eae015e8459f-console-oauth-config\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:42:07.025067 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:07.025044 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qtlt4\" (UniqueName: \"kubernetes.io/projected/3fa525fd-fa8e-4574-9203-eae015e8459f-kube-api-access-qtlt4\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:42:07.520223 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:07.520187 2559 generic.go:356] "Generic (PLEG): container finished" podID="3fa525fd-fa8e-4574-9203-eae015e8459f" containerID="204f331980b8f30111732bc493e350ce5c43709fe6760e8f746f91b72d68141a" exitCode=0 Jul 08 18:42:07.520621 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:07.520229 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bccf57766-gxkvr" Jul 08 18:42:07.520621 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:07.520255 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bccf57766-gxkvr" event={"ID":"3fa525fd-fa8e-4574-9203-eae015e8459f","Type":"ContainerDied","Data":"204f331980b8f30111732bc493e350ce5c43709fe6760e8f746f91b72d68141a"} Jul 08 18:42:07.520621 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:07.520295 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bccf57766-gxkvr" event={"ID":"3fa525fd-fa8e-4574-9203-eae015e8459f","Type":"ContainerDied","Data":"e17b6a3f7c9036c02e21d91a3fbb68a340c61a229aebc22eefbeb291d9d4af87"} Jul 08 18:42:07.520621 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:07.520315 2559 scope.go:117] "RemoveContainer" containerID="204f331980b8f30111732bc493e350ce5c43709fe6760e8f746f91b72d68141a" Jul 08 18:42:07.528699 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:07.528682 2559 scope.go:117] "RemoveContainer" containerID="204f331980b8f30111732bc493e350ce5c43709fe6760e8f746f91b72d68141a" Jul 08 18:42:07.528999 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:42:07.528968 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"204f331980b8f30111732bc493e350ce5c43709fe6760e8f746f91b72d68141a\": container with ID starting with 204f331980b8f30111732bc493e350ce5c43709fe6760e8f746f91b72d68141a not found: ID does not exist" containerID="204f331980b8f30111732bc493e350ce5c43709fe6760e8f746f91b72d68141a" Jul 08 18:42:07.529077 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:07.529007 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"204f331980b8f30111732bc493e350ce5c43709fe6760e8f746f91b72d68141a"} err="failed to get container status \"204f331980b8f30111732bc493e350ce5c43709fe6760e8f746f91b72d68141a\": rpc error: code = NotFound desc = could not find container \"204f331980b8f30111732bc493e350ce5c43709fe6760e8f746f91b72d68141a\": container with ID starting with 204f331980b8f30111732bc493e350ce5c43709fe6760e8f746f91b72d68141a not found: ID does not exist" Jul 08 18:42:07.544586 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:07.544558 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6bccf57766-gxkvr"] Jul 08 18:42:07.546718 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:07.546682 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6bccf57766-gxkvr"] Jul 08 18:42:07.691114 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:07.691080 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fa525fd-fa8e-4574-9203-eae015e8459f" path="/var/lib/kubelet/pods/3fa525fd-fa8e-4574-9203-eae015e8459f/volumes" Jul 08 18:42:08.672498 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:08.672467 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6dcfd8bc94-jlv5w" Jul 08 18:42:08.676327 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:08.676302 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6dcfd8bc94-jlv5w" Jul 08 18:42:44.422987 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:44.422949 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 18:42:44.423434 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:44.423225 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="df8759f2-760b-4f41-8579-af54892eeda8" containerName="alertmanager" containerID="cri-o://d241845cfbb28260bb9b4b2dece65e32e98eeb33ed83226f1ae36d5ae63a89c4" gracePeriod=120 Jul 08 18:42:44.423434 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:44.423299 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="df8759f2-760b-4f41-8579-af54892eeda8" containerName="kube-rbac-proxy-metric" containerID="cri-o://d487bb83bd00b6d4846ff72453693c7fa4dd6a1b74c9e50d220406ba1cd7ca7a" gracePeriod=120 Jul 08 18:42:44.423434 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:44.423323 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="df8759f2-760b-4f41-8579-af54892eeda8" containerName="kube-rbac-proxy-web" containerID="cri-o://1dd94a789bbc2462926658af6b73197cc0b8570408634ea5cb9595fdcdab68a6" gracePeriod=120 Jul 08 18:42:44.423434 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:44.423349 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="df8759f2-760b-4f41-8579-af54892eeda8" containerName="prom-label-proxy" containerID="cri-o://9370261e94d3f2a96fb802ea7a7adc05e9a50fe65a4e12feabc44affcc8d6393" gracePeriod=120 Jul 08 18:42:44.423434 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:44.423395 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="df8759f2-760b-4f41-8579-af54892eeda8" containerName="config-reloader" containerID="cri-o://2657c44bde85c9c53ad52db26c0603b537c420a7cf49e271d6931ab5c1950d4a" gracePeriod=120 Jul 08 18:42:44.423726 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:44.423394 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="df8759f2-760b-4f41-8579-af54892eeda8" containerName="kube-rbac-proxy" containerID="cri-o://71cbe39b06da045bc9b79179052ed976322ab3a4812afa69883f50686ae51b1f" gracePeriod=120 Jul 08 18:42:44.677391 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:44.677303 2559 generic.go:356] "Generic (PLEG): container finished" podID="df8759f2-760b-4f41-8579-af54892eeda8" containerID="9370261e94d3f2a96fb802ea7a7adc05e9a50fe65a4e12feabc44affcc8d6393" exitCode=0 Jul 08 18:42:44.677391 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:44.677331 2559 generic.go:356] "Generic (PLEG): container finished" podID="df8759f2-760b-4f41-8579-af54892eeda8" containerID="d487bb83bd00b6d4846ff72453693c7fa4dd6a1b74c9e50d220406ba1cd7ca7a" exitCode=0 Jul 08 18:42:44.677391 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:44.677341 2559 generic.go:356] "Generic (PLEG): container finished" podID="df8759f2-760b-4f41-8579-af54892eeda8" containerID="71cbe39b06da045bc9b79179052ed976322ab3a4812afa69883f50686ae51b1f" exitCode=0 Jul 08 18:42:44.677391 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:44.677348 2559 generic.go:356] "Generic (PLEG): container finished" podID="df8759f2-760b-4f41-8579-af54892eeda8" containerID="2657c44bde85c9c53ad52db26c0603b537c420a7cf49e271d6931ab5c1950d4a" exitCode=0 Jul 08 18:42:44.677391 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:44.677356 2559 generic.go:356] "Generic (PLEG): container finished" podID="df8759f2-760b-4f41-8579-af54892eeda8" containerID="d241845cfbb28260bb9b4b2dece65e32e98eeb33ed83226f1ae36d5ae63a89c4" exitCode=0 Jul 08 18:42:44.677391 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:44.677363 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"df8759f2-760b-4f41-8579-af54892eeda8","Type":"ContainerDied","Data":"9370261e94d3f2a96fb802ea7a7adc05e9a50fe65a4e12feabc44affcc8d6393"} Jul 08 18:42:44.677673 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:44.677450 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"df8759f2-760b-4f41-8579-af54892eeda8","Type":"ContainerDied","Data":"d487bb83bd00b6d4846ff72453693c7fa4dd6a1b74c9e50d220406ba1cd7ca7a"} Jul 08 18:42:44.677673 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:44.677484 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"df8759f2-760b-4f41-8579-af54892eeda8","Type":"ContainerDied","Data":"71cbe39b06da045bc9b79179052ed976322ab3a4812afa69883f50686ae51b1f"} Jul 08 18:42:44.677673 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:44.677509 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"df8759f2-760b-4f41-8579-af54892eeda8","Type":"ContainerDied","Data":"2657c44bde85c9c53ad52db26c0603b537c420a7cf49e271d6931ab5c1950d4a"} Jul 08 18:42:44.677673 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:44.677533 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"df8759f2-760b-4f41-8579-af54892eeda8","Type":"ContainerDied","Data":"d241845cfbb28260bb9b4b2dece65e32e98eeb33ed83226f1ae36d5ae63a89c4"} Jul 08 18:42:45.675899 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.675860 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:45.684161 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.684126 2559 generic.go:356] "Generic (PLEG): container finished" podID="df8759f2-760b-4f41-8579-af54892eeda8" containerID="1dd94a789bbc2462926658af6b73197cc0b8570408634ea5cb9595fdcdab68a6" exitCode=0 Jul 08 18:42:45.684330 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.684254 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:45.687465 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.687436 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"df8759f2-760b-4f41-8579-af54892eeda8","Type":"ContainerDied","Data":"1dd94a789bbc2462926658af6b73197cc0b8570408634ea5cb9595fdcdab68a6"} Jul 08 18:42:45.687575 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.687482 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"df8759f2-760b-4f41-8579-af54892eeda8","Type":"ContainerDied","Data":"47b4626130bff1bc7b8f5ff2ac48df97763bb9a7e3e3dd4503bc75ac61e6c8b1"} Jul 08 18:42:45.687575 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.687516 2559 scope.go:117] "RemoveContainer" containerID="9370261e94d3f2a96fb802ea7a7adc05e9a50fe65a4e12feabc44affcc8d6393" Jul 08 18:42:45.697938 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.697921 2559 scope.go:117] "RemoveContainer" containerID="d487bb83bd00b6d4846ff72453693c7fa4dd6a1b74c9e50d220406ba1cd7ca7a" Jul 08 18:42:45.705617 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.705568 2559 scope.go:117] "RemoveContainer" containerID="71cbe39b06da045bc9b79179052ed976322ab3a4812afa69883f50686ae51b1f" Jul 08 18:42:45.713906 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.713884 2559 scope.go:117] "RemoveContainer" containerID="1dd94a789bbc2462926658af6b73197cc0b8570408634ea5cb9595fdcdab68a6" Jul 08 18:42:45.722902 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.722879 2559 scope.go:117] "RemoveContainer" containerID="2657c44bde85c9c53ad52db26c0603b537c420a7cf49e271d6931ab5c1950d4a" Jul 08 18:42:45.730904 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.730883 2559 scope.go:117] "RemoveContainer" containerID="d241845cfbb28260bb9b4b2dece65e32e98eeb33ed83226f1ae36d5ae63a89c4" Jul 08 18:42:45.733104 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.733081 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-web-config\") pod \"df8759f2-760b-4f41-8579-af54892eeda8\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " Jul 08 18:42:45.733217 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.733120 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"df8759f2-760b-4f41-8579-af54892eeda8\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " Jul 08 18:42:45.733217 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.733157 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/df8759f2-760b-4f41-8579-af54892eeda8-config-out\") pod \"df8759f2-760b-4f41-8579-af54892eeda8\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " Jul 08 18:42:45.733217 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.733207 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-config-volume\") pod \"df8759f2-760b-4f41-8579-af54892eeda8\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " Jul 08 18:42:45.733368 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.733239 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-cluster-tls-config\") pod \"df8759f2-760b-4f41-8579-af54892eeda8\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " Jul 08 18:42:45.733368 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.733278 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-secret-alertmanager-kube-rbac-proxy\") pod \"df8759f2-760b-4f41-8579-af54892eeda8\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " Jul 08 18:42:45.733368 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.733318 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/df8759f2-760b-4f41-8579-af54892eeda8-metrics-client-ca\") pod \"df8759f2-760b-4f41-8579-af54892eeda8\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " Jul 08 18:42:45.733517 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.733368 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df8759f2-760b-4f41-8579-af54892eeda8-alertmanager-trusted-ca-bundle\") pod \"df8759f2-760b-4f41-8579-af54892eeda8\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " Jul 08 18:42:45.733517 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.733407 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-secret-alertmanager-main-tls\") pod \"df8759f2-760b-4f41-8579-af54892eeda8\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " Jul 08 18:42:45.733517 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.733485 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-txtn5\" (UniqueName: \"kubernetes.io/projected/df8759f2-760b-4f41-8579-af54892eeda8-kube-api-access-txtn5\") pod \"df8759f2-760b-4f41-8579-af54892eeda8\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " Jul 08 18:42:45.733662 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.733521 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/df8759f2-760b-4f41-8579-af54892eeda8-alertmanager-main-db\") pod \"df8759f2-760b-4f41-8579-af54892eeda8\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " Jul 08 18:42:45.733662 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.733566 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/df8759f2-760b-4f41-8579-af54892eeda8-tls-assets\") pod \"df8759f2-760b-4f41-8579-af54892eeda8\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " Jul 08 18:42:45.733662 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.733591 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-secret-alertmanager-kube-rbac-proxy-web\") pod \"df8759f2-760b-4f41-8579-af54892eeda8\" (UID: \"df8759f2-760b-4f41-8579-af54892eeda8\") " Jul 08 18:42:45.734173 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.734146 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df8759f2-760b-4f41-8579-af54892eeda8-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "df8759f2-760b-4f41-8579-af54892eeda8" (UID: "df8759f2-760b-4f41-8579-af54892eeda8"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:42:45.734543 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.734407 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df8759f2-760b-4f41-8579-af54892eeda8-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "df8759f2-760b-4f41-8579-af54892eeda8" (UID: "df8759f2-760b-4f41-8579-af54892eeda8"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:42:45.735172 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.734691 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df8759f2-760b-4f41-8579-af54892eeda8-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "df8759f2-760b-4f41-8579-af54892eeda8" (UID: "df8759f2-760b-4f41-8579-af54892eeda8"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:42:45.736618 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.736587 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "df8759f2-760b-4f41-8579-af54892eeda8" (UID: "df8759f2-760b-4f41-8579-af54892eeda8"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:42:45.738061 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.738035 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df8759f2-760b-4f41-8579-af54892eeda8-kube-api-access-txtn5" (OuterVolumeSpecName: "kube-api-access-txtn5") pod "df8759f2-760b-4f41-8579-af54892eeda8" (UID: "df8759f2-760b-4f41-8579-af54892eeda8"). InnerVolumeSpecName "kube-api-access-txtn5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:42:45.738188 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.738066 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-config-volume" (OuterVolumeSpecName: "config-volume") pod "df8759f2-760b-4f41-8579-af54892eeda8" (UID: "df8759f2-760b-4f41-8579-af54892eeda8"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:42:45.738447 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.738412 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "df8759f2-760b-4f41-8579-af54892eeda8" (UID: "df8759f2-760b-4f41-8579-af54892eeda8"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:42:45.738628 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.738602 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "df8759f2-760b-4f41-8579-af54892eeda8" (UID: "df8759f2-760b-4f41-8579-af54892eeda8"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:42:45.739107 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.739079 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df8759f2-760b-4f41-8579-af54892eeda8-config-out" (OuterVolumeSpecName: "config-out") pod "df8759f2-760b-4f41-8579-af54892eeda8" (UID: "df8759f2-760b-4f41-8579-af54892eeda8"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:42:45.739534 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.739504 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "df8759f2-760b-4f41-8579-af54892eeda8" (UID: "df8759f2-760b-4f41-8579-af54892eeda8"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:42:45.740298 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.740263 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df8759f2-760b-4f41-8579-af54892eeda8-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "df8759f2-760b-4f41-8579-af54892eeda8" (UID: "df8759f2-760b-4f41-8579-af54892eeda8"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:42:45.741575 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.741554 2559 scope.go:117] "RemoveContainer" containerID="98283d730fd7d0bf93237202b82984dba46fded80da3bb4eef97af3c59b59d3a" Jul 08 18:42:45.742991 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.742889 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "df8759f2-760b-4f41-8579-af54892eeda8" (UID: "df8759f2-760b-4f41-8579-af54892eeda8"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:42:45.750326 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.750295 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-web-config" (OuterVolumeSpecName: "web-config") pod "df8759f2-760b-4f41-8579-af54892eeda8" (UID: "df8759f2-760b-4f41-8579-af54892eeda8"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:42:45.757503 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.757484 2559 scope.go:117] "RemoveContainer" containerID="9370261e94d3f2a96fb802ea7a7adc05e9a50fe65a4e12feabc44affcc8d6393" Jul 08 18:42:45.757912 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:42:45.757891 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9370261e94d3f2a96fb802ea7a7adc05e9a50fe65a4e12feabc44affcc8d6393\": container with ID starting with 9370261e94d3f2a96fb802ea7a7adc05e9a50fe65a4e12feabc44affcc8d6393 not found: ID does not exist" containerID="9370261e94d3f2a96fb802ea7a7adc05e9a50fe65a4e12feabc44affcc8d6393" Jul 08 18:42:45.757983 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.757921 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9370261e94d3f2a96fb802ea7a7adc05e9a50fe65a4e12feabc44affcc8d6393"} err="failed to get container status \"9370261e94d3f2a96fb802ea7a7adc05e9a50fe65a4e12feabc44affcc8d6393\": rpc error: code = NotFound desc = could not find container \"9370261e94d3f2a96fb802ea7a7adc05e9a50fe65a4e12feabc44affcc8d6393\": container with ID starting with 9370261e94d3f2a96fb802ea7a7adc05e9a50fe65a4e12feabc44affcc8d6393 not found: ID does not exist" Jul 08 18:42:45.757983 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.757943 2559 scope.go:117] "RemoveContainer" containerID="d487bb83bd00b6d4846ff72453693c7fa4dd6a1b74c9e50d220406ba1cd7ca7a" Jul 08 18:42:45.758226 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:42:45.758211 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d487bb83bd00b6d4846ff72453693c7fa4dd6a1b74c9e50d220406ba1cd7ca7a\": container with ID starting with d487bb83bd00b6d4846ff72453693c7fa4dd6a1b74c9e50d220406ba1cd7ca7a not found: ID does not exist" containerID="d487bb83bd00b6d4846ff72453693c7fa4dd6a1b74c9e50d220406ba1cd7ca7a" Jul 08 18:42:45.758264 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.758233 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d487bb83bd00b6d4846ff72453693c7fa4dd6a1b74c9e50d220406ba1cd7ca7a"} err="failed to get container status \"d487bb83bd00b6d4846ff72453693c7fa4dd6a1b74c9e50d220406ba1cd7ca7a\": rpc error: code = NotFound desc = could not find container \"d487bb83bd00b6d4846ff72453693c7fa4dd6a1b74c9e50d220406ba1cd7ca7a\": container with ID starting with d487bb83bd00b6d4846ff72453693c7fa4dd6a1b74c9e50d220406ba1cd7ca7a not found: ID does not exist" Jul 08 18:42:45.758264 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.758247 2559 scope.go:117] "RemoveContainer" containerID="71cbe39b06da045bc9b79179052ed976322ab3a4812afa69883f50686ae51b1f" Jul 08 18:42:45.758456 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:42:45.758432 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71cbe39b06da045bc9b79179052ed976322ab3a4812afa69883f50686ae51b1f\": container with ID starting with 71cbe39b06da045bc9b79179052ed976322ab3a4812afa69883f50686ae51b1f not found: ID does not exist" containerID="71cbe39b06da045bc9b79179052ed976322ab3a4812afa69883f50686ae51b1f" Jul 08 18:42:45.758534 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.758461 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71cbe39b06da045bc9b79179052ed976322ab3a4812afa69883f50686ae51b1f"} err="failed to get container status \"71cbe39b06da045bc9b79179052ed976322ab3a4812afa69883f50686ae51b1f\": rpc error: code = NotFound desc = could not find container \"71cbe39b06da045bc9b79179052ed976322ab3a4812afa69883f50686ae51b1f\": container with ID starting with 71cbe39b06da045bc9b79179052ed976322ab3a4812afa69883f50686ae51b1f not found: ID does not exist" Jul 08 18:42:45.758534 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.758475 2559 scope.go:117] "RemoveContainer" containerID="1dd94a789bbc2462926658af6b73197cc0b8570408634ea5cb9595fdcdab68a6" Jul 08 18:42:45.758708 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:42:45.758684 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1dd94a789bbc2462926658af6b73197cc0b8570408634ea5cb9595fdcdab68a6\": container with ID starting with 1dd94a789bbc2462926658af6b73197cc0b8570408634ea5cb9595fdcdab68a6 not found: ID does not exist" containerID="1dd94a789bbc2462926658af6b73197cc0b8570408634ea5cb9595fdcdab68a6" Jul 08 18:42:45.758745 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.758720 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1dd94a789bbc2462926658af6b73197cc0b8570408634ea5cb9595fdcdab68a6"} err="failed to get container status \"1dd94a789bbc2462926658af6b73197cc0b8570408634ea5cb9595fdcdab68a6\": rpc error: code = NotFound desc = could not find container \"1dd94a789bbc2462926658af6b73197cc0b8570408634ea5cb9595fdcdab68a6\": container with ID starting with 1dd94a789bbc2462926658af6b73197cc0b8570408634ea5cb9595fdcdab68a6 not found: ID does not exist" Jul 08 18:42:45.758796 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.758743 2559 scope.go:117] "RemoveContainer" containerID="2657c44bde85c9c53ad52db26c0603b537c420a7cf49e271d6931ab5c1950d4a" Jul 08 18:42:45.759055 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:42:45.759038 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2657c44bde85c9c53ad52db26c0603b537c420a7cf49e271d6931ab5c1950d4a\": container with ID starting with 2657c44bde85c9c53ad52db26c0603b537c420a7cf49e271d6931ab5c1950d4a not found: ID does not exist" containerID="2657c44bde85c9c53ad52db26c0603b537c420a7cf49e271d6931ab5c1950d4a" Jul 08 18:42:45.759100 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.759058 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2657c44bde85c9c53ad52db26c0603b537c420a7cf49e271d6931ab5c1950d4a"} err="failed to get container status \"2657c44bde85c9c53ad52db26c0603b537c420a7cf49e271d6931ab5c1950d4a\": rpc error: code = NotFound desc = could not find container \"2657c44bde85c9c53ad52db26c0603b537c420a7cf49e271d6931ab5c1950d4a\": container with ID starting with 2657c44bde85c9c53ad52db26c0603b537c420a7cf49e271d6931ab5c1950d4a not found: ID does not exist" Jul 08 18:42:45.759100 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.759072 2559 scope.go:117] "RemoveContainer" containerID="d241845cfbb28260bb9b4b2dece65e32e98eeb33ed83226f1ae36d5ae63a89c4" Jul 08 18:42:45.759302 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:42:45.759281 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d241845cfbb28260bb9b4b2dece65e32e98eeb33ed83226f1ae36d5ae63a89c4\": container with ID starting with d241845cfbb28260bb9b4b2dece65e32e98eeb33ed83226f1ae36d5ae63a89c4 not found: ID does not exist" containerID="d241845cfbb28260bb9b4b2dece65e32e98eeb33ed83226f1ae36d5ae63a89c4" Jul 08 18:42:45.759369 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.759310 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d241845cfbb28260bb9b4b2dece65e32e98eeb33ed83226f1ae36d5ae63a89c4"} err="failed to get container status \"d241845cfbb28260bb9b4b2dece65e32e98eeb33ed83226f1ae36d5ae63a89c4\": rpc error: code = NotFound desc = could not find container \"d241845cfbb28260bb9b4b2dece65e32e98eeb33ed83226f1ae36d5ae63a89c4\": container with ID starting with d241845cfbb28260bb9b4b2dece65e32e98eeb33ed83226f1ae36d5ae63a89c4 not found: ID does not exist" Jul 08 18:42:45.759369 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.759330 2559 scope.go:117] "RemoveContainer" containerID="98283d730fd7d0bf93237202b82984dba46fded80da3bb4eef97af3c59b59d3a" Jul 08 18:42:45.759590 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:42:45.759568 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98283d730fd7d0bf93237202b82984dba46fded80da3bb4eef97af3c59b59d3a\": container with ID starting with 98283d730fd7d0bf93237202b82984dba46fded80da3bb4eef97af3c59b59d3a not found: ID does not exist" containerID="98283d730fd7d0bf93237202b82984dba46fded80da3bb4eef97af3c59b59d3a" Jul 08 18:42:45.759660 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.759595 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98283d730fd7d0bf93237202b82984dba46fded80da3bb4eef97af3c59b59d3a"} err="failed to get container status \"98283d730fd7d0bf93237202b82984dba46fded80da3bb4eef97af3c59b59d3a\": rpc error: code = NotFound desc = could not find container \"98283d730fd7d0bf93237202b82984dba46fded80da3bb4eef97af3c59b59d3a\": container with ID starting with 98283d730fd7d0bf93237202b82984dba46fded80da3bb4eef97af3c59b59d3a not found: ID does not exist" Jul 08 18:42:45.835078 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.835052 2559 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-web-config\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:42:45.835078 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.835077 2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:42:45.835261 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.835087 2559 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/df8759f2-760b-4f41-8579-af54892eeda8-config-out\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:42:45.835261 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.835098 2559 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-config-volume\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:42:45.835261 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.835106 2559 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-cluster-tls-config\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:42:45.835261 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.835116 2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:42:45.835261 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.835125 2559 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/df8759f2-760b-4f41-8579-af54892eeda8-metrics-client-ca\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:42:45.835261 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.835133 2559 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df8759f2-760b-4f41-8579-af54892eeda8-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:42:45.835261 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.835142 2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-secret-alertmanager-main-tls\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:42:45.835261 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.835150 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txtn5\" (UniqueName: \"kubernetes.io/projected/df8759f2-760b-4f41-8579-af54892eeda8-kube-api-access-txtn5\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:42:45.835261 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.835158 2559 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/df8759f2-760b-4f41-8579-af54892eeda8-alertmanager-main-db\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:42:45.835261 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.835166 2559 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/df8759f2-760b-4f41-8579-af54892eeda8-tls-assets\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:42:45.835261 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:45.835175 2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/df8759f2-760b-4f41-8579-af54892eeda8-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:42:46.008162 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.008126 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 18:42:46.012362 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.012334 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 18:42:46.036066 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.036039 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 18:42:46.036374 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.036361 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df8759f2-760b-4f41-8579-af54892eeda8" containerName="prom-label-proxy" Jul 08 18:42:46.036420 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.036378 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="df8759f2-760b-4f41-8579-af54892eeda8" containerName="prom-label-proxy" Jul 08 18:42:46.036420 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.036390 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d317a3e7-440a-4808-8ce1-b748106a5614" containerName="registry" Jul 08 18:42:46.036420 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.036397 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d317a3e7-440a-4808-8ce1-b748106a5614" containerName="registry" Jul 08 18:42:46.036420 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.036407 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df8759f2-760b-4f41-8579-af54892eeda8" containerName="kube-rbac-proxy-metric" Jul 08 18:42:46.036420 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.036416 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="df8759f2-760b-4f41-8579-af54892eeda8" containerName="kube-rbac-proxy-metric" Jul 08 18:42:46.036559 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.036425 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df8759f2-760b-4f41-8579-af54892eeda8" containerName="kube-rbac-proxy" Jul 08 18:42:46.036559 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.036431 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="df8759f2-760b-4f41-8579-af54892eeda8" containerName="kube-rbac-proxy" Jul 08 18:42:46.036559 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.036439 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df8759f2-760b-4f41-8579-af54892eeda8" containerName="alertmanager" Jul 08 18:42:46.036559 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.036444 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="df8759f2-760b-4f41-8579-af54892eeda8" containerName="alertmanager" Jul 08 18:42:46.036559 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.036457 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fa525fd-fa8e-4574-9203-eae015e8459f" containerName="console" Jul 08 18:42:46.036559 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.036462 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fa525fd-fa8e-4574-9203-eae015e8459f" containerName="console" Jul 08 18:42:46.036559 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.036470 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df8759f2-760b-4f41-8579-af54892eeda8" containerName="config-reloader" Jul 08 18:42:46.036559 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.036476 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="df8759f2-760b-4f41-8579-af54892eeda8" containerName="config-reloader" Jul 08 18:42:46.036559 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.036485 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df8759f2-760b-4f41-8579-af54892eeda8" containerName="init-config-reloader" Jul 08 18:42:46.036559 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.036495 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="df8759f2-760b-4f41-8579-af54892eeda8" containerName="init-config-reloader" Jul 08 18:42:46.036559 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.036505 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2299a3ad-27c4-4235-a567-83b92bd3e18f" containerName="console" Jul 08 18:42:46.036559 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.036509 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="2299a3ad-27c4-4235-a567-83b92bd3e18f" containerName="console" Jul 08 18:42:46.036559 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.036517 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df8759f2-760b-4f41-8579-af54892eeda8" containerName="kube-rbac-proxy-web" Jul 08 18:42:46.036559 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.036525 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="df8759f2-760b-4f41-8579-af54892eeda8" containerName="kube-rbac-proxy-web" Jul 08 18:42:46.037011 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.036579 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="2299a3ad-27c4-4235-a567-83b92bd3e18f" containerName="console" Jul 08 18:42:46.037011 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.036589 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="df8759f2-760b-4f41-8579-af54892eeda8" containerName="kube-rbac-proxy-metric" Jul 08 18:42:46.037011 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.036597 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fa525fd-fa8e-4574-9203-eae015e8459f" containerName="console" Jul 08 18:42:46.037011 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.036603 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="df8759f2-760b-4f41-8579-af54892eeda8" containerName="prom-label-proxy" Jul 08 18:42:46.037011 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.036611 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="df8759f2-760b-4f41-8579-af54892eeda8" containerName="config-reloader" Jul 08 18:42:46.037011 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.036618 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="df8759f2-760b-4f41-8579-af54892eeda8" containerName="alertmanager" Jul 08 18:42:46.037011 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.036623 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="df8759f2-760b-4f41-8579-af54892eeda8" containerName="kube-rbac-proxy-web" Jul 08 18:42:46.037011 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.036628 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="df8759f2-760b-4f41-8579-af54892eeda8" containerName="kube-rbac-proxy" Jul 08 18:42:46.037011 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.036633 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d317a3e7-440a-4808-8ce1-b748106a5614" containerName="registry" Jul 08 18:42:46.040541 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.040522 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:46.044262 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.044244 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 08 18:42:46.044692 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.044673 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 08 18:42:46.044796 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.044737 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-ncmwv\"" Jul 08 18:42:46.044865 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.044815 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 08 18:42:46.044932 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.044900 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 08 18:42:46.044932 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.044913 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 08 18:42:46.045082 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.044933 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 08 18:42:46.045082 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.044960 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 08 18:42:46.045186 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.045143 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 08 18:42:46.050402 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.050383 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 08 18:42:46.056440 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.056410 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 18:42:46.137254 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.137218 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d-config-volume\") pod \"alertmanager-main-0\" (UID: \"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:46.137429 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.137266 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:46.137429 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.137283 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:46.137429 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.137301 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:46.137429 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.137319 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d-web-config\") pod \"alertmanager-main-0\" (UID: \"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:46.137429 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.137340 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:46.137429 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.137364 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d-config-out\") pod \"alertmanager-main-0\" (UID: \"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:46.137429 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.137397 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:46.137429 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.137418 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:46.137667 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.137472 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:46.137667 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.137488 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cftqx\" (UniqueName: \"kubernetes.io/projected/ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d-kube-api-access-cftqx\") pod \"alertmanager-main-0\" (UID: \"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:46.137667 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.137504 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:46.137667 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.137522 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:46.238668 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.238629 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:46.238883 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.238679 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cftqx\" (UniqueName: \"kubernetes.io/projected/ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d-kube-api-access-cftqx\") pod \"alertmanager-main-0\" (UID: \"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:46.238883 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.238712 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:46.238883 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.238743 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:46.239093 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.238895 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d-config-volume\") pod \"alertmanager-main-0\" (UID: \"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:46.239093 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.239053 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:46.239178 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.239106 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:46.239178 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.239134 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:46.239178 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.239157 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d-web-config\") pod \"alertmanager-main-0\" (UID: \"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:46.239327 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.239197 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:46.239327 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.239240 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d-config-out\") pod \"alertmanager-main-0\" (UID: \"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:46.239327 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.239274 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:46.239327 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.239298 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:46.239901 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.239872 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:46.239990 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.239905 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:46.240107 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.240086 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:46.242070 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.241696 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d-config-volume\") pod \"alertmanager-main-0\" (UID: \"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:46.242070 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.242035 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:46.242282 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.242174 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:46.242332 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.242310 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d-config-out\") pod \"alertmanager-main-0\" (UID: \"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:46.242643 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.242619 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:46.242732 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.242711 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:46.243163 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.243142 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:46.243462 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.243446 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:46.244068 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.244051 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d-web-config\") pod \"alertmanager-main-0\" (UID: \"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:46.247239 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.247221 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cftqx\" (UniqueName: \"kubernetes.io/projected/ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d-kube-api-access-cftqx\") pod \"alertmanager-main-0\" (UID: \"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:46.349901 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.349868 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:42:46.508758 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.508731 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 18:42:46.510437 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:42:46.510414 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff2f3e01_aca5_4ca2_8f4b_addd13c59e8d.slice/crio-fdafc789e26c259683b829477b2348da38acee72b4da3164cbca4ac9fd0f60b7 WatchSource:0}: Error finding container fdafc789e26c259683b829477b2348da38acee72b4da3164cbca4ac9fd0f60b7: Status 404 returned error can't find the container with id fdafc789e26c259683b829477b2348da38acee72b4da3164cbca4ac9fd0f60b7 Jul 08 18:42:46.690033 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.690002 2559 generic.go:356] "Generic (PLEG): container finished" podID="ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d" containerID="f228a8c693f9927cb10ccc2df68f39c47df779885fd2852dbc1f050f2606a3d2" exitCode=0 Jul 08 18:42:46.690491 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.690090 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d","Type":"ContainerDied","Data":"f228a8c693f9927cb10ccc2df68f39c47df779885fd2852dbc1f050f2606a3d2"} Jul 08 18:42:46.690491 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:46.690160 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d","Type":"ContainerStarted","Data":"fdafc789e26c259683b829477b2348da38acee72b4da3164cbca4ac9fd0f60b7"} Jul 08 18:42:47.686608 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:47.686570 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df8759f2-760b-4f41-8579-af54892eeda8" path="/var/lib/kubelet/pods/df8759f2-760b-4f41-8579-af54892eeda8/volumes" Jul 08 18:42:47.697604 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:47.697573 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d","Type":"ContainerStarted","Data":"acb80ff8b82ba631d0c228c62bd1b15afef27180e551e95a106e0d68092a5459"} Jul 08 18:42:47.697959 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:47.697634 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d","Type":"ContainerStarted","Data":"c11f8c8bd1c44274c4193d4702c587f82402026fba177c86d16a0c15449ecf4f"} Jul 08 18:42:47.697959 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:47.697659 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d","Type":"ContainerStarted","Data":"46199acbaadc8412ba66bc1c4baebeffa3a89ca750adc6a9fcdeb652c3152183"} Jul 08 18:42:47.697959 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:47.697688 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d","Type":"ContainerStarted","Data":"7f078b1a17d36e2d214189c5438303c26e9c04e1645567cb83e50fe58357d9bc"} Jul 08 18:42:47.697959 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:47.697711 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d","Type":"ContainerStarted","Data":"671662a772d4aac14fe0fb9f9d0b8b68611a84ad1e60d6b63b3e7db65c316f39"} Jul 08 18:42:47.697959 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:47.697734 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d","Type":"ContainerStarted","Data":"dcb2ef0f1470ebff6edc1ab55382e8b20bac48163a4de8e0b17a48017625ed79"} Jul 08 18:42:47.731087 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:47.730978 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=1.714091366 podStartE2EDuration="1.730961308s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.869942ms" isStatefulPod=false podCreationTimestamp="2026-07-08 18:42:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:42:47.728296382 +0000 UTC m=+210.964594473" watchObservedRunningTime="2026-07-08 18:42:47.730961308 +0000 UTC m=+210.967259399" Jul 08 18:42:48.414524 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:48.414476 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-65bf6dc8b7-wwzgz"] Jul 08 18:42:48.418427 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:48.418406 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-65bf6dc8b7-wwzgz" Jul 08 18:42:48.421248 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:48.421211 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 08 18:42:48.421378 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:48.421243 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 08 18:42:48.421378 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:48.421336 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-jkvp8\"" Jul 08 18:42:48.421378 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:48.421354 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 08 18:42:48.421665 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:48.421647 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 08 18:42:48.421736 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:48.421654 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 08 18:42:48.427208 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:48.427187 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 08 18:42:48.431110 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:48.431088 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-65bf6dc8b7-wwzgz"] Jul 08 18:42:48.559942 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:48.559904 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-65bf6dc8b7-wwzgz\" (UID: \"0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e\") " pod="openshift-monitoring/telemeter-client-65bf6dc8b7-wwzgz" Jul 08 18:42:48.560119 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:48.559960 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e-serving-certs-ca-bundle\") pod \"telemeter-client-65bf6dc8b7-wwzgz\" (UID: \"0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e\") " pod="openshift-monitoring/telemeter-client-65bf6dc8b7-wwzgz" Jul 08 18:42:48.560119 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:48.559997 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-65bf6dc8b7-wwzgz\" (UID: \"0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e\") " pod="openshift-monitoring/telemeter-client-65bf6dc8b7-wwzgz" Jul 08 18:42:48.560119 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:48.560028 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7nxg\" (UniqueName: \"kubernetes.io/projected/0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e-kube-api-access-c7nxg\") pod \"telemeter-client-65bf6dc8b7-wwzgz\" (UID: \"0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e\") " pod="openshift-monitoring/telemeter-client-65bf6dc8b7-wwzgz" Jul 08 18:42:48.560119 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:48.560081 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e-secret-telemeter-client\") pod \"telemeter-client-65bf6dc8b7-wwzgz\" (UID: \"0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e\") " pod="openshift-monitoring/telemeter-client-65bf6dc8b7-wwzgz" Jul 08 18:42:48.560119 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:48.560108 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e-federate-client-tls\") pod \"telemeter-client-65bf6dc8b7-wwzgz\" (UID: \"0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e\") " pod="openshift-monitoring/telemeter-client-65bf6dc8b7-wwzgz" Jul 08 18:42:48.560280 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:48.560134 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e-metrics-client-ca\") pod \"telemeter-client-65bf6dc8b7-wwzgz\" (UID: \"0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e\") " pod="openshift-monitoring/telemeter-client-65bf6dc8b7-wwzgz" Jul 08 18:42:48.560280 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:48.560207 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e-telemeter-client-tls\") pod \"telemeter-client-65bf6dc8b7-wwzgz\" (UID: \"0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e\") " pod="openshift-monitoring/telemeter-client-65bf6dc8b7-wwzgz" Jul 08 18:42:48.661515 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:48.661473 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e-telemeter-client-tls\") pod \"telemeter-client-65bf6dc8b7-wwzgz\" (UID: \"0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e\") " pod="openshift-monitoring/telemeter-client-65bf6dc8b7-wwzgz" Jul 08 18:42:48.661690 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:48.661542 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-65bf6dc8b7-wwzgz\" (UID: \"0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e\") " pod="openshift-monitoring/telemeter-client-65bf6dc8b7-wwzgz" Jul 08 18:42:48.661690 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:48.661575 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e-serving-certs-ca-bundle\") pod \"telemeter-client-65bf6dc8b7-wwzgz\" (UID: \"0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e\") " pod="openshift-monitoring/telemeter-client-65bf6dc8b7-wwzgz" Jul 08 18:42:48.661690 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:48.661592 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-65bf6dc8b7-wwzgz\" (UID: \"0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e\") " pod="openshift-monitoring/telemeter-client-65bf6dc8b7-wwzgz" Jul 08 18:42:48.661690 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:48.661615 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7nxg\" (UniqueName: \"kubernetes.io/projected/0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e-kube-api-access-c7nxg\") pod \"telemeter-client-65bf6dc8b7-wwzgz\" (UID: \"0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e\") " pod="openshift-monitoring/telemeter-client-65bf6dc8b7-wwzgz" Jul 08 18:42:48.661690 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:48.661649 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e-secret-telemeter-client\") pod \"telemeter-client-65bf6dc8b7-wwzgz\" (UID: \"0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e\") " pod="openshift-monitoring/telemeter-client-65bf6dc8b7-wwzgz" Jul 08 18:42:48.661690 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:48.661687 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e-federate-client-tls\") pod \"telemeter-client-65bf6dc8b7-wwzgz\" (UID: \"0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e\") " pod="openshift-monitoring/telemeter-client-65bf6dc8b7-wwzgz" Jul 08 18:42:48.662008 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:48.661706 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e-metrics-client-ca\") pod \"telemeter-client-65bf6dc8b7-wwzgz\" (UID: \"0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e\") " pod="openshift-monitoring/telemeter-client-65bf6dc8b7-wwzgz" Jul 08 18:42:48.662442 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:48.662375 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e-serving-certs-ca-bundle\") pod \"telemeter-client-65bf6dc8b7-wwzgz\" (UID: \"0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e\") " pod="openshift-monitoring/telemeter-client-65bf6dc8b7-wwzgz" Jul 08 18:42:48.662569 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:48.662503 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e-metrics-client-ca\") pod \"telemeter-client-65bf6dc8b7-wwzgz\" (UID: \"0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e\") " pod="openshift-monitoring/telemeter-client-65bf6dc8b7-wwzgz" Jul 08 18:42:48.662638 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:48.662609 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-65bf6dc8b7-wwzgz\" (UID: \"0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e\") " pod="openshift-monitoring/telemeter-client-65bf6dc8b7-wwzgz" Jul 08 18:42:48.664117 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:48.664094 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e-federate-client-tls\") pod \"telemeter-client-65bf6dc8b7-wwzgz\" (UID: \"0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e\") " pod="openshift-monitoring/telemeter-client-65bf6dc8b7-wwzgz" Jul 08 18:42:48.664233 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:48.664210 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e-telemeter-client-tls\") pod \"telemeter-client-65bf6dc8b7-wwzgz\" (UID: \"0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e\") " pod="openshift-monitoring/telemeter-client-65bf6dc8b7-wwzgz" Jul 08 18:42:48.664282 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:48.664227 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-65bf6dc8b7-wwzgz\" (UID: \"0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e\") " pod="openshift-monitoring/telemeter-client-65bf6dc8b7-wwzgz" Jul 08 18:42:48.664329 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:48.664314 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e-secret-telemeter-client\") pod \"telemeter-client-65bf6dc8b7-wwzgz\" (UID: \"0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e\") " pod="openshift-monitoring/telemeter-client-65bf6dc8b7-wwzgz" Jul 08 18:42:48.673418 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:48.673367 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7nxg\" (UniqueName: \"kubernetes.io/projected/0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e-kube-api-access-c7nxg\") pod \"telemeter-client-65bf6dc8b7-wwzgz\" (UID: \"0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e\") " pod="openshift-monitoring/telemeter-client-65bf6dc8b7-wwzgz" Jul 08 18:42:48.729900 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:48.729865 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-65bf6dc8b7-wwzgz" Jul 08 18:42:48.890368 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:48.890211 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-65bf6dc8b7-wwzgz"] Jul 08 18:42:48.892560 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:42:48.892532 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d91d2f4_2a9f_4017_8cec_c02f8c9fcb5e.slice/crio-717f3dc26b203be6a7acd1223596f4b0f70b4acf186fc34bb57bc7e5f5c25b75 WatchSource:0}: Error finding container 717f3dc26b203be6a7acd1223596f4b0f70b4acf186fc34bb57bc7e5f5c25b75: Status 404 returned error can't find the container with id 717f3dc26b203be6a7acd1223596f4b0f70b4acf186fc34bb57bc7e5f5c25b75 Jul 08 18:42:49.708050 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:49.708009 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-65bf6dc8b7-wwzgz" event={"ID":"0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e","Type":"ContainerStarted","Data":"717f3dc26b203be6a7acd1223596f4b0f70b4acf186fc34bb57bc7e5f5c25b75"} Jul 08 18:42:51.720509 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:51.720475 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-65bf6dc8b7-wwzgz" event={"ID":"0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e","Type":"ContainerStarted","Data":"0ce8c5b9ea3fc60f2492b88c5b94047f5ccf696d58d25a8ccec09eb8c945e3fc"} Jul 08 18:42:51.720910 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:51.720535 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-65bf6dc8b7-wwzgz" event={"ID":"0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e","Type":"ContainerStarted","Data":"de59263dbf0034136dbc6ab1177b8fa83c8703cdd7ae9d14a3d936bef1a8c8cc"} Jul 08 18:42:51.720910 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:51.720554 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-65bf6dc8b7-wwzgz" event={"ID":"0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e","Type":"ContainerStarted","Data":"1b653d382766a36e2cb3b410534dd31ac385512f930b044aefb555e5267eb67b"} Jul 08 18:42:51.745285 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:51.745228 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-65bf6dc8b7-wwzgz" podStartSLOduration=1.5157986380000001 podStartE2EDuration="3.745207769s" totalImagesPullingTime="2.229409131s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:42:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:42:51.743752965 +0000 UTC m=+214.980051056" watchObservedRunningTime="2026-07-08 18:42:51.745207769 +0000 UTC m=+214.981505859" Jul 08 18:42:52.529878 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:52.529844 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-bffdf9879-ddqcp"] Jul 08 18:42:52.532335 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:52.532307 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bffdf9879-ddqcp" Jul 08 18:42:52.547706 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:52.547676 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bffdf9879-ddqcp"] Jul 08 18:42:52.599297 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:52.599260 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jx94\" (UniqueName: \"kubernetes.io/projected/5dfb3fde-489b-4e60-8907-35ae0b77933e-kube-api-access-6jx94\") pod \"console-bffdf9879-ddqcp\" (UID: \"5dfb3fde-489b-4e60-8907-35ae0b77933e\") " pod="openshift-console/console-bffdf9879-ddqcp" Jul 08 18:42:52.599445 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:52.599314 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5dfb3fde-489b-4e60-8907-35ae0b77933e-oauth-serving-cert\") pod \"console-bffdf9879-ddqcp\" (UID: \"5dfb3fde-489b-4e60-8907-35ae0b77933e\") " pod="openshift-console/console-bffdf9879-ddqcp" Jul 08 18:42:52.599445 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:52.599395 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5dfb3fde-489b-4e60-8907-35ae0b77933e-service-ca\") pod \"console-bffdf9879-ddqcp\" (UID: \"5dfb3fde-489b-4e60-8907-35ae0b77933e\") " pod="openshift-console/console-bffdf9879-ddqcp" Jul 08 18:42:52.599445 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:52.599426 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5dfb3fde-489b-4e60-8907-35ae0b77933e-console-config\") pod \"console-bffdf9879-ddqcp\" (UID: \"5dfb3fde-489b-4e60-8907-35ae0b77933e\") " pod="openshift-console/console-bffdf9879-ddqcp" Jul 08 18:42:52.599593 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:52.599449 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5dfb3fde-489b-4e60-8907-35ae0b77933e-console-oauth-config\") pod \"console-bffdf9879-ddqcp\" (UID: \"5dfb3fde-489b-4e60-8907-35ae0b77933e\") " pod="openshift-console/console-bffdf9879-ddqcp" Jul 08 18:42:52.599593 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:52.599513 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5dfb3fde-489b-4e60-8907-35ae0b77933e-console-serving-cert\") pod \"console-bffdf9879-ddqcp\" (UID: \"5dfb3fde-489b-4e60-8907-35ae0b77933e\") " pod="openshift-console/console-bffdf9879-ddqcp" Jul 08 18:42:52.599593 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:52.599533 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5dfb3fde-489b-4e60-8907-35ae0b77933e-trusted-ca-bundle\") pod \"console-bffdf9879-ddqcp\" (UID: \"5dfb3fde-489b-4e60-8907-35ae0b77933e\") " pod="openshift-console/console-bffdf9879-ddqcp" Jul 08 18:42:52.700078 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:52.700039 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jx94\" (UniqueName: \"kubernetes.io/projected/5dfb3fde-489b-4e60-8907-35ae0b77933e-kube-api-access-6jx94\") pod \"console-bffdf9879-ddqcp\" (UID: \"5dfb3fde-489b-4e60-8907-35ae0b77933e\") " pod="openshift-console/console-bffdf9879-ddqcp" Jul 08 18:42:52.700273 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:52.700093 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5dfb3fde-489b-4e60-8907-35ae0b77933e-oauth-serving-cert\") pod \"console-bffdf9879-ddqcp\" (UID: \"5dfb3fde-489b-4e60-8907-35ae0b77933e\") " pod="openshift-console/console-bffdf9879-ddqcp" Jul 08 18:42:52.700273 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:52.700135 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5dfb3fde-489b-4e60-8907-35ae0b77933e-service-ca\") pod \"console-bffdf9879-ddqcp\" (UID: \"5dfb3fde-489b-4e60-8907-35ae0b77933e\") " pod="openshift-console/console-bffdf9879-ddqcp" Jul 08 18:42:52.700273 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:52.700162 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5dfb3fde-489b-4e60-8907-35ae0b77933e-console-config\") pod \"console-bffdf9879-ddqcp\" (UID: \"5dfb3fde-489b-4e60-8907-35ae0b77933e\") " pod="openshift-console/console-bffdf9879-ddqcp" Jul 08 18:42:52.700273 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:52.700191 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5dfb3fde-489b-4e60-8907-35ae0b77933e-console-oauth-config\") pod \"console-bffdf9879-ddqcp\" (UID: \"5dfb3fde-489b-4e60-8907-35ae0b77933e\") " pod="openshift-console/console-bffdf9879-ddqcp" Jul 08 18:42:52.700273 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:52.700234 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5dfb3fde-489b-4e60-8907-35ae0b77933e-console-serving-cert\") pod \"console-bffdf9879-ddqcp\" (UID: \"5dfb3fde-489b-4e60-8907-35ae0b77933e\") " pod="openshift-console/console-bffdf9879-ddqcp" Jul 08 18:42:52.700273 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:52.700258 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5dfb3fde-489b-4e60-8907-35ae0b77933e-trusted-ca-bundle\") pod \"console-bffdf9879-ddqcp\" (UID: \"5dfb3fde-489b-4e60-8907-35ae0b77933e\") " pod="openshift-console/console-bffdf9879-ddqcp" Jul 08 18:42:52.700987 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:52.700959 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5dfb3fde-489b-4e60-8907-35ae0b77933e-console-config\") pod \"console-bffdf9879-ddqcp\" (UID: \"5dfb3fde-489b-4e60-8907-35ae0b77933e\") " pod="openshift-console/console-bffdf9879-ddqcp" Jul 08 18:42:52.701106 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:52.701015 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5dfb3fde-489b-4e60-8907-35ae0b77933e-service-ca\") pod \"console-bffdf9879-ddqcp\" (UID: \"5dfb3fde-489b-4e60-8907-35ae0b77933e\") " pod="openshift-console/console-bffdf9879-ddqcp" Jul 08 18:42:52.701151 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:52.701113 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5dfb3fde-489b-4e60-8907-35ae0b77933e-oauth-serving-cert\") pod \"console-bffdf9879-ddqcp\" (UID: \"5dfb3fde-489b-4e60-8907-35ae0b77933e\") " pod="openshift-console/console-bffdf9879-ddqcp" Jul 08 18:42:52.701274 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:52.701257 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5dfb3fde-489b-4e60-8907-35ae0b77933e-trusted-ca-bundle\") pod \"console-bffdf9879-ddqcp\" (UID: \"5dfb3fde-489b-4e60-8907-35ae0b77933e\") " pod="openshift-console/console-bffdf9879-ddqcp" Jul 08 18:42:52.703718 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:52.702968 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5dfb3fde-489b-4e60-8907-35ae0b77933e-console-oauth-config\") pod \"console-bffdf9879-ddqcp\" (UID: \"5dfb3fde-489b-4e60-8907-35ae0b77933e\") " pod="openshift-console/console-bffdf9879-ddqcp" Jul 08 18:42:52.703718 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:52.703218 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5dfb3fde-489b-4e60-8907-35ae0b77933e-console-serving-cert\") pod \"console-bffdf9879-ddqcp\" (UID: \"5dfb3fde-489b-4e60-8907-35ae0b77933e\") " pod="openshift-console/console-bffdf9879-ddqcp" Jul 08 18:42:52.710508 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:52.710460 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jx94\" (UniqueName: \"kubernetes.io/projected/5dfb3fde-489b-4e60-8907-35ae0b77933e-kube-api-access-6jx94\") pod \"console-bffdf9879-ddqcp\" (UID: \"5dfb3fde-489b-4e60-8907-35ae0b77933e\") " pod="openshift-console/console-bffdf9879-ddqcp" Jul 08 18:42:52.844194 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:52.844108 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bffdf9879-ddqcp" Jul 08 18:42:52.988660 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:52.988635 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bffdf9879-ddqcp"] Jul 08 18:42:52.991066 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:42:52.991039 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dfb3fde_489b_4e60_8907_35ae0b77933e.slice/crio-1c3365619602764105bd8f155c763c1c7f805f18ccbc7dc1a38f9e27f70cb94c WatchSource:0}: Error finding container 1c3365619602764105bd8f155c763c1c7f805f18ccbc7dc1a38f9e27f70cb94c: Status 404 returned error can't find the container with id 1c3365619602764105bd8f155c763c1c7f805f18ccbc7dc1a38f9e27f70cb94c Jul 08 18:42:53.730778 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:53.730728 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bffdf9879-ddqcp" event={"ID":"5dfb3fde-489b-4e60-8907-35ae0b77933e","Type":"ContainerStarted","Data":"7cf72d8bd5f430d044544a065843d0e5e86bc177484054180257aef0b9603e3b"} Jul 08 18:42:53.730943 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:53.730809 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bffdf9879-ddqcp" event={"ID":"5dfb3fde-489b-4e60-8907-35ae0b77933e","Type":"ContainerStarted","Data":"1c3365619602764105bd8f155c763c1c7f805f18ccbc7dc1a38f9e27f70cb94c"} Jul 08 18:42:53.751274 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:42:53.751221 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-bffdf9879-ddqcp" podStartSLOduration=1.7512035940000001 podStartE2EDuration="1.751203594s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:42:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:42:53.750702976 +0000 UTC m=+216.987001068" watchObservedRunningTime="2026-07-08 18:42:53.751203594 +0000 UTC m=+216.987501684" Jul 08 18:43:02.844323 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:43:02.844281 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-bffdf9879-ddqcp" Jul 08 18:43:02.844710 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:43:02.844382 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-bffdf9879-ddqcp" Jul 08 18:43:02.849144 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:43:02.849124 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-bffdf9879-ddqcp" Jul 08 18:43:03.775676 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:43:03.775649 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-bffdf9879-ddqcp" Jul 08 18:43:03.828871 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:43:03.828843 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f8c787784-rfdrg"] Jul 08 18:43:28.850449 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:43:28.850402 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-f8c787784-rfdrg" podUID="b36f17f1-a05f-44b2-9304-291e12d3edb8" containerName="console" containerID="cri-o://7b6e26ea26dccb21544ae058ad7d4c566ddc6aaca2a1cd239ba0e2c70818b005" gracePeriod=15 Jul 08 18:43:29.090993 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:43:29.090969 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f8c787784-rfdrg" Jul 08 18:43:29.211800 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:43:29.211748 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b36f17f1-a05f-44b2-9304-291e12d3edb8-console-oauth-config\") pod \"b36f17f1-a05f-44b2-9304-291e12d3edb8\" (UID: \"b36f17f1-a05f-44b2-9304-291e12d3edb8\") " Jul 08 18:43:29.211800 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:43:29.211804 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b36f17f1-a05f-44b2-9304-291e12d3edb8-console-serving-cert\") pod \"b36f17f1-a05f-44b2-9304-291e12d3edb8\" (UID: \"b36f17f1-a05f-44b2-9304-291e12d3edb8\") " Jul 08 18:43:29.212056 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:43:29.211839 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vzclk\" (UniqueName: \"kubernetes.io/projected/b36f17f1-a05f-44b2-9304-291e12d3edb8-kube-api-access-vzclk\") pod \"b36f17f1-a05f-44b2-9304-291e12d3edb8\" (UID: \"b36f17f1-a05f-44b2-9304-291e12d3edb8\") " Jul 08 18:43:29.212056 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:43:29.211862 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b36f17f1-a05f-44b2-9304-291e12d3edb8-trusted-ca-bundle\") pod \"b36f17f1-a05f-44b2-9304-291e12d3edb8\" (UID: \"b36f17f1-a05f-44b2-9304-291e12d3edb8\") " Jul 08 18:43:29.212056 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:43:29.211892 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b36f17f1-a05f-44b2-9304-291e12d3edb8-oauth-serving-cert\") pod \"b36f17f1-a05f-44b2-9304-291e12d3edb8\" (UID: \"b36f17f1-a05f-44b2-9304-291e12d3edb8\") " Jul 08 18:43:29.212056 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:43:29.211983 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b36f17f1-a05f-44b2-9304-291e12d3edb8-console-config\") pod \"b36f17f1-a05f-44b2-9304-291e12d3edb8\" (UID: \"b36f17f1-a05f-44b2-9304-291e12d3edb8\") " Jul 08 18:43:29.212056 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:43:29.212040 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b36f17f1-a05f-44b2-9304-291e12d3edb8-service-ca\") pod \"b36f17f1-a05f-44b2-9304-291e12d3edb8\" (UID: \"b36f17f1-a05f-44b2-9304-291e12d3edb8\") " Jul 08 18:43:29.212372 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:43:29.212344 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b36f17f1-a05f-44b2-9304-291e12d3edb8-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b36f17f1-a05f-44b2-9304-291e12d3edb8" (UID: "b36f17f1-a05f-44b2-9304-291e12d3edb8"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:43:29.212430 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:43:29.212343 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b36f17f1-a05f-44b2-9304-291e12d3edb8-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b36f17f1-a05f-44b2-9304-291e12d3edb8" (UID: "b36f17f1-a05f-44b2-9304-291e12d3edb8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:43:29.212523 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:43:29.212495 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b36f17f1-a05f-44b2-9304-291e12d3edb8-console-config" (OuterVolumeSpecName: "console-config") pod "b36f17f1-a05f-44b2-9304-291e12d3edb8" (UID: "b36f17f1-a05f-44b2-9304-291e12d3edb8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:43:29.212625 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:43:29.212581 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b36f17f1-a05f-44b2-9304-291e12d3edb8-service-ca" (OuterVolumeSpecName: "service-ca") pod "b36f17f1-a05f-44b2-9304-291e12d3edb8" (UID: "b36f17f1-a05f-44b2-9304-291e12d3edb8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:43:29.214187 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:43:29.214162 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b36f17f1-a05f-44b2-9304-291e12d3edb8-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b36f17f1-a05f-44b2-9304-291e12d3edb8" (UID: "b36f17f1-a05f-44b2-9304-291e12d3edb8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:43:29.214187 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:43:29.214182 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b36f17f1-a05f-44b2-9304-291e12d3edb8-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b36f17f1-a05f-44b2-9304-291e12d3edb8" (UID: "b36f17f1-a05f-44b2-9304-291e12d3edb8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:43:29.214334 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:43:29.214163 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b36f17f1-a05f-44b2-9304-291e12d3edb8-kube-api-access-vzclk" (OuterVolumeSpecName: "kube-api-access-vzclk") pod "b36f17f1-a05f-44b2-9304-291e12d3edb8" (UID: "b36f17f1-a05f-44b2-9304-291e12d3edb8"). InnerVolumeSpecName "kube-api-access-vzclk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:43:29.313518 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:43:29.313473 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzclk\" (UniqueName: \"kubernetes.io/projected/b36f17f1-a05f-44b2-9304-291e12d3edb8-kube-api-access-vzclk\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:43:29.313518 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:43:29.313506 2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b36f17f1-a05f-44b2-9304-291e12d3edb8-trusted-ca-bundle\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:43:29.313518 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:43:29.313515 2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b36f17f1-a05f-44b2-9304-291e12d3edb8-oauth-serving-cert\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:43:29.313518 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:43:29.313525 2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b36f17f1-a05f-44b2-9304-291e12d3edb8-console-config\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:43:29.313518 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:43:29.313533 2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b36f17f1-a05f-44b2-9304-291e12d3edb8-service-ca\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:43:29.313847 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:43:29.313541 2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b36f17f1-a05f-44b2-9304-291e12d3edb8-console-oauth-config\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:43:29.313847 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:43:29.313549 2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b36f17f1-a05f-44b2-9304-291e12d3edb8-console-serving-cert\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:43:29.880518 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:43:29.880482 2559 generic.go:356] "Generic (PLEG): container finished" podID="b36f17f1-a05f-44b2-9304-291e12d3edb8" containerID="7b6e26ea26dccb21544ae058ad7d4c566ddc6aaca2a1cd239ba0e2c70818b005" exitCode=0 Jul 08 18:43:29.880962 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:43:29.880530 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f8c787784-rfdrg" Jul 08 18:43:29.880962 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:43:29.880578 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f8c787784-rfdrg" event={"ID":"b36f17f1-a05f-44b2-9304-291e12d3edb8","Type":"ContainerDied","Data":"7b6e26ea26dccb21544ae058ad7d4c566ddc6aaca2a1cd239ba0e2c70818b005"} Jul 08 18:43:29.880962 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:43:29.880628 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f8c787784-rfdrg" event={"ID":"b36f17f1-a05f-44b2-9304-291e12d3edb8","Type":"ContainerDied","Data":"ed62431a9e45499cf9aeac2d206ccee72a30cf88cab06e043539780c64cbf78f"} Jul 08 18:43:29.880962 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:43:29.880648 2559 scope.go:117] "RemoveContainer" containerID="7b6e26ea26dccb21544ae058ad7d4c566ddc6aaca2a1cd239ba0e2c70818b005" Jul 08 18:43:29.889138 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:43:29.889122 2559 scope.go:117] "RemoveContainer" containerID="7b6e26ea26dccb21544ae058ad7d4c566ddc6aaca2a1cd239ba0e2c70818b005" Jul 08 18:43:29.889389 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:43:29.889370 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b6e26ea26dccb21544ae058ad7d4c566ddc6aaca2a1cd239ba0e2c70818b005\": container with ID starting with 7b6e26ea26dccb21544ae058ad7d4c566ddc6aaca2a1cd239ba0e2c70818b005 not found: ID does not exist" containerID="7b6e26ea26dccb21544ae058ad7d4c566ddc6aaca2a1cd239ba0e2c70818b005" Jul 08 18:43:29.889430 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:43:29.889396 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b6e26ea26dccb21544ae058ad7d4c566ddc6aaca2a1cd239ba0e2c70818b005"} err="failed to get container status \"7b6e26ea26dccb21544ae058ad7d4c566ddc6aaca2a1cd239ba0e2c70818b005\": rpc error: code = NotFound desc = could not find container \"7b6e26ea26dccb21544ae058ad7d4c566ddc6aaca2a1cd239ba0e2c70818b005\": container with ID starting with 7b6e26ea26dccb21544ae058ad7d4c566ddc6aaca2a1cd239ba0e2c70818b005 not found: ID does not exist" Jul 08 18:43:29.897818 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:43:29.897795 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f8c787784-rfdrg"] Jul 08 18:43:29.901106 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:43:29.901084 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-f8c787784-rfdrg"] Jul 08 18:43:31.687139 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:43:31.687105 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b36f17f1-a05f-44b2-9304-291e12d3edb8" path="/var/lib/kubelet/pods/b36f17f1-a05f-44b2-9304-291e12d3edb8/volumes" Jul 08 18:44:05.902842 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:05.902804 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7dc8c55599-t9v7x"] Jul 08 18:44:05.903475 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:05.903167 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b36f17f1-a05f-44b2-9304-291e12d3edb8" containerName="console" Jul 08 18:44:05.903475 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:05.903179 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b36f17f1-a05f-44b2-9304-291e12d3edb8" containerName="console" Jul 08 18:44:05.903475 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:05.903249 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b36f17f1-a05f-44b2-9304-291e12d3edb8" containerName="console" Jul 08 18:44:05.905322 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:05.905303 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dc8c55599-t9v7x" Jul 08 18:44:05.918621 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:05.918593 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7dc8c55599-t9v7x"] Jul 08 18:44:05.930218 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:05.930181 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a-trusted-ca-bundle\") pod \"console-7dc8c55599-t9v7x\" (UID: \"dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a\") " pod="openshift-console/console-7dc8c55599-t9v7x" Jul 08 18:44:05.930329 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:05.930225 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a-console-config\") pod \"console-7dc8c55599-t9v7x\" (UID: \"dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a\") " pod="openshift-console/console-7dc8c55599-t9v7x" Jul 08 18:44:05.930329 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:05.930245 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a-service-ca\") pod \"console-7dc8c55599-t9v7x\" (UID: \"dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a\") " pod="openshift-console/console-7dc8c55599-t9v7x" Jul 08 18:44:05.930329 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:05.930304 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a-oauth-serving-cert\") pod \"console-7dc8c55599-t9v7x\" (UID: \"dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a\") " pod="openshift-console/console-7dc8c55599-t9v7x" Jul 08 18:44:05.930431 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:05.930396 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a-console-serving-cert\") pod \"console-7dc8c55599-t9v7x\" (UID: \"dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a\") " pod="openshift-console/console-7dc8c55599-t9v7x" Jul 08 18:44:05.930431 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:05.930415 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vll64\" (UniqueName: \"kubernetes.io/projected/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a-kube-api-access-vll64\") pod \"console-7dc8c55599-t9v7x\" (UID: \"dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a\") " pod="openshift-console/console-7dc8c55599-t9v7x" Jul 08 18:44:05.930494 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:05.930438 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a-console-oauth-config\") pod \"console-7dc8c55599-t9v7x\" (UID: \"dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a\") " pod="openshift-console/console-7dc8c55599-t9v7x" Jul 08 18:44:06.030833 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:06.030799 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a-console-serving-cert\") pod \"console-7dc8c55599-t9v7x\" (UID: \"dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a\") " pod="openshift-console/console-7dc8c55599-t9v7x" Jul 08 18:44:06.030833 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:06.030834 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vll64\" (UniqueName: \"kubernetes.io/projected/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a-kube-api-access-vll64\") pod \"console-7dc8c55599-t9v7x\" (UID: \"dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a\") " pod="openshift-console/console-7dc8c55599-t9v7x" Jul 08 18:44:06.031047 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:06.030950 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a-console-oauth-config\") pod \"console-7dc8c55599-t9v7x\" (UID: \"dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a\") " pod="openshift-console/console-7dc8c55599-t9v7x" Jul 08 18:44:06.031047 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:06.031006 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a-trusted-ca-bundle\") pod \"console-7dc8c55599-t9v7x\" (UID: \"dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a\") " pod="openshift-console/console-7dc8c55599-t9v7x" Jul 08 18:44:06.031134 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:06.031059 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a-console-config\") pod \"console-7dc8c55599-t9v7x\" (UID: \"dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a\") " pod="openshift-console/console-7dc8c55599-t9v7x" Jul 08 18:44:06.031134 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:06.031084 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a-service-ca\") pod \"console-7dc8c55599-t9v7x\" (UID: \"dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a\") " pod="openshift-console/console-7dc8c55599-t9v7x" Jul 08 18:44:06.031134 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:06.031105 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a-oauth-serving-cert\") pod \"console-7dc8c55599-t9v7x\" (UID: \"dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a\") " pod="openshift-console/console-7dc8c55599-t9v7x" Jul 08 18:44:06.031858 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:06.031833 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a-console-config\") pod \"console-7dc8c55599-t9v7x\" (UID: \"dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a\") " pod="openshift-console/console-7dc8c55599-t9v7x" Jul 08 18:44:06.031959 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:06.031871 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a-trusted-ca-bundle\") pod \"console-7dc8c55599-t9v7x\" (UID: \"dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a\") " pod="openshift-console/console-7dc8c55599-t9v7x" Jul 08 18:44:06.031959 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:06.031879 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a-service-ca\") pod \"console-7dc8c55599-t9v7x\" (UID: \"dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a\") " pod="openshift-console/console-7dc8c55599-t9v7x" Jul 08 18:44:06.032036 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:06.031952 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a-oauth-serving-cert\") pod \"console-7dc8c55599-t9v7x\" (UID: \"dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a\") " pod="openshift-console/console-7dc8c55599-t9v7x" Jul 08 18:44:06.033422 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:06.033405 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a-console-oauth-config\") pod \"console-7dc8c55599-t9v7x\" (UID: \"dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a\") " pod="openshift-console/console-7dc8c55599-t9v7x" Jul 08 18:44:06.033494 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:06.033474 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a-console-serving-cert\") pod \"console-7dc8c55599-t9v7x\" (UID: \"dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a\") " pod="openshift-console/console-7dc8c55599-t9v7x" Jul 08 18:44:06.043417 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:06.043383 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vll64\" (UniqueName: \"kubernetes.io/projected/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a-kube-api-access-vll64\") pod \"console-7dc8c55599-t9v7x\" (UID: \"dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a\") " pod="openshift-console/console-7dc8c55599-t9v7x" Jul 08 18:44:06.216227 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:06.216137 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dc8c55599-t9v7x" Jul 08 18:44:06.349447 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:06.349417 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7dc8c55599-t9v7x"] Jul 08 18:44:06.350844 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:44:06.350809 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbc9c6d7_beb1_41aa_8dc3_5a0857e8a33a.slice/crio-f37851cf4478b2e2a97bf94f91686f0f45c51ad8c4326dbd8566885d3c18a748 WatchSource:0}: Error finding container f37851cf4478b2e2a97bf94f91686f0f45c51ad8c4326dbd8566885d3c18a748: Status 404 returned error can't find the container with id f37851cf4478b2e2a97bf94f91686f0f45c51ad8c4326dbd8566885d3c18a748 Jul 08 18:44:07.035711 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:07.035659 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dc8c55599-t9v7x" event={"ID":"dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a","Type":"ContainerStarted","Data":"a9e52255429cbe181873b4a7968f0ebc8976032b1dabe76c5a7f437d445b858b"} Jul 08 18:44:07.036202 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:07.035723 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dc8c55599-t9v7x" event={"ID":"dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a","Type":"ContainerStarted","Data":"f37851cf4478b2e2a97bf94f91686f0f45c51ad8c4326dbd8566885d3c18a748"} Jul 08 18:44:07.057110 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:07.057053 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7dc8c55599-t9v7x" podStartSLOduration=2.057035199 podStartE2EDuration="2.057035199s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:44:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:44:07.056108493 +0000 UTC m=+290.292406582" watchObservedRunningTime="2026-07-08 18:44:07.057035199 +0000 UTC m=+290.293333288" Jul 08 18:44:16.216288 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:16.216238 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7dc8c55599-t9v7x" Jul 08 18:44:16.216690 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:16.216320 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7dc8c55599-t9v7x" Jul 08 18:44:16.221173 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:16.221150 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7dc8c55599-t9v7x" Jul 08 18:44:17.081442 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:17.081414 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7dc8c55599-t9v7x" Jul 08 18:44:17.176914 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:17.176888 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zzt4h_d94a3566-fcf3-47a1-b176-a33b71c3826d/console-operator/1.log" Jul 08 18:44:17.183312 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:17.183286 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rj4xk_56014c3a-384a-4775-96e5-290968b196b1/ovn-acl-logging/0.log" Jul 08 18:44:17.186737 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:17.186715 2559 kubelet.go:1648] "Image garbage collection succeeded" Jul 08 18:44:17.203820 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:17.203798 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bffdf9879-ddqcp"] Jul 08 18:44:17.633949 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:17.633919 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zzt4h_d94a3566-fcf3-47a1-b176-a33b71c3826d/console-operator/1.log" Jul 08 18:44:17.640927 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:17.640905 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rj4xk_56014c3a-384a-4775-96e5-290968b196b1/ovn-acl-logging/0.log" Jul 08 18:44:42.223664 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:42.223597 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-bffdf9879-ddqcp" podUID="5dfb3fde-489b-4e60-8907-35ae0b77933e" containerName="console" containerID="cri-o://7cf72d8bd5f430d044544a065843d0e5e86bc177484054180257aef0b9603e3b" gracePeriod=15 Jul 08 18:44:42.471475 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:42.471444 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bffdf9879-ddqcp" Jul 08 18:44:42.548368 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:42.548278 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5dfb3fde-489b-4e60-8907-35ae0b77933e-console-oauth-config\") pod \"5dfb3fde-489b-4e60-8907-35ae0b77933e\" (UID: \"5dfb3fde-489b-4e60-8907-35ae0b77933e\") " Jul 08 18:44:42.548368 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:42.548320 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5dfb3fde-489b-4e60-8907-35ae0b77933e-console-config\") pod \"5dfb3fde-489b-4e60-8907-35ae0b77933e\" (UID: \"5dfb3fde-489b-4e60-8907-35ae0b77933e\") " Jul 08 18:44:42.548368 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:42.548342 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5dfb3fde-489b-4e60-8907-35ae0b77933e-console-serving-cert\") pod \"5dfb3fde-489b-4e60-8907-35ae0b77933e\" (UID: \"5dfb3fde-489b-4e60-8907-35ae0b77933e\") " Jul 08 18:44:42.548368 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:42.548370 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6jx94\" (UniqueName: \"kubernetes.io/projected/5dfb3fde-489b-4e60-8907-35ae0b77933e-kube-api-access-6jx94\") pod \"5dfb3fde-489b-4e60-8907-35ae0b77933e\" (UID: \"5dfb3fde-489b-4e60-8907-35ae0b77933e\") " Jul 08 18:44:42.548692 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:42.548402 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5dfb3fde-489b-4e60-8907-35ae0b77933e-oauth-serving-cert\") pod \"5dfb3fde-489b-4e60-8907-35ae0b77933e\" (UID: \"5dfb3fde-489b-4e60-8907-35ae0b77933e\") " Jul 08 18:44:42.548692 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:42.548422 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5dfb3fde-489b-4e60-8907-35ae0b77933e-service-ca\") pod \"5dfb3fde-489b-4e60-8907-35ae0b77933e\" (UID: \"5dfb3fde-489b-4e60-8907-35ae0b77933e\") " Jul 08 18:44:42.548692 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:42.548439 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5dfb3fde-489b-4e60-8907-35ae0b77933e-trusted-ca-bundle\") pod \"5dfb3fde-489b-4e60-8907-35ae0b77933e\" (UID: \"5dfb3fde-489b-4e60-8907-35ae0b77933e\") " Jul 08 18:44:42.548882 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:42.548743 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dfb3fde-489b-4e60-8907-35ae0b77933e-console-config" (OuterVolumeSpecName: "console-config") pod "5dfb3fde-489b-4e60-8907-35ae0b77933e" (UID: "5dfb3fde-489b-4e60-8907-35ae0b77933e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:44:42.548882 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:42.548852 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dfb3fde-489b-4e60-8907-35ae0b77933e-service-ca" (OuterVolumeSpecName: "service-ca") pod "5dfb3fde-489b-4e60-8907-35ae0b77933e" (UID: "5dfb3fde-489b-4e60-8907-35ae0b77933e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:44:42.549049 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:42.549022 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dfb3fde-489b-4e60-8907-35ae0b77933e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5dfb3fde-489b-4e60-8907-35ae0b77933e" (UID: "5dfb3fde-489b-4e60-8907-35ae0b77933e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:44:42.549143 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:42.549120 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dfb3fde-489b-4e60-8907-35ae0b77933e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5dfb3fde-489b-4e60-8907-35ae0b77933e" (UID: "5dfb3fde-489b-4e60-8907-35ae0b77933e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:44:42.550683 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:42.550656 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5dfb3fde-489b-4e60-8907-35ae0b77933e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5dfb3fde-489b-4e60-8907-35ae0b77933e" (UID: "5dfb3fde-489b-4e60-8907-35ae0b77933e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:44:42.550762 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:42.550664 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dfb3fde-489b-4e60-8907-35ae0b77933e-kube-api-access-6jx94" (OuterVolumeSpecName: "kube-api-access-6jx94") pod "5dfb3fde-489b-4e60-8907-35ae0b77933e" (UID: "5dfb3fde-489b-4e60-8907-35ae0b77933e"). InnerVolumeSpecName "kube-api-access-6jx94". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:44:42.550762 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:42.550725 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5dfb3fde-489b-4e60-8907-35ae0b77933e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5dfb3fde-489b-4e60-8907-35ae0b77933e" (UID: "5dfb3fde-489b-4e60-8907-35ae0b77933e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:44:42.649651 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:42.649607 2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5dfb3fde-489b-4e60-8907-35ae0b77933e-console-oauth-config\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:44:42.649651 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:42.649644 2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5dfb3fde-489b-4e60-8907-35ae0b77933e-console-config\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:44:42.649651 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:42.649657 2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5dfb3fde-489b-4e60-8907-35ae0b77933e-console-serving-cert\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:44:42.649940 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:42.649668 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6jx94\" (UniqueName: \"kubernetes.io/projected/5dfb3fde-489b-4e60-8907-35ae0b77933e-kube-api-access-6jx94\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:44:42.649940 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:42.649681 2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5dfb3fde-489b-4e60-8907-35ae0b77933e-oauth-serving-cert\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:44:42.649940 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:42.649692 2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5dfb3fde-489b-4e60-8907-35ae0b77933e-service-ca\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:44:42.649940 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:42.649742 2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5dfb3fde-489b-4e60-8907-35ae0b77933e-trusted-ca-bundle\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:44:43.184065 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:43.184033 2559 generic.go:356] "Generic (PLEG): container finished" podID="5dfb3fde-489b-4e60-8907-35ae0b77933e" containerID="7cf72d8bd5f430d044544a065843d0e5e86bc177484054180257aef0b9603e3b" exitCode=0 Jul 08 18:44:43.184230 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:43.184083 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bffdf9879-ddqcp" Jul 08 18:44:43.184230 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:43.184114 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bffdf9879-ddqcp" event={"ID":"5dfb3fde-489b-4e60-8907-35ae0b77933e","Type":"ContainerDied","Data":"7cf72d8bd5f430d044544a065843d0e5e86bc177484054180257aef0b9603e3b"} Jul 08 18:44:43.184230 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:43.184167 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bffdf9879-ddqcp" event={"ID":"5dfb3fde-489b-4e60-8907-35ae0b77933e","Type":"ContainerDied","Data":"1c3365619602764105bd8f155c763c1c7f805f18ccbc7dc1a38f9e27f70cb94c"} Jul 08 18:44:43.184230 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:43.184187 2559 scope.go:117] "RemoveContainer" containerID="7cf72d8bd5f430d044544a065843d0e5e86bc177484054180257aef0b9603e3b" Jul 08 18:44:43.192482 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:43.192464 2559 scope.go:117] "RemoveContainer" containerID="7cf72d8bd5f430d044544a065843d0e5e86bc177484054180257aef0b9603e3b" Jul 08 18:44:43.192730 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:44:43.192710 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cf72d8bd5f430d044544a065843d0e5e86bc177484054180257aef0b9603e3b\": container with ID starting with 7cf72d8bd5f430d044544a065843d0e5e86bc177484054180257aef0b9603e3b not found: ID does not exist" containerID="7cf72d8bd5f430d044544a065843d0e5e86bc177484054180257aef0b9603e3b" Jul 08 18:44:43.192891 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:43.192742 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cf72d8bd5f430d044544a065843d0e5e86bc177484054180257aef0b9603e3b"} err="failed to get container status \"7cf72d8bd5f430d044544a065843d0e5e86bc177484054180257aef0b9603e3b\": rpc error: code = NotFound desc = could not find container \"7cf72d8bd5f430d044544a065843d0e5e86bc177484054180257aef0b9603e3b\": container with ID starting with 7cf72d8bd5f430d044544a065843d0e5e86bc177484054180257aef0b9603e3b not found: ID does not exist" Jul 08 18:44:43.204677 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:43.204645 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bffdf9879-ddqcp"] Jul 08 18:44:43.206897 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:43.206874 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-bffdf9879-ddqcp"] Jul 08 18:44:43.686762 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:43.686732 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5dfb3fde-489b-4e60-8907-35ae0b77933e" path="/var/lib/kubelet/pods/5dfb3fde-489b-4e60-8907-35ae0b77933e/volumes" Jul 08 18:44:55.979001 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:55.978917 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk9wc"] Jul 08 18:44:55.979398 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:55.979264 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5dfb3fde-489b-4e60-8907-35ae0b77933e" containerName="console" Jul 08 18:44:55.979398 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:55.979277 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5dfb3fde-489b-4e60-8907-35ae0b77933e" containerName="console" Jul 08 18:44:55.979398 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:55.979346 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5dfb3fde-489b-4e60-8907-35ae0b77933e" containerName="console" Jul 08 18:44:55.983996 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:55.983979 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk9wc" Jul 08 18:44:55.986907 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:55.986888 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 08 18:44:55.986981 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:55.986888 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 08 18:44:55.987035 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:55.987012 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-v8p5v\"" Jul 08 18:44:55.992522 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:55.992501 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk9wc"] Jul 08 18:44:56.069933 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:56.069884 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/02955325-e3a6-4ed2-90f8-815e88582d03-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk9wc\" (UID: \"02955325-e3a6-4ed2-90f8-815e88582d03\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk9wc" Jul 08 18:44:56.070129 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:56.070019 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lglfw\" (UniqueName: \"kubernetes.io/projected/02955325-e3a6-4ed2-90f8-815e88582d03-kube-api-access-lglfw\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk9wc\" (UID: \"02955325-e3a6-4ed2-90f8-815e88582d03\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk9wc" Jul 08 18:44:56.070129 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:56.070080 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/02955325-e3a6-4ed2-90f8-815e88582d03-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk9wc\" (UID: \"02955325-e3a6-4ed2-90f8-815e88582d03\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk9wc" Jul 08 18:44:56.170478 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:56.170438 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lglfw\" (UniqueName: \"kubernetes.io/projected/02955325-e3a6-4ed2-90f8-815e88582d03-kube-api-access-lglfw\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk9wc\" (UID: \"02955325-e3a6-4ed2-90f8-815e88582d03\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk9wc" Jul 08 18:44:56.170633 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:56.170486 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/02955325-e3a6-4ed2-90f8-815e88582d03-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk9wc\" (UID: \"02955325-e3a6-4ed2-90f8-815e88582d03\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk9wc" Jul 08 18:44:56.170633 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:56.170516 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/02955325-e3a6-4ed2-90f8-815e88582d03-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk9wc\" (UID: \"02955325-e3a6-4ed2-90f8-815e88582d03\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk9wc" Jul 08 18:44:56.170913 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:56.170895 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/02955325-e3a6-4ed2-90f8-815e88582d03-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk9wc\" (UID: \"02955325-e3a6-4ed2-90f8-815e88582d03\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk9wc" Jul 08 18:44:56.170981 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:56.170934 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/02955325-e3a6-4ed2-90f8-815e88582d03-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk9wc\" (UID: \"02955325-e3a6-4ed2-90f8-815e88582d03\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk9wc" Jul 08 18:44:56.179018 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:56.178990 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lglfw\" (UniqueName: \"kubernetes.io/projected/02955325-e3a6-4ed2-90f8-815e88582d03-kube-api-access-lglfw\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk9wc\" (UID: \"02955325-e3a6-4ed2-90f8-815e88582d03\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk9wc" Jul 08 18:44:56.293572 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:56.293477 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk9wc" Jul 08 18:44:56.421511 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:56.421485 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk9wc"] Jul 08 18:44:56.423600 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:44:56.423567 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02955325_e3a6_4ed2_90f8_815e88582d03.slice/crio-ced02dc08f007c66fdf26231eadbb52636560ee776218e89eca759cf58604564 WatchSource:0}: Error finding container ced02dc08f007c66fdf26231eadbb52636560ee776218e89eca759cf58604564: Status 404 returned error can't find the container with id ced02dc08f007c66fdf26231eadbb52636560ee776218e89eca759cf58604564 Jul 08 18:44:56.425580 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:56.425561 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 18:44:57.243730 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:44:57.243685 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk9wc" event={"ID":"02955325-e3a6-4ed2-90f8-815e88582d03","Type":"ContainerStarted","Data":"ced02dc08f007c66fdf26231eadbb52636560ee776218e89eca759cf58604564"} Jul 08 18:45:06.290295 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:06.290254 2559 generic.go:356] "Generic (PLEG): container finished" podID="02955325-e3a6-4ed2-90f8-815e88582d03" containerID="383afc9228f97ba127947e56ce21e15c4ddf67f21620819d75ec0b2ad7114d3e" exitCode=0 Jul 08 18:45:06.290690 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:06.290344 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk9wc" event={"ID":"02955325-e3a6-4ed2-90f8-815e88582d03","Type":"ContainerDied","Data":"383afc9228f97ba127947e56ce21e15c4ddf67f21620819d75ec0b2ad7114d3e"} Jul 08 18:45:09.306270 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:09.306189 2559 generic.go:356] "Generic (PLEG): container finished" podID="02955325-e3a6-4ed2-90f8-815e88582d03" containerID="2d2b2aa042bfa31abf3218431fab565ad47cd44382f93de23c5842fcf94af6f8" exitCode=0 Jul 08 18:45:09.306638 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:09.306273 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk9wc" event={"ID":"02955325-e3a6-4ed2-90f8-815e88582d03","Type":"ContainerDied","Data":"2d2b2aa042bfa31abf3218431fab565ad47cd44382f93de23c5842fcf94af6f8"} Jul 08 18:45:20.356527 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:20.356491 2559 generic.go:356] "Generic (PLEG): container finished" podID="02955325-e3a6-4ed2-90f8-815e88582d03" containerID="2f2f14c623e56e2b9cb3813e11043beccd31c56104221b28ecc05f7634a546f0" exitCode=0 Jul 08 18:45:20.356935 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:20.356575 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk9wc" event={"ID":"02955325-e3a6-4ed2-90f8-815e88582d03","Type":"ContainerDied","Data":"2f2f14c623e56e2b9cb3813e11043beccd31c56104221b28ecc05f7634a546f0"} Jul 08 18:45:21.515868 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:21.515837 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk9wc" Jul 08 18:45:21.593573 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:21.593542 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/02955325-e3a6-4ed2-90f8-815e88582d03-util\") pod \"02955325-e3a6-4ed2-90f8-815e88582d03\" (UID: \"02955325-e3a6-4ed2-90f8-815e88582d03\") " Jul 08 18:45:21.593731 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:21.593596 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lglfw\" (UniqueName: \"kubernetes.io/projected/02955325-e3a6-4ed2-90f8-815e88582d03-kube-api-access-lglfw\") pod \"02955325-e3a6-4ed2-90f8-815e88582d03\" (UID: \"02955325-e3a6-4ed2-90f8-815e88582d03\") " Jul 08 18:45:21.593731 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:21.593641 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/02955325-e3a6-4ed2-90f8-815e88582d03-bundle\") pod \"02955325-e3a6-4ed2-90f8-815e88582d03\" (UID: \"02955325-e3a6-4ed2-90f8-815e88582d03\") " Jul 08 18:45:21.594458 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:21.594424 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02955325-e3a6-4ed2-90f8-815e88582d03-bundle" (OuterVolumeSpecName: "bundle") pod "02955325-e3a6-4ed2-90f8-815e88582d03" (UID: "02955325-e3a6-4ed2-90f8-815e88582d03"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:45:21.595823 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:21.595802 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02955325-e3a6-4ed2-90f8-815e88582d03-kube-api-access-lglfw" (OuterVolumeSpecName: "kube-api-access-lglfw") pod "02955325-e3a6-4ed2-90f8-815e88582d03" (UID: "02955325-e3a6-4ed2-90f8-815e88582d03"). InnerVolumeSpecName "kube-api-access-lglfw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:45:21.598723 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:21.598703 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02955325-e3a6-4ed2-90f8-815e88582d03-util" (OuterVolumeSpecName: "util") pod "02955325-e3a6-4ed2-90f8-815e88582d03" (UID: "02955325-e3a6-4ed2-90f8-815e88582d03"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:45:21.694874 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:21.694850 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/02955325-e3a6-4ed2-90f8-815e88582d03-util\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:45:21.694874 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:21.694874 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lglfw\" (UniqueName: \"kubernetes.io/projected/02955325-e3a6-4ed2-90f8-815e88582d03-kube-api-access-lglfw\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:45:21.695025 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:21.694888 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/02955325-e3a6-4ed2-90f8-815e88582d03-bundle\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:45:22.367476 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:22.367442 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk9wc" event={"ID":"02955325-e3a6-4ed2-90f8-815e88582d03","Type":"ContainerDied","Data":"ced02dc08f007c66fdf26231eadbb52636560ee776218e89eca759cf58604564"} Jul 08 18:45:22.367681 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:22.367493 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ced02dc08f007c66fdf26231eadbb52636560ee776218e89eca759cf58604564" Jul 08 18:45:22.367681 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:22.367502 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dk9wc" Jul 08 18:45:29.092862 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:29.092826 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jjhjm"] Jul 08 18:45:29.093274 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:29.093160 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02955325-e3a6-4ed2-90f8-815e88582d03" containerName="extract" Jul 08 18:45:29.093274 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:29.093171 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="02955325-e3a6-4ed2-90f8-815e88582d03" containerName="extract" Jul 08 18:45:29.093274 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:29.093181 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02955325-e3a6-4ed2-90f8-815e88582d03" containerName="util" Jul 08 18:45:29.093274 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:29.093187 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="02955325-e3a6-4ed2-90f8-815e88582d03" containerName="util" Jul 08 18:45:29.093274 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:29.093203 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02955325-e3a6-4ed2-90f8-815e88582d03" containerName="pull" Jul 08 18:45:29.093274 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:29.093209 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="02955325-e3a6-4ed2-90f8-815e88582d03" containerName="pull" Jul 08 18:45:29.093274 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:29.093263 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="02955325-e3a6-4ed2-90f8-815e88582d03" containerName="extract" Jul 08 18:45:29.095244 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:29.095230 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jjhjm" Jul 08 18:45:29.098535 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:29.098504 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 08 18:45:29.098662 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:29.098575 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-l49tg\"" Jul 08 18:45:29.098662 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:29.098578 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 08 18:45:29.114404 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:29.114370 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jjhjm"] Jul 08 18:45:29.264316 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:29.264273 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9d35c124-5595-4248-9c55-1238058ca9a7-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jjhjm\" (UID: \"9d35c124-5595-4248-9c55-1238058ca9a7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jjhjm" Jul 08 18:45:29.264316 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:29.264322 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t56c\" (UniqueName: \"kubernetes.io/projected/9d35c124-5595-4248-9c55-1238058ca9a7-kube-api-access-5t56c\") pod \"cert-manager-operator-controller-manager-55c444bf86-jjhjm\" (UID: \"9d35c124-5595-4248-9c55-1238058ca9a7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jjhjm" Jul 08 18:45:29.365750 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:29.365671 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t56c\" (UniqueName: \"kubernetes.io/projected/9d35c124-5595-4248-9c55-1238058ca9a7-kube-api-access-5t56c\") pod \"cert-manager-operator-controller-manager-55c444bf86-jjhjm\" (UID: \"9d35c124-5595-4248-9c55-1238058ca9a7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jjhjm" Jul 08 18:45:29.365877 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:29.365787 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9d35c124-5595-4248-9c55-1238058ca9a7-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jjhjm\" (UID: \"9d35c124-5595-4248-9c55-1238058ca9a7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jjhjm" Jul 08 18:45:29.366120 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:29.366105 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9d35c124-5595-4248-9c55-1238058ca9a7-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jjhjm\" (UID: \"9d35c124-5595-4248-9c55-1238058ca9a7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jjhjm" Jul 08 18:45:29.375464 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:29.375442 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t56c\" (UniqueName: \"kubernetes.io/projected/9d35c124-5595-4248-9c55-1238058ca9a7-kube-api-access-5t56c\") pod \"cert-manager-operator-controller-manager-55c444bf86-jjhjm\" (UID: \"9d35c124-5595-4248-9c55-1238058ca9a7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jjhjm" Jul 08 18:45:29.404264 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:29.404234 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jjhjm" Jul 08 18:45:29.538583 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:29.538553 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jjhjm"] Jul 08 18:45:29.541568 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:45:29.541542 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d35c124_5595_4248_9c55_1238058ca9a7.slice/crio-2fd064abbfdb1517adfb16997e9e509d338e025cdd3a8cee5ed558508c2c1544 WatchSource:0}: Error finding container 2fd064abbfdb1517adfb16997e9e509d338e025cdd3a8cee5ed558508c2c1544: Status 404 returned error can't find the container with id 2fd064abbfdb1517adfb16997e9e509d338e025cdd3a8cee5ed558508c2c1544 Jul 08 18:45:30.403446 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:30.403402 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jjhjm" event={"ID":"9d35c124-5595-4248-9c55-1238058ca9a7","Type":"ContainerStarted","Data":"2fd064abbfdb1517adfb16997e9e509d338e025cdd3a8cee5ed558508c2c1544"} Jul 08 18:45:31.409810 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:31.409759 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jjhjm" event={"ID":"9d35c124-5595-4248-9c55-1238058ca9a7","Type":"ContainerStarted","Data":"ba7f53b102f70ab53b55af68648df06f5cfd27388cf4efebcaabfcd45343ad31"} Jul 08 18:45:31.435681 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:31.435615 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jjhjm" podStartSLOduration=0.768520714 podStartE2EDuration="2.435596158s" totalImagesPullingTime="1.667075444s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:45:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:45:31.43488689 +0000 UTC m=+374.671184976" watchObservedRunningTime="2026-07-08 18:45:31.435596158 +0000 UTC m=+374.671894249" Jul 08 18:45:32.806062 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:32.806029 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdsh5g"] Jul 08 18:45:32.808431 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:32.808411 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdsh5g" Jul 08 18:45:32.811269 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:32.811247 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 08 18:45:32.812328 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:32.812313 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-v8p5v\"" Jul 08 18:45:32.812435 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:32.812314 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 08 18:45:32.819611 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:32.819586 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdsh5g"] Jul 08 18:45:32.894309 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:32.894274 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlkjb\" (UniqueName: \"kubernetes.io/projected/cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055-kube-api-access-rlkjb\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdsh5g\" (UID: \"cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdsh5g" Jul 08 18:45:32.894309 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:32.894317 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdsh5g\" (UID: \"cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdsh5g" Jul 08 18:45:32.894518 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:32.894360 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdsh5g\" (UID: \"cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdsh5g" Jul 08 18:45:32.995716 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:32.995674 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlkjb\" (UniqueName: \"kubernetes.io/projected/cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055-kube-api-access-rlkjb\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdsh5g\" (UID: \"cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdsh5g" Jul 08 18:45:32.995897 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:32.995722 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdsh5g\" (UID: \"cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdsh5g" Jul 08 18:45:32.995897 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:32.995803 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdsh5g\" (UID: \"cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdsh5g" Jul 08 18:45:32.996149 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:32.996131 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdsh5g\" (UID: \"cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdsh5g" Jul 08 18:45:32.996188 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:32.996158 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdsh5g\" (UID: \"cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdsh5g" Jul 08 18:45:33.012063 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:33.012031 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlkjb\" (UniqueName: \"kubernetes.io/projected/cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055-kube-api-access-rlkjb\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdsh5g\" (UID: \"cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdsh5g" Jul 08 18:45:33.118006 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:33.117977 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdsh5g" Jul 08 18:45:33.252178 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:33.252150 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdsh5g"] Jul 08 18:45:33.254663 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:45:33.254639 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdfcdb82_ff05_47ad_be7a_9c7d8bcc8055.slice/crio-c354f04244bc76b7848229be47317179844b9a31ace8cccad250cc1338ad230e WatchSource:0}: Error finding container c354f04244bc76b7848229be47317179844b9a31ace8cccad250cc1338ad230e: Status 404 returned error can't find the container with id c354f04244bc76b7848229be47317179844b9a31ace8cccad250cc1338ad230e Jul 08 18:45:33.420535 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:33.420446 2559 generic.go:356] "Generic (PLEG): container finished" podID="cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055" containerID="b0b1928ddf6746c49743bc033742d4612f5efab05ba7ad634b82b04a82630f51" exitCode=0 Jul 08 18:45:33.420672 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:33.420528 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdsh5g" event={"ID":"cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055","Type":"ContainerDied","Data":"b0b1928ddf6746c49743bc033742d4612f5efab05ba7ad634b82b04a82630f51"} Jul 08 18:45:33.420672 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:33.420603 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdsh5g" event={"ID":"cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055","Type":"ContainerStarted","Data":"c354f04244bc76b7848229be47317179844b9a31ace8cccad250cc1338ad230e"} Jul 08 18:45:34.030214 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:34.030178 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-n86n9"] Jul 08 18:45:34.032825 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:34.032804 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-n86n9" Jul 08 18:45:34.035888 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:34.035869 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 08 18:45:34.037141 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:34.037125 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 08 18:45:34.037244 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:34.037143 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-lxm8s\"" Jul 08 18:45:34.052233 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:34.052198 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-n86n9"] Jul 08 18:45:34.207494 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:34.207448 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/191d819a-6d9d-429f-bccf-0833f73bedcb-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-n86n9\" (UID: \"191d819a-6d9d-429f-bccf-0833f73bedcb\") " pod="cert-manager/cert-manager-webhook-6486ffd997-n86n9" Jul 08 18:45:34.207685 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:34.207562 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlc44\" (UniqueName: \"kubernetes.io/projected/191d819a-6d9d-429f-bccf-0833f73bedcb-kube-api-access-qlc44\") pod \"cert-manager-webhook-6486ffd997-n86n9\" (UID: \"191d819a-6d9d-429f-bccf-0833f73bedcb\") " pod="cert-manager/cert-manager-webhook-6486ffd997-n86n9" Jul 08 18:45:34.308686 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:34.308590 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlc44\" (UniqueName: \"kubernetes.io/projected/191d819a-6d9d-429f-bccf-0833f73bedcb-kube-api-access-qlc44\") pod \"cert-manager-webhook-6486ffd997-n86n9\" (UID: \"191d819a-6d9d-429f-bccf-0833f73bedcb\") " pod="cert-manager/cert-manager-webhook-6486ffd997-n86n9" Jul 08 18:45:34.308863 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:34.308730 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/191d819a-6d9d-429f-bccf-0833f73bedcb-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-n86n9\" (UID: \"191d819a-6d9d-429f-bccf-0833f73bedcb\") " pod="cert-manager/cert-manager-webhook-6486ffd997-n86n9" Jul 08 18:45:34.318389 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:34.318357 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/191d819a-6d9d-429f-bccf-0833f73bedcb-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-n86n9\" (UID: \"191d819a-6d9d-429f-bccf-0833f73bedcb\") " pod="cert-manager/cert-manager-webhook-6486ffd997-n86n9" Jul 08 18:45:34.318531 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:34.318362 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlc44\" (UniqueName: \"kubernetes.io/projected/191d819a-6d9d-429f-bccf-0833f73bedcb-kube-api-access-qlc44\") pod \"cert-manager-webhook-6486ffd997-n86n9\" (UID: \"191d819a-6d9d-429f-bccf-0833f73bedcb\") " pod="cert-manager/cert-manager-webhook-6486ffd997-n86n9" Jul 08 18:45:34.348666 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:34.348634 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-n86n9" Jul 08 18:45:34.502658 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:34.502628 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-n86n9"] Jul 08 18:45:34.505902 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:45:34.505864 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod191d819a_6d9d_429f_bccf_0833f73bedcb.slice/crio-429f39862a24e62af5805dbcc58f0469473f6dd1c067d36b95c215cc2798e79a WatchSource:0}: Error finding container 429f39862a24e62af5805dbcc58f0469473f6dd1c067d36b95c215cc2798e79a: Status 404 returned error can't find the container with id 429f39862a24e62af5805dbcc58f0469473f6dd1c067d36b95c215cc2798e79a Jul 08 18:45:35.346479 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:35.346436 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-r4wq6"] Jul 08 18:45:35.348494 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:35.348478 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-r4wq6" Jul 08 18:45:35.351088 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:35.351065 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-fskcb\"" Jul 08 18:45:35.361376 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:35.361353 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-r4wq6"] Jul 08 18:45:35.433859 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:35.433821 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdsh5g" event={"ID":"cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055","Type":"ContainerStarted","Data":"4d0deb34387da0630d887976fb192de18e7423ce84781494c58b5810d7000d93"} Jul 08 18:45:35.435048 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:35.435022 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-n86n9" event={"ID":"191d819a-6d9d-429f-bccf-0833f73bedcb","Type":"ContainerStarted","Data":"429f39862a24e62af5805dbcc58f0469473f6dd1c067d36b95c215cc2798e79a"} Jul 08 18:45:35.518761 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:35.518718 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2550f52c-4d9c-4e68-af12-e815afedcad9-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-r4wq6\" (UID: \"2550f52c-4d9c-4e68-af12-e815afedcad9\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-r4wq6" Jul 08 18:45:35.518971 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:35.518841 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlzr6\" (UniqueName: \"kubernetes.io/projected/2550f52c-4d9c-4e68-af12-e815afedcad9-kube-api-access-nlzr6\") pod \"cert-manager-cainjector-559cdd8c87-r4wq6\" (UID: \"2550f52c-4d9c-4e68-af12-e815afedcad9\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-r4wq6" Jul 08 18:45:35.620290 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:35.620253 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2550f52c-4d9c-4e68-af12-e815afedcad9-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-r4wq6\" (UID: \"2550f52c-4d9c-4e68-af12-e815afedcad9\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-r4wq6" Jul 08 18:45:35.620485 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:35.620316 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlzr6\" (UniqueName: \"kubernetes.io/projected/2550f52c-4d9c-4e68-af12-e815afedcad9-kube-api-access-nlzr6\") pod \"cert-manager-cainjector-559cdd8c87-r4wq6\" (UID: \"2550f52c-4d9c-4e68-af12-e815afedcad9\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-r4wq6" Jul 08 18:45:35.629245 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:35.629220 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2550f52c-4d9c-4e68-af12-e815afedcad9-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-r4wq6\" (UID: \"2550f52c-4d9c-4e68-af12-e815afedcad9\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-r4wq6" Jul 08 18:45:35.629394 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:35.629311 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlzr6\" (UniqueName: \"kubernetes.io/projected/2550f52c-4d9c-4e68-af12-e815afedcad9-kube-api-access-nlzr6\") pod \"cert-manager-cainjector-559cdd8c87-r4wq6\" (UID: \"2550f52c-4d9c-4e68-af12-e815afedcad9\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-r4wq6" Jul 08 18:45:35.658478 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:35.658411 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-r4wq6" Jul 08 18:45:35.805302 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:35.805279 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-r4wq6"] Jul 08 18:45:35.808021 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:45:35.807990 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2550f52c_4d9c_4e68_af12_e815afedcad9.slice/crio-664173a6f79369aa31f6bd0bc6a8318cf2c3cb4cba59b8b0ef1f616b3416f3cd WatchSource:0}: Error finding container 664173a6f79369aa31f6bd0bc6a8318cf2c3cb4cba59b8b0ef1f616b3416f3cd: Status 404 returned error can't find the container with id 664173a6f79369aa31f6bd0bc6a8318cf2c3cb4cba59b8b0ef1f616b3416f3cd Jul 08 18:45:36.443004 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:36.442968 2559 generic.go:356] "Generic (PLEG): container finished" podID="cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055" containerID="4d0deb34387da0630d887976fb192de18e7423ce84781494c58b5810d7000d93" exitCode=0 Jul 08 18:45:36.443496 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:36.443061 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdsh5g" event={"ID":"cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055","Type":"ContainerDied","Data":"4d0deb34387da0630d887976fb192de18e7423ce84781494c58b5810d7000d93"} Jul 08 18:45:36.444544 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:36.444517 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-r4wq6" event={"ID":"2550f52c-4d9c-4e68-af12-e815afedcad9","Type":"ContainerStarted","Data":"664173a6f79369aa31f6bd0bc6a8318cf2c3cb4cba59b8b0ef1f616b3416f3cd"} Jul 08 18:45:37.456365 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:37.456328 2559 generic.go:356] "Generic (PLEG): container finished" podID="cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055" containerID="93e0b73226cb585bd4fc20577143903cd2aba55ebfe61162fc5343eab1b15953" exitCode=0 Jul 08 18:45:37.456734 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:37.456372 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdsh5g" event={"ID":"cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055","Type":"ContainerDied","Data":"93e0b73226cb585bd4fc20577143903cd2aba55ebfe61162fc5343eab1b15953"} Jul 08 18:45:38.462174 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:38.462136 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-n86n9" event={"ID":"191d819a-6d9d-429f-bccf-0833f73bedcb","Type":"ContainerStarted","Data":"22f42ef7dedb3682161c911c2d267536e811f47d7e14dd7a28cd2ef38fc8b34c"} Jul 08 18:45:38.462649 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:38.462207 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-n86n9" Jul 08 18:45:38.463626 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:38.463601 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-r4wq6" event={"ID":"2550f52c-4d9c-4e68-af12-e815afedcad9","Type":"ContainerStarted","Data":"4660f94f76fa38f4745e25931665320e59d9b59140efa4e65e505bf5b9805abf"} Jul 08 18:45:38.484224 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:38.484174 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-n86n9" podStartSLOduration=1.512888576 podStartE2EDuration="4.484159258s" totalImagesPullingTime="2.971270682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:45:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:45:38.481732935 +0000 UTC m=+381.718031020" watchObservedRunningTime="2026-07-08 18:45:38.484159258 +0000 UTC m=+381.720457348" Jul 08 18:45:38.502355 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:38.502306 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-r4wq6" podStartSLOduration=1.83278726 podStartE2EDuration="3.502290626s" totalImagesPullingTime="1.669503366s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:45:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:45:38.500956381 +0000 UTC m=+381.737254472" watchObservedRunningTime="2026-07-08 18:45:38.502290626 +0000 UTC m=+381.738588716" Jul 08 18:45:38.601399 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:38.601367 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdsh5g" Jul 08 18:45:38.748541 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:38.748463 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055-util\") pod \"cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055\" (UID: \"cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055\") " Jul 08 18:45:38.748541 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:38.748513 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055-bundle\") pod \"cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055\" (UID: \"cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055\") " Jul 08 18:45:38.748541 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:38.748534 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rlkjb\" (UniqueName: \"kubernetes.io/projected/cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055-kube-api-access-rlkjb\") pod \"cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055\" (UID: \"cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055\") " Jul 08 18:45:38.748843 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:38.748821 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055-bundle" (OuterVolumeSpecName: "bundle") pod "cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055" (UID: "cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:45:38.750680 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:38.750649 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055-kube-api-access-rlkjb" (OuterVolumeSpecName: "kube-api-access-rlkjb") pod "cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055" (UID: "cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055"). InnerVolumeSpecName "kube-api-access-rlkjb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:45:38.821920 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:38.821870 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055-util" (OuterVolumeSpecName: "util") pod "cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055" (UID: "cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:45:38.850202 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:38.850167 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055-util\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:45:38.850202 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:38.850195 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055-bundle\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:45:38.850202 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:38.850207 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rlkjb\" (UniqueName: \"kubernetes.io/projected/cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055-kube-api-access-rlkjb\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:45:39.470145 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:39.470115 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdsh5g" Jul 08 18:45:39.470526 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:39.470145 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdsh5g" event={"ID":"cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055","Type":"ContainerDied","Data":"c354f04244bc76b7848229be47317179844b9a31ace8cccad250cc1338ad230e"} Jul 08 18:45:39.470526 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:39.470189 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c354f04244bc76b7848229be47317179844b9a31ace8cccad250cc1338ad230e" Jul 08 18:45:43.649863 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:43.649824 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-rsp5v"] Jul 08 18:45:43.650243 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:43.650223 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055" containerName="extract" Jul 08 18:45:43.650243 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:43.650237 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055" containerName="extract" Jul 08 18:45:43.650316 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:43.650248 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055" containerName="pull" Jul 08 18:45:43.650316 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:43.650257 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055" containerName="pull" Jul 08 18:45:43.650316 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:43.650274 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055" containerName="util" Jul 08 18:45:43.650316 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:43.650280 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055" containerName="util" Jul 08 18:45:43.650435 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:43.650345 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdfcdb82-ff05-47ad-be7a-9c7d8bcc8055" containerName="extract" Jul 08 18:45:43.654071 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:43.654051 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-rsp5v" Jul 08 18:45:43.657824 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:43.657804 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-w5f7c\"" Jul 08 18:45:43.669092 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:43.669066 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-rsp5v"] Jul 08 18:45:43.794473 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:43.794438 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9l6gl\" (UniqueName: \"kubernetes.io/projected/48e2a3eb-987b-4ecf-924d-6c17afb370a2-kube-api-access-9l6gl\") pod \"cert-manager-55b4d954d5-rsp5v\" (UID: \"48e2a3eb-987b-4ecf-924d-6c17afb370a2\") " pod="cert-manager/cert-manager-55b4d954d5-rsp5v" Jul 08 18:45:43.794710 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:43.794632 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/48e2a3eb-987b-4ecf-924d-6c17afb370a2-bound-sa-token\") pod \"cert-manager-55b4d954d5-rsp5v\" (UID: \"48e2a3eb-987b-4ecf-924d-6c17afb370a2\") " pod="cert-manager/cert-manager-55b4d954d5-rsp5v" Jul 08 18:45:43.896043 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:43.895998 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/48e2a3eb-987b-4ecf-924d-6c17afb370a2-bound-sa-token\") pod \"cert-manager-55b4d954d5-rsp5v\" (UID: \"48e2a3eb-987b-4ecf-924d-6c17afb370a2\") " pod="cert-manager/cert-manager-55b4d954d5-rsp5v" Jul 08 18:45:43.896227 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:43.896100 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9l6gl\" (UniqueName: \"kubernetes.io/projected/48e2a3eb-987b-4ecf-924d-6c17afb370a2-kube-api-access-9l6gl\") pod \"cert-manager-55b4d954d5-rsp5v\" (UID: \"48e2a3eb-987b-4ecf-924d-6c17afb370a2\") " pod="cert-manager/cert-manager-55b4d954d5-rsp5v" Jul 08 18:45:43.905026 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:43.904948 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/48e2a3eb-987b-4ecf-924d-6c17afb370a2-bound-sa-token\") pod \"cert-manager-55b4d954d5-rsp5v\" (UID: \"48e2a3eb-987b-4ecf-924d-6c17afb370a2\") " pod="cert-manager/cert-manager-55b4d954d5-rsp5v" Jul 08 18:45:43.905188 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:43.905167 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9l6gl\" (UniqueName: \"kubernetes.io/projected/48e2a3eb-987b-4ecf-924d-6c17afb370a2-kube-api-access-9l6gl\") pod \"cert-manager-55b4d954d5-rsp5v\" (UID: \"48e2a3eb-987b-4ecf-924d-6c17afb370a2\") " pod="cert-manager/cert-manager-55b4d954d5-rsp5v" Jul 08 18:45:43.962661 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:43.962622 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-rsp5v" Jul 08 18:45:44.099579 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:44.099556 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-rsp5v"] Jul 08 18:45:44.101250 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:45:44.101223 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod48e2a3eb_987b_4ecf_924d_6c17afb370a2.slice/crio-ea5b97febf3334a727279f281312466e07323724c5318dedf0e7d89ce349b5a9 WatchSource:0}: Error finding container ea5b97febf3334a727279f281312466e07323724c5318dedf0e7d89ce349b5a9: Status 404 returned error can't find the container with id ea5b97febf3334a727279f281312466e07323724c5318dedf0e7d89ce349b5a9 Jul 08 18:45:44.473096 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:44.473004 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-n86n9" Jul 08 18:45:44.495630 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:44.495590 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-rsp5v" event={"ID":"48e2a3eb-987b-4ecf-924d-6c17afb370a2","Type":"ContainerStarted","Data":"ebcddb97dd64c6dc7ea1ba3088e4e09791014c1185446c152528b2229f33627a"} Jul 08 18:45:44.495630 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:44.495631 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-rsp5v" event={"ID":"48e2a3eb-987b-4ecf-924d-6c17afb370a2","Type":"ContainerStarted","Data":"ea5b97febf3334a727279f281312466e07323724c5318dedf0e7d89ce349b5a9"} Jul 08 18:45:44.526422 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:44.526371 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-rsp5v" podStartSLOduration=1.5263543450000001 podStartE2EDuration="1.526354345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:45:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:45:44.524463054 +0000 UTC m=+387.760761147" watchObservedRunningTime="2026-07-08 18:45:44.526354345 +0000 UTC m=+387.762652435" Jul 08 18:45:54.617282 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:54.617244 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7qqhw"] Jul 08 18:45:54.622145 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:54.622127 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7qqhw" Jul 08 18:45:54.624916 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:54.624894 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 08 18:45:54.625042 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:54.625026 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 08 18:45:54.625982 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:54.625967 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-v8p5v\"" Jul 08 18:45:54.629721 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:54.629691 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7qqhw"] Jul 08 18:45:54.674219 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:54.674180 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wv8j8\" (UniqueName: \"kubernetes.io/projected/a655b5e2-e368-4342-951c-5e5ac3023cb4-kube-api-access-wv8j8\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7qqhw\" (UID: \"a655b5e2-e368-4342-951c-5e5ac3023cb4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7qqhw" Jul 08 18:45:54.674380 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:54.674247 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a655b5e2-e368-4342-951c-5e5ac3023cb4-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7qqhw\" (UID: \"a655b5e2-e368-4342-951c-5e5ac3023cb4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7qqhw" Jul 08 18:45:54.674380 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:54.674270 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a655b5e2-e368-4342-951c-5e5ac3023cb4-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7qqhw\" (UID: \"a655b5e2-e368-4342-951c-5e5ac3023cb4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7qqhw" Jul 08 18:45:54.775524 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:54.775483 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wv8j8\" (UniqueName: \"kubernetes.io/projected/a655b5e2-e368-4342-951c-5e5ac3023cb4-kube-api-access-wv8j8\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7qqhw\" (UID: \"a655b5e2-e368-4342-951c-5e5ac3023cb4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7qqhw" Jul 08 18:45:54.775717 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:54.775551 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a655b5e2-e368-4342-951c-5e5ac3023cb4-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7qqhw\" (UID: \"a655b5e2-e368-4342-951c-5e5ac3023cb4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7qqhw" Jul 08 18:45:54.775717 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:54.775572 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a655b5e2-e368-4342-951c-5e5ac3023cb4-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7qqhw\" (UID: \"a655b5e2-e368-4342-951c-5e5ac3023cb4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7qqhw" Jul 08 18:45:54.775957 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:54.775935 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a655b5e2-e368-4342-951c-5e5ac3023cb4-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7qqhw\" (UID: \"a655b5e2-e368-4342-951c-5e5ac3023cb4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7qqhw" Jul 08 18:45:54.776029 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:54.776001 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a655b5e2-e368-4342-951c-5e5ac3023cb4-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7qqhw\" (UID: \"a655b5e2-e368-4342-951c-5e5ac3023cb4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7qqhw" Jul 08 18:45:54.784655 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:54.784627 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wv8j8\" (UniqueName: \"kubernetes.io/projected/a655b5e2-e368-4342-951c-5e5ac3023cb4-kube-api-access-wv8j8\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7qqhw\" (UID: \"a655b5e2-e368-4342-951c-5e5ac3023cb4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7qqhw" Jul 08 18:45:54.932593 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:54.932499 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7qqhw" Jul 08 18:45:55.063358 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:55.063336 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7qqhw"] Jul 08 18:45:55.065274 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:45:55.065248 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda655b5e2_e368_4342_951c_5e5ac3023cb4.slice/crio-ba671677f0ad7f15f4e39dd04d8064f248f82956d063576e90872fdd2a13510e WatchSource:0}: Error finding container ba671677f0ad7f15f4e39dd04d8064f248f82956d063576e90872fdd2a13510e: Status 404 returned error can't find the container with id ba671677f0ad7f15f4e39dd04d8064f248f82956d063576e90872fdd2a13510e Jul 08 18:45:55.554298 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:55.554262 2559 generic.go:356] "Generic (PLEG): container finished" podID="a655b5e2-e368-4342-951c-5e5ac3023cb4" containerID="7316f4d0961fcaa0eddf898cd8e6b0ec38aed430784854e336e07f6200c8d307" exitCode=0 Jul 08 18:45:55.554471 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:55.554345 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7qqhw" event={"ID":"a655b5e2-e368-4342-951c-5e5ac3023cb4","Type":"ContainerDied","Data":"7316f4d0961fcaa0eddf898cd8e6b0ec38aed430784854e336e07f6200c8d307"} Jul 08 18:45:55.554471 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:55.554422 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7qqhw" event={"ID":"a655b5e2-e368-4342-951c-5e5ac3023cb4","Type":"ContainerStarted","Data":"ba671677f0ad7f15f4e39dd04d8064f248f82956d063576e90872fdd2a13510e"} Jul 08 18:45:56.560937 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:56.560852 2559 generic.go:356] "Generic (PLEG): container finished" podID="a655b5e2-e368-4342-951c-5e5ac3023cb4" containerID="78ec62a76278d191f27f12d429da427467e42d40b79481634fab953a6313bc01" exitCode=0 Jul 08 18:45:56.561277 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:56.560932 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7qqhw" event={"ID":"a655b5e2-e368-4342-951c-5e5ac3023cb4","Type":"ContainerDied","Data":"78ec62a76278d191f27f12d429da427467e42d40b79481634fab953a6313bc01"} Jul 08 18:45:57.568369 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:57.568331 2559 generic.go:356] "Generic (PLEG): container finished" podID="a655b5e2-e368-4342-951c-5e5ac3023cb4" containerID="4ce277137cb2a7f94dbc1b3f7a115ed3e2abd340c54b8dce1cc82180d0f3680d" exitCode=0 Jul 08 18:45:57.568812 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:57.568425 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7qqhw" event={"ID":"a655b5e2-e368-4342-951c-5e5ac3023cb4","Type":"ContainerDied","Data":"4ce277137cb2a7f94dbc1b3f7a115ed3e2abd340c54b8dce1cc82180d0f3680d"} Jul 08 18:45:58.697246 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:58.697221 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7qqhw" Jul 08 18:45:58.703472 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:58.703449 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wv8j8\" (UniqueName: \"kubernetes.io/projected/a655b5e2-e368-4342-951c-5e5ac3023cb4-kube-api-access-wv8j8\") pod \"a655b5e2-e368-4342-951c-5e5ac3023cb4\" (UID: \"a655b5e2-e368-4342-951c-5e5ac3023cb4\") " Jul 08 18:45:58.703542 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:58.703488 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a655b5e2-e368-4342-951c-5e5ac3023cb4-util\") pod \"a655b5e2-e368-4342-951c-5e5ac3023cb4\" (UID: \"a655b5e2-e368-4342-951c-5e5ac3023cb4\") " Jul 08 18:45:58.703542 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:58.703514 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a655b5e2-e368-4342-951c-5e5ac3023cb4-bundle\") pod \"a655b5e2-e368-4342-951c-5e5ac3023cb4\" (UID: \"a655b5e2-e368-4342-951c-5e5ac3023cb4\") " Jul 08 18:45:58.704229 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:58.704203 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a655b5e2-e368-4342-951c-5e5ac3023cb4-bundle" (OuterVolumeSpecName: "bundle") pod "a655b5e2-e368-4342-951c-5e5ac3023cb4" (UID: "a655b5e2-e368-4342-951c-5e5ac3023cb4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:45:58.705448 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:58.705424 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a655b5e2-e368-4342-951c-5e5ac3023cb4-kube-api-access-wv8j8" (OuterVolumeSpecName: "kube-api-access-wv8j8") pod "a655b5e2-e368-4342-951c-5e5ac3023cb4" (UID: "a655b5e2-e368-4342-951c-5e5ac3023cb4"). InnerVolumeSpecName "kube-api-access-wv8j8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:45:58.708749 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:58.708731 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a655b5e2-e368-4342-951c-5e5ac3023cb4-util" (OuterVolumeSpecName: "util") pod "a655b5e2-e368-4342-951c-5e5ac3023cb4" (UID: "a655b5e2-e368-4342-951c-5e5ac3023cb4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:45:58.804195 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:58.804163 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wv8j8\" (UniqueName: \"kubernetes.io/projected/a655b5e2-e368-4342-951c-5e5ac3023cb4-kube-api-access-wv8j8\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:45:58.804195 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:58.804189 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a655b5e2-e368-4342-951c-5e5ac3023cb4-util\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:45:58.804195 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:58.804198 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a655b5e2-e368-4342-951c-5e5ac3023cb4-bundle\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:45:59.580438 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:59.580401 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7qqhw" event={"ID":"a655b5e2-e368-4342-951c-5e5ac3023cb4","Type":"ContainerDied","Data":"ba671677f0ad7f15f4e39dd04d8064f248f82956d063576e90872fdd2a13510e"} Jul 08 18:45:59.580438 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:59.580425 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7qqhw" Jul 08 18:45:59.580646 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:45:59.580450 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ba671677f0ad7f15f4e39dd04d8064f248f82956d063576e90872fdd2a13510e" Jul 08 18:46:07.456841 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:07.456805 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c5cf545b9-n999r"] Jul 08 18:46:07.457217 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:07.457147 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a655b5e2-e368-4342-951c-5e5ac3023cb4" containerName="extract" Jul 08 18:46:07.457217 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:07.457159 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a655b5e2-e368-4342-951c-5e5ac3023cb4" containerName="extract" Jul 08 18:46:07.457217 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:07.457170 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a655b5e2-e368-4342-951c-5e5ac3023cb4" containerName="util" Jul 08 18:46:07.457217 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:07.457176 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a655b5e2-e368-4342-951c-5e5ac3023cb4" containerName="util" Jul 08 18:46:07.457217 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:07.457192 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a655b5e2-e368-4342-951c-5e5ac3023cb4" containerName="pull" Jul 08 18:46:07.457217 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:07.457197 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a655b5e2-e368-4342-951c-5e5ac3023cb4" containerName="pull" Jul 08 18:46:07.457442 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:07.457268 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a655b5e2-e368-4342-951c-5e5ac3023cb4" containerName="extract" Jul 08 18:46:07.459751 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:07.459734 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6c5cf545b9-n999r" Jul 08 18:46:07.462624 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:07.462599 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-56jvk\"" Jul 08 18:46:07.462747 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:07.462662 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 08 18:46:07.462747 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:07.462690 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 08 18:46:07.462747 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:07.462699 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 08 18:46:07.462747 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:07.462739 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 08 18:46:07.475364 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:07.475263 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d57edb35-475a-478b-b938-1b9c93165c16-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c5cf545b9-n999r\" (UID: \"d57edb35-475a-478b-b938-1b9c93165c16\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5cf545b9-n999r" Jul 08 18:46:07.475364 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:07.475328 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-td292\" (UniqueName: \"kubernetes.io/projected/d57edb35-475a-478b-b938-1b9c93165c16-kube-api-access-td292\") pod \"opendatahub-operator-controller-manager-6c5cf545b9-n999r\" (UID: \"d57edb35-475a-478b-b938-1b9c93165c16\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5cf545b9-n999r" Jul 08 18:46:07.475566 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:07.475412 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d57edb35-475a-478b-b938-1b9c93165c16-manifests\") pod \"opendatahub-operator-controller-manager-6c5cf545b9-n999r\" (UID: \"d57edb35-475a-478b-b938-1b9c93165c16\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5cf545b9-n999r" Jul 08 18:46:07.475566 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:07.475478 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d57edb35-475a-478b-b938-1b9c93165c16-tmp\") pod \"opendatahub-operator-controller-manager-6c5cf545b9-n999r\" (UID: \"d57edb35-475a-478b-b938-1b9c93165c16\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5cf545b9-n999r" Jul 08 18:46:07.475659 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:07.475575 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d57edb35-475a-478b-b938-1b9c93165c16-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c5cf545b9-n999r\" (UID: \"d57edb35-475a-478b-b938-1b9c93165c16\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5cf545b9-n999r" Jul 08 18:46:07.475854 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:07.475837 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c5cf545b9-n999r"] Jul 08 18:46:07.576272 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:07.576230 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-td292\" (UniqueName: \"kubernetes.io/projected/d57edb35-475a-478b-b938-1b9c93165c16-kube-api-access-td292\") pod \"opendatahub-operator-controller-manager-6c5cf545b9-n999r\" (UID: \"d57edb35-475a-478b-b938-1b9c93165c16\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5cf545b9-n999r" Jul 08 18:46:07.576470 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:07.576277 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d57edb35-475a-478b-b938-1b9c93165c16-manifests\") pod \"opendatahub-operator-controller-manager-6c5cf545b9-n999r\" (UID: \"d57edb35-475a-478b-b938-1b9c93165c16\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5cf545b9-n999r" Jul 08 18:46:07.576470 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:07.576322 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d57edb35-475a-478b-b938-1b9c93165c16-tmp\") pod \"opendatahub-operator-controller-manager-6c5cf545b9-n999r\" (UID: \"d57edb35-475a-478b-b938-1b9c93165c16\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5cf545b9-n999r" Jul 08 18:46:07.576470 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:07.576390 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d57edb35-475a-478b-b938-1b9c93165c16-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c5cf545b9-n999r\" (UID: \"d57edb35-475a-478b-b938-1b9c93165c16\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5cf545b9-n999r" Jul 08 18:46:07.576470 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:07.576420 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d57edb35-475a-478b-b938-1b9c93165c16-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c5cf545b9-n999r\" (UID: \"d57edb35-475a-478b-b938-1b9c93165c16\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5cf545b9-n999r" Jul 08 18:46:07.576858 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:07.576821 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d57edb35-475a-478b-b938-1b9c93165c16-manifests\") pod \"opendatahub-operator-controller-manager-6c5cf545b9-n999r\" (UID: \"d57edb35-475a-478b-b938-1b9c93165c16\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5cf545b9-n999r" Jul 08 18:46:07.578685 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:07.578653 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d57edb35-475a-478b-b938-1b9c93165c16-tmp\") pod \"opendatahub-operator-controller-manager-6c5cf545b9-n999r\" (UID: \"d57edb35-475a-478b-b938-1b9c93165c16\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5cf545b9-n999r" Jul 08 18:46:07.578921 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:07.578902 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d57edb35-475a-478b-b938-1b9c93165c16-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c5cf545b9-n999r\" (UID: \"d57edb35-475a-478b-b938-1b9c93165c16\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5cf545b9-n999r" Jul 08 18:46:07.579017 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:07.578995 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d57edb35-475a-478b-b938-1b9c93165c16-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c5cf545b9-n999r\" (UID: \"d57edb35-475a-478b-b938-1b9c93165c16\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5cf545b9-n999r" Jul 08 18:46:07.587869 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:07.587846 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-td292\" (UniqueName: \"kubernetes.io/projected/d57edb35-475a-478b-b938-1b9c93165c16-kube-api-access-td292\") pod \"opendatahub-operator-controller-manager-6c5cf545b9-n999r\" (UID: \"d57edb35-475a-478b-b938-1b9c93165c16\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5cf545b9-n999r" Jul 08 18:46:07.770987 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:07.770891 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6c5cf545b9-n999r" Jul 08 18:46:07.942563 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:07.942532 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c5cf545b9-n999r"] Jul 08 18:46:07.944510 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:46:07.944480 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd57edb35_475a_478b_b938_1b9c93165c16.slice/crio-027fc859ffb9222ba0cd6a89f51748958a832422faa56d40abaa272b08eff768 WatchSource:0}: Error finding container 027fc859ffb9222ba0cd6a89f51748958a832422faa56d40abaa272b08eff768: Status 404 returned error can't find the container with id 027fc859ffb9222ba0cd6a89f51748958a832422faa56d40abaa272b08eff768 Jul 08 18:46:08.626000 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:08.625949 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c5cf545b9-n999r" event={"ID":"d57edb35-475a-478b-b938-1b9c93165c16","Type":"ContainerStarted","Data":"027fc859ffb9222ba0cd6a89f51748958a832422faa56d40abaa272b08eff768"} Jul 08 18:46:11.644406 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:11.644370 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c5cf545b9-n999r" event={"ID":"d57edb35-475a-478b-b938-1b9c93165c16","Type":"ContainerStarted","Data":"16c8826d8f3b4eae20c808bf4bb8ba71542dd25c68b84326d2238391082a64b8"} Jul 08 18:46:12.650633 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:12.650593 2559 generic.go:356] "Generic (PLEG): container finished" podID="d57edb35-475a-478b-b938-1b9c93165c16" containerID="16c8826d8f3b4eae20c808bf4bb8ba71542dd25c68b84326d2238391082a64b8" exitCode=0 Jul 08 18:46:12.651044 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:12.650660 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c5cf545b9-n999r" event={"ID":"d57edb35-475a-478b-b938-1b9c93165c16","Type":"ContainerDied","Data":"16c8826d8f3b4eae20c808bf4bb8ba71542dd25c68b84326d2238391082a64b8"} Jul 08 18:46:13.658304 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:13.658264 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c5cf545b9-n999r" event={"ID":"d57edb35-475a-478b-b938-1b9c93165c16","Type":"ContainerStarted","Data":"4868b5ef2bdabdfe1a5e31e1682704da57b86f3fb4034fd0b733538f85aaabcd"} Jul 08 18:46:13.658807 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:13.658412 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6c5cf545b9-n999r" Jul 08 18:46:13.682218 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:13.682160 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6c5cf545b9-n999r" podStartSLOduration=2.43910582 podStartE2EDuration="6.682140585s" totalImagesPullingTime="3.227366153s" totalInitContainerRuntime="1.015668612s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:46:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:46:13.680391497 +0000 UTC m=+416.916689589" watchObservedRunningTime="2026-07-08 18:46:13.682140585 +0000 UTC m=+416.918438675" Jul 08 18:46:24.665902 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:24.665863 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6c5cf545b9-n999r" Jul 08 18:46:28.983165 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:28.983125 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-56m64"] Jul 08 18:46:28.995685 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:28.995657 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-56m64" Jul 08 18:46:29.002385 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:29.002354 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 08 18:46:29.002669 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:29.002646 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-56m64"] Jul 08 18:46:29.073424 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:29.073385 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2shwg\" (UniqueName: \"kubernetes.io/projected/393da8af-a671-4e2c-8ba2-da604d82cdd3-kube-api-access-2shwg\") pod \"model-serving-api-54d89bd5fc-56m64\" (UID: \"393da8af-a671-4e2c-8ba2-da604d82cdd3\") " pod="opendatahub/model-serving-api-54d89bd5fc-56m64" Jul 08 18:46:29.073603 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:29.073456 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/393da8af-a671-4e2c-8ba2-da604d82cdd3-tls-certs\") pod \"model-serving-api-54d89bd5fc-56m64\" (UID: \"393da8af-a671-4e2c-8ba2-da604d82cdd3\") " pod="opendatahub/model-serving-api-54d89bd5fc-56m64" Jul 08 18:46:29.174301 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:29.174262 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/393da8af-a671-4e2c-8ba2-da604d82cdd3-tls-certs\") pod \"model-serving-api-54d89bd5fc-56m64\" (UID: \"393da8af-a671-4e2c-8ba2-da604d82cdd3\") " pod="opendatahub/model-serving-api-54d89bd5fc-56m64" Jul 08 18:46:29.174488 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:29.174405 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2shwg\" (UniqueName: \"kubernetes.io/projected/393da8af-a671-4e2c-8ba2-da604d82cdd3-kube-api-access-2shwg\") pod \"model-serving-api-54d89bd5fc-56m64\" (UID: \"393da8af-a671-4e2c-8ba2-da604d82cdd3\") " pod="opendatahub/model-serving-api-54d89bd5fc-56m64" Jul 08 18:46:29.174488 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:46:29.174432 2559 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 08 18:46:29.174605 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:46:29.174509 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/393da8af-a671-4e2c-8ba2-da604d82cdd3-tls-certs podName:393da8af-a671-4e2c-8ba2-da604d82cdd3 nodeName:}" failed. No retries permitted until 2026-07-08 18:46:29.674486692 +0000 UTC m=+432.910784765 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/393da8af-a671-4e2c-8ba2-da604d82cdd3-tls-certs") pod "model-serving-api-54d89bd5fc-56m64" (UID: "393da8af-a671-4e2c-8ba2-da604d82cdd3") : secret "model-serving-api-tls" not found Jul 08 18:46:29.187704 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:29.187669 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2shwg\" (UniqueName: \"kubernetes.io/projected/393da8af-a671-4e2c-8ba2-da604d82cdd3-kube-api-access-2shwg\") pod \"model-serving-api-54d89bd5fc-56m64\" (UID: \"393da8af-a671-4e2c-8ba2-da604d82cdd3\") " pod="opendatahub/model-serving-api-54d89bd5fc-56m64" Jul 08 18:46:29.679331 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:29.679289 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/393da8af-a671-4e2c-8ba2-da604d82cdd3-tls-certs\") pod \"model-serving-api-54d89bd5fc-56m64\" (UID: \"393da8af-a671-4e2c-8ba2-da604d82cdd3\") " pod="opendatahub/model-serving-api-54d89bd5fc-56m64" Jul 08 18:46:29.681900 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:29.681873 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/393da8af-a671-4e2c-8ba2-da604d82cdd3-tls-certs\") pod \"model-serving-api-54d89bd5fc-56m64\" (UID: \"393da8af-a671-4e2c-8ba2-da604d82cdd3\") " pod="opendatahub/model-serving-api-54d89bd5fc-56m64" Jul 08 18:46:29.907614 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:29.907557 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-56m64" Jul 08 18:46:30.048694 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:30.048657 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-56m64"] Jul 08 18:46:30.054340 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:46:30.054311 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod393da8af_a671_4e2c_8ba2_da604d82cdd3.slice/crio-b78b5810563854e2abab63ec420c232cb5ade374856340beca0cb0abe7df52ad WatchSource:0}: Error finding container b78b5810563854e2abab63ec420c232cb5ade374856340beca0cb0abe7df52ad: Status 404 returned error can't find the container with id b78b5810563854e2abab63ec420c232cb5ade374856340beca0cb0abe7df52ad Jul 08 18:46:30.751616 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:30.751574 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-56m64" event={"ID":"393da8af-a671-4e2c-8ba2-da604d82cdd3","Type":"ContainerStarted","Data":"b78b5810563854e2abab63ec420c232cb5ade374856340beca0cb0abe7df52ad"} Jul 08 18:46:32.764578 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:32.764534 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-56m64" event={"ID":"393da8af-a671-4e2c-8ba2-da604d82cdd3","Type":"ContainerStarted","Data":"d867f9b71dbd36030582b1aa99d64fc9432d33562293e998c2c82cf868bf10ed"} Jul 08 18:46:32.765011 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:32.764678 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-56m64" Jul 08 18:46:32.783876 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:32.783818 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-56m64" podStartSLOduration=2.952558369 podStartE2EDuration="4.783799918s" totalImagesPullingTime="1.831241549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:46:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:46:32.782686005 +0000 UTC m=+436.018984096" watchObservedRunningTime="2026-07-08 18:46:32.783799918 +0000 UTC m=+436.020098008" Jul 08 18:46:34.741949 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:34.741914 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6782447k"] Jul 08 18:46:34.745485 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:34.745465 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6782447k" Jul 08 18:46:34.748875 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:34.748848 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-l7wqq\"" Jul 08 18:46:34.749041 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:34.748996 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 08 18:46:34.749138 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:34.749119 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 08 18:46:34.749950 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:34.749934 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 08 18:46:34.764672 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:34.764641 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6782447k"] Jul 08 18:46:34.825098 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:34.825066 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa13ab45-55ef-49b7-bd0c-614c3d561a86-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6782447k\" (UID: \"aa13ab45-55ef-49b7-bd0c-614c3d561a86\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6782447k" Jul 08 18:46:34.825098 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:34.825101 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfxc2\" (UniqueName: \"kubernetes.io/projected/aa13ab45-55ef-49b7-bd0c-614c3d561a86-kube-api-access-qfxc2\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6782447k\" (UID: \"aa13ab45-55ef-49b7-bd0c-614c3d561a86\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6782447k" Jul 08 18:46:34.825325 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:34.825186 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa13ab45-55ef-49b7-bd0c-614c3d561a86-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6782447k\" (UID: \"aa13ab45-55ef-49b7-bd0c-614c3d561a86\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6782447k" Jul 08 18:46:34.825325 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:34.825218 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa13ab45-55ef-49b7-bd0c-614c3d561a86-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6782447k\" (UID: \"aa13ab45-55ef-49b7-bd0c-614c3d561a86\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6782447k" Jul 08 18:46:34.825325 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:34.825259 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa13ab45-55ef-49b7-bd0c-614c3d561a86-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6782447k\" (UID: \"aa13ab45-55ef-49b7-bd0c-614c3d561a86\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6782447k" Jul 08 18:46:34.825325 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:34.825319 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa13ab45-55ef-49b7-bd0c-614c3d561a86-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6782447k\" (UID: \"aa13ab45-55ef-49b7-bd0c-614c3d561a86\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6782447k" Jul 08 18:46:34.825490 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:34.825344 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa13ab45-55ef-49b7-bd0c-614c3d561a86-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6782447k\" (UID: \"aa13ab45-55ef-49b7-bd0c-614c3d561a86\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6782447k" Jul 08 18:46:34.825490 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:34.825391 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa13ab45-55ef-49b7-bd0c-614c3d561a86-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6782447k\" (UID: \"aa13ab45-55ef-49b7-bd0c-614c3d561a86\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6782447k" Jul 08 18:46:34.825490 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:34.825420 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa13ab45-55ef-49b7-bd0c-614c3d561a86-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6782447k\" (UID: \"aa13ab45-55ef-49b7-bd0c-614c3d561a86\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6782447k" Jul 08 18:46:34.926363 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:34.926326 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa13ab45-55ef-49b7-bd0c-614c3d561a86-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6782447k\" (UID: \"aa13ab45-55ef-49b7-bd0c-614c3d561a86\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6782447k" Jul 08 18:46:34.926363 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:34.926371 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa13ab45-55ef-49b7-bd0c-614c3d561a86-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6782447k\" (UID: \"aa13ab45-55ef-49b7-bd0c-614c3d561a86\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6782447k" Jul 08 18:46:34.926600 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:34.926403 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa13ab45-55ef-49b7-bd0c-614c3d561a86-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6782447k\" (UID: \"aa13ab45-55ef-49b7-bd0c-614c3d561a86\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6782447k" Jul 08 18:46:34.926600 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:34.926428 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfxc2\" (UniqueName: \"kubernetes.io/projected/aa13ab45-55ef-49b7-bd0c-614c3d561a86-kube-api-access-qfxc2\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6782447k\" (UID: \"aa13ab45-55ef-49b7-bd0c-614c3d561a86\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6782447k" Jul 08 18:46:34.926600 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:34.926464 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa13ab45-55ef-49b7-bd0c-614c3d561a86-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6782447k\" (UID: \"aa13ab45-55ef-49b7-bd0c-614c3d561a86\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6782447k" Jul 08 18:46:34.926600 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:34.926494 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa13ab45-55ef-49b7-bd0c-614c3d561a86-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6782447k\" (UID: \"aa13ab45-55ef-49b7-bd0c-614c3d561a86\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6782447k" Jul 08 18:46:34.926600 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:34.926546 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa13ab45-55ef-49b7-bd0c-614c3d561a86-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6782447k\" (UID: \"aa13ab45-55ef-49b7-bd0c-614c3d561a86\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6782447k" Jul 08 18:46:34.926600 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:34.926590 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa13ab45-55ef-49b7-bd0c-614c3d561a86-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6782447k\" (UID: \"aa13ab45-55ef-49b7-bd0c-614c3d561a86\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6782447k" Jul 08 18:46:34.926958 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:34.926619 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa13ab45-55ef-49b7-bd0c-614c3d561a86-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6782447k\" (UID: \"aa13ab45-55ef-49b7-bd0c-614c3d561a86\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6782447k" Jul 08 18:46:34.926958 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:34.926890 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa13ab45-55ef-49b7-bd0c-614c3d561a86-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6782447k\" (UID: \"aa13ab45-55ef-49b7-bd0c-614c3d561a86\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6782447k" Jul 08 18:46:34.926958 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:34.926911 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa13ab45-55ef-49b7-bd0c-614c3d561a86-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6782447k\" (UID: \"aa13ab45-55ef-49b7-bd0c-614c3d561a86\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6782447k" Jul 08 18:46:34.927119 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:34.926994 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa13ab45-55ef-49b7-bd0c-614c3d561a86-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6782447k\" (UID: \"aa13ab45-55ef-49b7-bd0c-614c3d561a86\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6782447k" Jul 08 18:46:34.927119 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:34.927065 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa13ab45-55ef-49b7-bd0c-614c3d561a86-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6782447k\" (UID: \"aa13ab45-55ef-49b7-bd0c-614c3d561a86\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6782447k" Jul 08 18:46:34.927412 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:34.927388 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa13ab45-55ef-49b7-bd0c-614c3d561a86-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6782447k\" (UID: \"aa13ab45-55ef-49b7-bd0c-614c3d561a86\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6782447k" Jul 08 18:46:34.928903 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:34.928880 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa13ab45-55ef-49b7-bd0c-614c3d561a86-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6782447k\" (UID: \"aa13ab45-55ef-49b7-bd0c-614c3d561a86\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6782447k" Jul 08 18:46:34.929466 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:34.929442 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa13ab45-55ef-49b7-bd0c-614c3d561a86-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6782447k\" (UID: \"aa13ab45-55ef-49b7-bd0c-614c3d561a86\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6782447k" Jul 08 18:46:34.936850 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:34.936821 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa13ab45-55ef-49b7-bd0c-614c3d561a86-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6782447k\" (UID: \"aa13ab45-55ef-49b7-bd0c-614c3d561a86\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6782447k" Jul 08 18:46:34.937066 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:34.937015 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfxc2\" (UniqueName: \"kubernetes.io/projected/aa13ab45-55ef-49b7-bd0c-614c3d561a86-kube-api-access-qfxc2\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6782447k\" (UID: \"aa13ab45-55ef-49b7-bd0c-614c3d561a86\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6782447k" Jul 08 18:46:35.056643 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:35.056563 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6782447k" Jul 08 18:46:35.261397 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:35.261361 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6782447k"] Jul 08 18:46:35.276792 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:46:35.276745 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa13ab45_55ef_49b7_bd0c_614c3d561a86.slice/crio-5cd3d017ac78b32c7f8f354d5213f92efff848d6d65d8934b9688dbacd4232c9 WatchSource:0}: Error finding container 5cd3d017ac78b32c7f8f354d5213f92efff848d6d65d8934b9688dbacd4232c9: Status 404 returned error can't find the container with id 5cd3d017ac78b32c7f8f354d5213f92efff848d6d65d8934b9688dbacd4232c9 Jul 08 18:46:35.788462 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:35.788421 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6782447k" event={"ID":"aa13ab45-55ef-49b7-bd0c-614c3d561a86","Type":"ContainerStarted","Data":"5cd3d017ac78b32c7f8f354d5213f92efff848d6d65d8934b9688dbacd4232c9"} Jul 08 18:46:36.970112 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:36.970074 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-58c5974cc6-cc72p"] Jul 08 18:46:36.973565 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:36.973544 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-58c5974cc6-cc72p" Jul 08 18:46:36.979877 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:36.979852 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 08 18:46:36.980028 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:36.979855 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 08 18:46:36.980167 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:36.980150 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 08 18:46:36.980746 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:36.980720 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-l9x86\"" Jul 08 18:46:36.980875 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:36.980720 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 08 18:46:36.980875 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:36.980837 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 08 18:46:37.023075 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:37.023035 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-58c5974cc6-cc72p"] Jul 08 18:46:37.147699 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:37.147419 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgv8r\" (UniqueName: \"kubernetes.io/projected/078cff9b-7f14-4518-8c61-1551c7355f81-kube-api-access-lgv8r\") pod \"lws-controller-manager-58c5974cc6-cc72p\" (UID: \"078cff9b-7f14-4518-8c61-1551c7355f81\") " pod="openshift-lws-operator/lws-controller-manager-58c5974cc6-cc72p" Jul 08 18:46:37.147699 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:37.147502 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/078cff9b-7f14-4518-8c61-1551c7355f81-manager-config\") pod \"lws-controller-manager-58c5974cc6-cc72p\" (UID: \"078cff9b-7f14-4518-8c61-1551c7355f81\") " pod="openshift-lws-operator/lws-controller-manager-58c5974cc6-cc72p" Jul 08 18:46:37.147699 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:37.147567 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/078cff9b-7f14-4518-8c61-1551c7355f81-metrics-cert\") pod \"lws-controller-manager-58c5974cc6-cc72p\" (UID: \"078cff9b-7f14-4518-8c61-1551c7355f81\") " pod="openshift-lws-operator/lws-controller-manager-58c5974cc6-cc72p" Jul 08 18:46:37.147699 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:37.147615 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/078cff9b-7f14-4518-8c61-1551c7355f81-cert\") pod \"lws-controller-manager-58c5974cc6-cc72p\" (UID: \"078cff9b-7f14-4518-8c61-1551c7355f81\") " pod="openshift-lws-operator/lws-controller-manager-58c5974cc6-cc72p" Jul 08 18:46:37.249284 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:37.249204 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgv8r\" (UniqueName: \"kubernetes.io/projected/078cff9b-7f14-4518-8c61-1551c7355f81-kube-api-access-lgv8r\") pod \"lws-controller-manager-58c5974cc6-cc72p\" (UID: \"078cff9b-7f14-4518-8c61-1551c7355f81\") " pod="openshift-lws-operator/lws-controller-manager-58c5974cc6-cc72p" Jul 08 18:46:37.249284 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:37.249248 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/078cff9b-7f14-4518-8c61-1551c7355f81-manager-config\") pod \"lws-controller-manager-58c5974cc6-cc72p\" (UID: \"078cff9b-7f14-4518-8c61-1551c7355f81\") " pod="openshift-lws-operator/lws-controller-manager-58c5974cc6-cc72p" Jul 08 18:46:37.249750 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:37.249306 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/078cff9b-7f14-4518-8c61-1551c7355f81-metrics-cert\") pod \"lws-controller-manager-58c5974cc6-cc72p\" (UID: \"078cff9b-7f14-4518-8c61-1551c7355f81\") " pod="openshift-lws-operator/lws-controller-manager-58c5974cc6-cc72p" Jul 08 18:46:37.249750 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:37.249342 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/078cff9b-7f14-4518-8c61-1551c7355f81-cert\") pod \"lws-controller-manager-58c5974cc6-cc72p\" (UID: \"078cff9b-7f14-4518-8c61-1551c7355f81\") " pod="openshift-lws-operator/lws-controller-manager-58c5974cc6-cc72p" Jul 08 18:46:37.250172 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:37.250131 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/078cff9b-7f14-4518-8c61-1551c7355f81-manager-config\") pod \"lws-controller-manager-58c5974cc6-cc72p\" (UID: \"078cff9b-7f14-4518-8c61-1551c7355f81\") " pod="openshift-lws-operator/lws-controller-manager-58c5974cc6-cc72p" Jul 08 18:46:37.252576 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:37.252556 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/078cff9b-7f14-4518-8c61-1551c7355f81-metrics-cert\") pod \"lws-controller-manager-58c5974cc6-cc72p\" (UID: \"078cff9b-7f14-4518-8c61-1551c7355f81\") " pod="openshift-lws-operator/lws-controller-manager-58c5974cc6-cc72p" Jul 08 18:46:37.252716 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:37.252694 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/078cff9b-7f14-4518-8c61-1551c7355f81-cert\") pod \"lws-controller-manager-58c5974cc6-cc72p\" (UID: \"078cff9b-7f14-4518-8c61-1551c7355f81\") " pod="openshift-lws-operator/lws-controller-manager-58c5974cc6-cc72p" Jul 08 18:46:37.258885 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:37.258856 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgv8r\" (UniqueName: \"kubernetes.io/projected/078cff9b-7f14-4518-8c61-1551c7355f81-kube-api-access-lgv8r\") pod \"lws-controller-manager-58c5974cc6-cc72p\" (UID: \"078cff9b-7f14-4518-8c61-1551c7355f81\") " pod="openshift-lws-operator/lws-controller-manager-58c5974cc6-cc72p" Jul 08 18:46:37.283457 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:37.283409 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-58c5974cc6-cc72p" Jul 08 18:46:37.470574 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:37.470544 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-58c5974cc6-cc72p"] Jul 08 18:46:37.473236 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:46:37.473197 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod078cff9b_7f14_4518_8c61_1551c7355f81.slice/crio-ae571707e9dda53f27868583be724f0028f00a2f5a43a41454fc2c0649638cfd WatchSource:0}: Error finding container ae571707e9dda53f27868583be724f0028f00a2f5a43a41454fc2c0649638cfd: Status 404 returned error can't find the container with id ae571707e9dda53f27868583be724f0028f00a2f5a43a41454fc2c0649638cfd Jul 08 18:46:37.801265 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:37.801229 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-58c5974cc6-cc72p" event={"ID":"078cff9b-7f14-4518-8c61-1551c7355f81","Type":"ContainerStarted","Data":"ae571707e9dda53f27868583be724f0028f00a2f5a43a41454fc2c0649638cfd"} Jul 08 18:46:39.034054 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:39.034015 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-g7b8j"] Jul 08 18:46:39.053615 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:39.053579 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-589jt"] Jul 08 18:46:39.053796 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:39.053682 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-g7b8j" Jul 08 18:46:39.059007 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:39.058982 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-4zqpq\"" Jul 08 18:46:39.059328 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:39.059304 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 08 18:46:39.059595 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:39.059575 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-g7b8j"] Jul 08 18:46:39.059702 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:39.059661 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-589jt" Jul 08 18:46:39.064190 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:39.064169 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 08 18:46:39.065313 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:39.065290 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-8plvq\"" Jul 08 18:46:39.067837 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:39.067813 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-589jt"] Jul 08 18:46:39.137841 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:39.137802 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 18:46:39.137934 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:39.137894 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 18:46:39.137934 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:39.137928 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 08 18:46:39.167048 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:39.167019 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/83674853-108f-4426-924b-38f003c0949a-cert\") pod \"kserve-controller-manager-d6f4b65b8-g7b8j\" (UID: \"83674853-108f-4426-924b-38f003c0949a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-g7b8j" Jul 08 18:46:39.167170 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:39.167102 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9jq2\" (UniqueName: \"kubernetes.io/projected/b1814dc5-9140-480f-9eb8-de0de2a37225-kube-api-access-n9jq2\") pod \"llmisvc-controller-manager-777c6bd7d5-589jt\" (UID: \"b1814dc5-9140-480f-9eb8-de0de2a37225\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-589jt" Jul 08 18:46:39.167170 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:39.167150 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnx8q\" (UniqueName: \"kubernetes.io/projected/83674853-108f-4426-924b-38f003c0949a-kube-api-access-rnx8q\") pod \"kserve-controller-manager-d6f4b65b8-g7b8j\" (UID: \"83674853-108f-4426-924b-38f003c0949a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-g7b8j" Jul 08 18:46:39.167241 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:39.167206 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b1814dc5-9140-480f-9eb8-de0de2a37225-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-589jt\" (UID: \"b1814dc5-9140-480f-9eb8-de0de2a37225\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-589jt" Jul 08 18:46:39.268176 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:39.268135 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnx8q\" (UniqueName: \"kubernetes.io/projected/83674853-108f-4426-924b-38f003c0949a-kube-api-access-rnx8q\") pod \"kserve-controller-manager-d6f4b65b8-g7b8j\" (UID: \"83674853-108f-4426-924b-38f003c0949a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-g7b8j" Jul 08 18:46:39.268350 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:39.268219 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b1814dc5-9140-480f-9eb8-de0de2a37225-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-589jt\" (UID: \"b1814dc5-9140-480f-9eb8-de0de2a37225\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-589jt" Jul 08 18:46:39.268350 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:39.268254 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/83674853-108f-4426-924b-38f003c0949a-cert\") pod \"kserve-controller-manager-d6f4b65b8-g7b8j\" (UID: \"83674853-108f-4426-924b-38f003c0949a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-g7b8j" Jul 08 18:46:39.268350 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:39.268336 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9jq2\" (UniqueName: \"kubernetes.io/projected/b1814dc5-9140-480f-9eb8-de0de2a37225-kube-api-access-n9jq2\") pod \"llmisvc-controller-manager-777c6bd7d5-589jt\" (UID: \"b1814dc5-9140-480f-9eb8-de0de2a37225\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-589jt" Jul 08 18:46:39.271582 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:39.271554 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/83674853-108f-4426-924b-38f003c0949a-cert\") pod \"kserve-controller-manager-d6f4b65b8-g7b8j\" (UID: \"83674853-108f-4426-924b-38f003c0949a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-g7b8j" Jul 08 18:46:39.271714 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:39.271554 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b1814dc5-9140-480f-9eb8-de0de2a37225-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-589jt\" (UID: \"b1814dc5-9140-480f-9eb8-de0de2a37225\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-589jt" Jul 08 18:46:39.282731 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:39.282678 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnx8q\" (UniqueName: \"kubernetes.io/projected/83674853-108f-4426-924b-38f003c0949a-kube-api-access-rnx8q\") pod \"kserve-controller-manager-d6f4b65b8-g7b8j\" (UID: \"83674853-108f-4426-924b-38f003c0949a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-g7b8j" Jul 08 18:46:39.282731 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:39.282678 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9jq2\" (UniqueName: \"kubernetes.io/projected/b1814dc5-9140-480f-9eb8-de0de2a37225-kube-api-access-n9jq2\") pod \"llmisvc-controller-manager-777c6bd7d5-589jt\" (UID: \"b1814dc5-9140-480f-9eb8-de0de2a37225\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-589jt" Jul 08 18:46:39.368900 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:39.368836 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-g7b8j" Jul 08 18:46:39.375115 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:39.375080 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-589jt" Jul 08 18:46:39.553173 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:39.552723 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-g7b8j"] Jul 08 18:46:39.556036 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:46:39.555998 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83674853_108f_4426_924b_38f003c0949a.slice/crio-bc207620c71423dad8a02755abda7508dfe1d48162ee31888d8eafaad806529d WatchSource:0}: Error finding container bc207620c71423dad8a02755abda7508dfe1d48162ee31888d8eafaad806529d: Status 404 returned error can't find the container with id bc207620c71423dad8a02755abda7508dfe1d48162ee31888d8eafaad806529d Jul 08 18:46:39.582554 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:39.582524 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-589jt"] Jul 08 18:46:39.584666 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:46:39.584610 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb1814dc5_9140_480f_9eb8_de0de2a37225.slice/crio-88ef292a4aa3077d9350471b6bcde8e24034f5ba036b2a99793e163bd6d6b1e4 WatchSource:0}: Error finding container 88ef292a4aa3077d9350471b6bcde8e24034f5ba036b2a99793e163bd6d6b1e4: Status 404 returned error can't find the container with id 88ef292a4aa3077d9350471b6bcde8e24034f5ba036b2a99793e163bd6d6b1e4 Jul 08 18:46:39.814815 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:39.814717 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-589jt" event={"ID":"b1814dc5-9140-480f-9eb8-de0de2a37225","Type":"ContainerStarted","Data":"88ef292a4aa3077d9350471b6bcde8e24034f5ba036b2a99793e163bd6d6b1e4"} Jul 08 18:46:39.816400 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:39.816278 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6782447k" event={"ID":"aa13ab45-55ef-49b7-bd0c-614c3d561a86","Type":"ContainerStarted","Data":"de8fed0666400e91973abd4b917c5920e9e27dafae305f845350208eda675329"} Jul 08 18:46:39.817775 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:39.817738 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-g7b8j" event={"ID":"83674853-108f-4426-924b-38f003c0949a","Type":"ContainerStarted","Data":"bc207620c71423dad8a02755abda7508dfe1d48162ee31888d8eafaad806529d"} Jul 08 18:46:39.841173 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:39.841122 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6782447k" podStartSLOduration=1.9824950609999998 podStartE2EDuration="5.84110459s" totalImagesPullingTime="3.858609529s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:46:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:46:39.839067623 +0000 UTC m=+443.075365715" watchObservedRunningTime="2026-07-08 18:46:39.84110459 +0000 UTC m=+443.077402680" Jul 08 18:46:40.057886 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:40.057838 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6782447k" Jul 08 18:46:40.063933 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:40.063875 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6782447k" Jul 08 18:46:40.827510 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:40.827468 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-58c5974cc6-cc72p" event={"ID":"078cff9b-7f14-4518-8c61-1551c7355f81","Type":"ContainerStarted","Data":"f85a5f2f93066cb9792908541fb8fa36e3a4e5577e76b800c641fe091f2c4f57"} Jul 08 18:46:40.828042 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:40.828016 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6782447k" Jul 08 18:46:40.829235 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:40.829214 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6782447k" Jul 08 18:46:40.851242 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:40.850836 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-58c5974cc6-cc72p" podStartSLOduration=1.72739302 podStartE2EDuration="4.850816086s" totalImagesPullingTime="3.123423066s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:46:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:46:40.847087005 +0000 UTC m=+444.083385102" watchObservedRunningTime="2026-07-08 18:46:40.850816086 +0000 UTC m=+444.087114176" Jul 08 18:46:41.833780 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:41.833733 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-58c5974cc6-cc72p" Jul 08 18:46:43.775190 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:43.775147 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-56m64" Jul 08 18:46:44.852110 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:44.852066 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-g7b8j" event={"ID":"83674853-108f-4426-924b-38f003c0949a","Type":"ContainerStarted","Data":"d00ffbda4e5b3c6cbabe9a5e499940d10dcef0d49727c849d216c5239a056520"} Jul 08 18:46:44.852575 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:44.852221 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-g7b8j" Jul 08 18:46:44.853627 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:44.853606 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-589jt" event={"ID":"b1814dc5-9140-480f-9eb8-de0de2a37225","Type":"ContainerStarted","Data":"7872b6f3f2c5128b3ce91af3b6826f321228413481e36a40c0e1ea5e9245ed7d"} Jul 08 18:46:44.853742 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:44.853722 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-589jt" Jul 08 18:46:44.874062 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:44.874017 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-g7b8j" podStartSLOduration=1.603895504 podStartE2EDuration="5.874004133s" totalImagesPullingTime="4.270108629s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:46:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:46:44.872048469 +0000 UTC m=+448.108346559" watchObservedRunningTime="2026-07-08 18:46:44.874004133 +0000 UTC m=+448.110302223" Jul 08 18:46:44.891352 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:44.891298 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-589jt" podStartSLOduration=1.588565719 podStartE2EDuration="5.891281318s" totalImagesPullingTime="4.302715599s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:46:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:46:44.890274589 +0000 UTC m=+448.126572703" watchObservedRunningTime="2026-07-08 18:46:44.891281318 +0000 UTC m=+448.127579408" Jul 08 18:46:52.841915 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:46:52.841878 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-58c5974cc6-cc72p" Jul 08 18:47:15.861446 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:15.861410 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-589jt" Jul 08 18:47:15.864844 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:15.864825 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-g7b8j" Jul 08 18:47:44.054490 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:44.054451 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759phldm"] Jul 08 18:47:44.058235 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:44.058218 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759phldm" Jul 08 18:47:44.061146 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:44.061120 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-fzfvr\"" Jul 08 18:47:44.061270 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:44.061215 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 18:47:44.062274 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:44.062258 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 18:47:44.067113 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:44.067089 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759phldm"] Jul 08 18:47:44.177660 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:44.177620 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/048eec7c-f6e5-4f32-9085-6d1c85683bd4-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759phldm\" (UID: \"048eec7c-f6e5-4f32-9085-6d1c85683bd4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759phldm" Jul 08 18:47:44.177660 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:44.177663 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/048eec7c-f6e5-4f32-9085-6d1c85683bd4-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759phldm\" (UID: \"048eec7c-f6e5-4f32-9085-6d1c85683bd4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759phldm" Jul 08 18:47:44.178102 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:44.177743 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pwgj\" (UniqueName: \"kubernetes.io/projected/048eec7c-f6e5-4f32-9085-6d1c85683bd4-kube-api-access-8pwgj\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759phldm\" (UID: \"048eec7c-f6e5-4f32-9085-6d1c85683bd4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759phldm" Jul 08 18:47:44.279101 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:44.279066 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/048eec7c-f6e5-4f32-9085-6d1c85683bd4-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759phldm\" (UID: \"048eec7c-f6e5-4f32-9085-6d1c85683bd4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759phldm" Jul 08 18:47:44.279101 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:44.279108 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/048eec7c-f6e5-4f32-9085-6d1c85683bd4-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759phldm\" (UID: \"048eec7c-f6e5-4f32-9085-6d1c85683bd4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759phldm" Jul 08 18:47:44.279305 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:44.279169 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pwgj\" (UniqueName: \"kubernetes.io/projected/048eec7c-f6e5-4f32-9085-6d1c85683bd4-kube-api-access-8pwgj\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759phldm\" (UID: \"048eec7c-f6e5-4f32-9085-6d1c85683bd4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759phldm" Jul 08 18:47:44.279467 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:44.279440 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/048eec7c-f6e5-4f32-9085-6d1c85683bd4-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759phldm\" (UID: \"048eec7c-f6e5-4f32-9085-6d1c85683bd4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759phldm" Jul 08 18:47:44.279535 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:44.279514 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/048eec7c-f6e5-4f32-9085-6d1c85683bd4-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759phldm\" (UID: \"048eec7c-f6e5-4f32-9085-6d1c85683bd4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759phldm" Jul 08 18:47:44.291030 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:44.290997 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pwgj\" (UniqueName: \"kubernetes.io/projected/048eec7c-f6e5-4f32-9085-6d1c85683bd4-kube-api-access-8pwgj\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759phldm\" (UID: \"048eec7c-f6e5-4f32-9085-6d1c85683bd4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759phldm" Jul 08 18:47:44.368806 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:44.368752 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759phldm" Jul 08 18:47:44.506982 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:44.506956 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759phldm"] Jul 08 18:47:44.509305 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:47:44.509277 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod048eec7c_f6e5_4f32_9085_6d1c85683bd4.slice/crio-b73ca7447d6dd7613a6714dc6199d589fc8262c4d5673f34a564c2bf6ee465c3 WatchSource:0}: Error finding container b73ca7447d6dd7613a6714dc6199d589fc8262c4d5673f34a564c2bf6ee465c3: Status 404 returned error can't find the container with id b73ca7447d6dd7613a6714dc6199d589fc8262c4d5673f34a564c2bf6ee465c3 Jul 08 18:47:44.858491 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:44.858407 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vlvcm"] Jul 08 18:47:44.862016 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:44.861999 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vlvcm" Jul 08 18:47:44.872799 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:44.872761 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vlvcm"] Jul 08 18:47:44.986082 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:44.986043 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6fe8b5ae-c089-4636-a79e-f2a4241f443f-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vlvcm\" (UID: \"6fe8b5ae-c089-4636-a79e-f2a4241f443f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vlvcm" Jul 08 18:47:44.986082 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:44.986084 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jl4dv\" (UniqueName: \"kubernetes.io/projected/6fe8b5ae-c089-4636-a79e-f2a4241f443f-kube-api-access-jl4dv\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vlvcm\" (UID: \"6fe8b5ae-c089-4636-a79e-f2a4241f443f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vlvcm" Jul 08 18:47:44.986307 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:44.986104 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6fe8b5ae-c089-4636-a79e-f2a4241f443f-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vlvcm\" (UID: \"6fe8b5ae-c089-4636-a79e-f2a4241f443f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vlvcm" Jul 08 18:47:45.087393 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:45.087352 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6fe8b5ae-c089-4636-a79e-f2a4241f443f-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vlvcm\" (UID: \"6fe8b5ae-c089-4636-a79e-f2a4241f443f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vlvcm" Jul 08 18:47:45.087393 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:45.087394 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jl4dv\" (UniqueName: \"kubernetes.io/projected/6fe8b5ae-c089-4636-a79e-f2a4241f443f-kube-api-access-jl4dv\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vlvcm\" (UID: \"6fe8b5ae-c089-4636-a79e-f2a4241f443f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vlvcm" Jul 08 18:47:45.087859 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:45.087417 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6fe8b5ae-c089-4636-a79e-f2a4241f443f-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vlvcm\" (UID: \"6fe8b5ae-c089-4636-a79e-f2a4241f443f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vlvcm" Jul 08 18:47:45.087859 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:45.087847 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6fe8b5ae-c089-4636-a79e-f2a4241f443f-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vlvcm\" (UID: \"6fe8b5ae-c089-4636-a79e-f2a4241f443f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vlvcm" Jul 08 18:47:45.087932 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:45.087854 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6fe8b5ae-c089-4636-a79e-f2a4241f443f-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vlvcm\" (UID: \"6fe8b5ae-c089-4636-a79e-f2a4241f443f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vlvcm" Jul 08 18:47:45.096221 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:45.096194 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jl4dv\" (UniqueName: \"kubernetes.io/projected/6fe8b5ae-c089-4636-a79e-f2a4241f443f-kube-api-access-jl4dv\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vlvcm\" (UID: \"6fe8b5ae-c089-4636-a79e-f2a4241f443f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vlvcm" Jul 08 18:47:45.159169 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:45.159137 2559 generic.go:356] "Generic (PLEG): container finished" podID="048eec7c-f6e5-4f32-9085-6d1c85683bd4" containerID="19b5d7d962c796a6407634a35a55c97a258c3deeec9799ca2afdb14cdcd1b7ea" exitCode=0 Jul 08 18:47:45.159337 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:45.159225 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759phldm" event={"ID":"048eec7c-f6e5-4f32-9085-6d1c85683bd4","Type":"ContainerDied","Data":"19b5d7d962c796a6407634a35a55c97a258c3deeec9799ca2afdb14cdcd1b7ea"} Jul 08 18:47:45.159337 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:45.159276 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759phldm" event={"ID":"048eec7c-f6e5-4f32-9085-6d1c85683bd4","Type":"ContainerStarted","Data":"b73ca7447d6dd7613a6714dc6199d589fc8262c4d5673f34a564c2bf6ee465c3"} Jul 08 18:47:45.170923 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:45.170901 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vlvcm" Jul 08 18:47:45.322675 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:45.309338 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vlvcm"] Jul 08 18:47:45.327690 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:47:45.327663 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fe8b5ae_c089_4636_a79e_f2a4241f443f.slice/crio-e2a5605d1fefa56783560cd6a74c08fcf177ad29934b9744a1e05ef5ef6ced70 WatchSource:0}: Error finding container e2a5605d1fefa56783560cd6a74c08fcf177ad29934b9744a1e05ef5ef6ced70: Status 404 returned error can't find the container with id e2a5605d1fefa56783560cd6a74c08fcf177ad29934b9744a1e05ef5ef6ced70 Jul 08 18:47:45.653402 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:45.653370 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vlh9b"] Jul 08 18:47:45.657156 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:45.657133 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vlh9b" Jul 08 18:47:45.669646 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:45.669616 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vlh9b"] Jul 08 18:47:45.692977 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:45.692893 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e75d9e1a-840c-417a-a19f-e7eb51aa6d37-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vlh9b\" (UID: \"e75d9e1a-840c-417a-a19f-e7eb51aa6d37\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vlh9b" Jul 08 18:47:45.692977 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:45.692932 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lg4th\" (UniqueName: \"kubernetes.io/projected/e75d9e1a-840c-417a-a19f-e7eb51aa6d37-kube-api-access-lg4th\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vlh9b\" (UID: \"e75d9e1a-840c-417a-a19f-e7eb51aa6d37\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vlh9b" Jul 08 18:47:45.692977 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:45.692960 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e75d9e1a-840c-417a-a19f-e7eb51aa6d37-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vlh9b\" (UID: \"e75d9e1a-840c-417a-a19f-e7eb51aa6d37\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vlh9b" Jul 08 18:47:45.794212 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:45.794171 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e75d9e1a-840c-417a-a19f-e7eb51aa6d37-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vlh9b\" (UID: \"e75d9e1a-840c-417a-a19f-e7eb51aa6d37\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vlh9b" Jul 08 18:47:45.794360 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:45.794229 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lg4th\" (UniqueName: \"kubernetes.io/projected/e75d9e1a-840c-417a-a19f-e7eb51aa6d37-kube-api-access-lg4th\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vlh9b\" (UID: \"e75d9e1a-840c-417a-a19f-e7eb51aa6d37\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vlh9b" Jul 08 18:47:45.794360 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:45.794267 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e75d9e1a-840c-417a-a19f-e7eb51aa6d37-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vlh9b\" (UID: \"e75d9e1a-840c-417a-a19f-e7eb51aa6d37\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vlh9b" Jul 08 18:47:45.794575 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:45.794541 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e75d9e1a-840c-417a-a19f-e7eb51aa6d37-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vlh9b\" (UID: \"e75d9e1a-840c-417a-a19f-e7eb51aa6d37\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vlh9b" Jul 08 18:47:45.794709 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:45.794688 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e75d9e1a-840c-417a-a19f-e7eb51aa6d37-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vlh9b\" (UID: \"e75d9e1a-840c-417a-a19f-e7eb51aa6d37\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vlh9b" Jul 08 18:47:45.803101 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:45.803073 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lg4th\" (UniqueName: \"kubernetes.io/projected/e75d9e1a-840c-417a-a19f-e7eb51aa6d37-kube-api-access-lg4th\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vlh9b\" (UID: \"e75d9e1a-840c-417a-a19f-e7eb51aa6d37\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vlh9b" Jul 08 18:47:45.968050 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:45.968028 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vlh9b" Jul 08 18:47:46.060178 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:46.060121 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f8tt8"] Jul 08 18:47:46.064963 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:46.064939 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f8tt8" Jul 08 18:47:46.072456 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:46.072428 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f8tt8"] Jul 08 18:47:46.097644 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:46.097607 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/34fb2988-24da-4856-89ff-161e7ed86c63-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f8tt8\" (UID: \"34fb2988-24da-4856-89ff-161e7ed86c63\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f8tt8" Jul 08 18:47:46.098078 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:46.097841 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/34fb2988-24da-4856-89ff-161e7ed86c63-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f8tt8\" (UID: \"34fb2988-24da-4856-89ff-161e7ed86c63\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f8tt8" Jul 08 18:47:46.098078 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:46.097879 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpfq9\" (UniqueName: \"kubernetes.io/projected/34fb2988-24da-4856-89ff-161e7ed86c63-kube-api-access-xpfq9\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f8tt8\" (UID: \"34fb2988-24da-4856-89ff-161e7ed86c63\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f8tt8" Jul 08 18:47:46.112540 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:46.112518 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vlh9b"] Jul 08 18:47:46.114313 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:47:46.114290 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode75d9e1a_840c_417a_a19f_e7eb51aa6d37.slice/crio-f3db995f4ecb7779e2acc32c3a2477a56b74447632b5e02ed9f98865517dafdf WatchSource:0}: Error finding container f3db995f4ecb7779e2acc32c3a2477a56b74447632b5e02ed9f98865517dafdf: Status 404 returned error can't find the container with id f3db995f4ecb7779e2acc32c3a2477a56b74447632b5e02ed9f98865517dafdf Jul 08 18:47:46.166417 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:46.166388 2559 generic.go:356] "Generic (PLEG): container finished" podID="048eec7c-f6e5-4f32-9085-6d1c85683bd4" containerID="a0c50655b36cafa24711d8a4976031e3b5db5e4f1a11913bad8101d79a7c5175" exitCode=0 Jul 08 18:47:46.166538 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:46.166460 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759phldm" event={"ID":"048eec7c-f6e5-4f32-9085-6d1c85683bd4","Type":"ContainerDied","Data":"a0c50655b36cafa24711d8a4976031e3b5db5e4f1a11913bad8101d79a7c5175"} Jul 08 18:47:46.168311 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:46.168286 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vlh9b" event={"ID":"e75d9e1a-840c-417a-a19f-e7eb51aa6d37","Type":"ContainerStarted","Data":"f3db995f4ecb7779e2acc32c3a2477a56b74447632b5e02ed9f98865517dafdf"} Jul 08 18:47:46.169796 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:46.169760 2559 generic.go:356] "Generic (PLEG): container finished" podID="6fe8b5ae-c089-4636-a79e-f2a4241f443f" containerID="106a8c5f32a299d3d54591259e43431680e1d5b655e02225a84b7f442384bfbe" exitCode=0 Jul 08 18:47:46.169903 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:46.169868 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vlvcm" event={"ID":"6fe8b5ae-c089-4636-a79e-f2a4241f443f","Type":"ContainerDied","Data":"106a8c5f32a299d3d54591259e43431680e1d5b655e02225a84b7f442384bfbe"} Jul 08 18:47:46.169971 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:46.169901 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vlvcm" event={"ID":"6fe8b5ae-c089-4636-a79e-f2a4241f443f","Type":"ContainerStarted","Data":"e2a5605d1fefa56783560cd6a74c08fcf177ad29934b9744a1e05ef5ef6ced70"} Jul 08 18:47:46.198899 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:46.198871 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/34fb2988-24da-4856-89ff-161e7ed86c63-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f8tt8\" (UID: \"34fb2988-24da-4856-89ff-161e7ed86c63\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f8tt8" Jul 08 18:47:46.199042 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:46.198904 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpfq9\" (UniqueName: \"kubernetes.io/projected/34fb2988-24da-4856-89ff-161e7ed86c63-kube-api-access-xpfq9\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f8tt8\" (UID: \"34fb2988-24da-4856-89ff-161e7ed86c63\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f8tt8" Jul 08 18:47:46.199042 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:46.198991 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/34fb2988-24da-4856-89ff-161e7ed86c63-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f8tt8\" (UID: \"34fb2988-24da-4856-89ff-161e7ed86c63\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f8tt8" Jul 08 18:47:46.199305 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:46.199282 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/34fb2988-24da-4856-89ff-161e7ed86c63-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f8tt8\" (UID: \"34fb2988-24da-4856-89ff-161e7ed86c63\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f8tt8" Jul 08 18:47:46.199357 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:46.199307 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/34fb2988-24da-4856-89ff-161e7ed86c63-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f8tt8\" (UID: \"34fb2988-24da-4856-89ff-161e7ed86c63\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f8tt8" Jul 08 18:47:46.228732 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:46.228703 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpfq9\" (UniqueName: \"kubernetes.io/projected/34fb2988-24da-4856-89ff-161e7ed86c63-kube-api-access-xpfq9\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f8tt8\" (UID: \"34fb2988-24da-4856-89ff-161e7ed86c63\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f8tt8" Jul 08 18:47:46.420416 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:46.420381 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f8tt8" Jul 08 18:47:46.555201 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:46.555176 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f8tt8"] Jul 08 18:47:46.556856 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:47:46.556823 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34fb2988_24da_4856_89ff_161e7ed86c63.slice/crio-3b28706a83aecaad685a9edf5407a94894771cefa9801db46ad91b9576f72935 WatchSource:0}: Error finding container 3b28706a83aecaad685a9edf5407a94894771cefa9801db46ad91b9576f72935: Status 404 returned error can't find the container with id 3b28706a83aecaad685a9edf5407a94894771cefa9801db46ad91b9576f72935 Jul 08 18:47:47.178232 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:47.178193 2559 generic.go:356] "Generic (PLEG): container finished" podID="048eec7c-f6e5-4f32-9085-6d1c85683bd4" containerID="aea01a7589656bdc2db093a1c9340d9d4b5eee8f86e222c126334baeb72e73e0" exitCode=0 Jul 08 18:47:47.178705 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:47.178279 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759phldm" event={"ID":"048eec7c-f6e5-4f32-9085-6d1c85683bd4","Type":"ContainerDied","Data":"aea01a7589656bdc2db093a1c9340d9d4b5eee8f86e222c126334baeb72e73e0"} Jul 08 18:47:47.179684 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:47.179663 2559 generic.go:356] "Generic (PLEG): container finished" podID="e75d9e1a-840c-417a-a19f-e7eb51aa6d37" containerID="907e44fbf52002a56d7e11fbf95b5a69e413f5c975553f135e7e7f631509d1b7" exitCode=0 Jul 08 18:47:47.179797 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:47.179748 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vlh9b" event={"ID":"e75d9e1a-840c-417a-a19f-e7eb51aa6d37","Type":"ContainerDied","Data":"907e44fbf52002a56d7e11fbf95b5a69e413f5c975553f135e7e7f631509d1b7"} Jul 08 18:47:47.181504 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:47.181485 2559 generic.go:356] "Generic (PLEG): container finished" podID="6fe8b5ae-c089-4636-a79e-f2a4241f443f" containerID="4f8bea15e149c9b49290e2a728d99fdb427956ee4c9fc1936ca8c9a01bfbcfd5" exitCode=0 Jul 08 18:47:47.181607 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:47.181563 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vlvcm" event={"ID":"6fe8b5ae-c089-4636-a79e-f2a4241f443f","Type":"ContainerDied","Data":"4f8bea15e149c9b49290e2a728d99fdb427956ee4c9fc1936ca8c9a01bfbcfd5"} Jul 08 18:47:47.183250 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:47.183225 2559 generic.go:356] "Generic (PLEG): container finished" podID="34fb2988-24da-4856-89ff-161e7ed86c63" containerID="de4c53733e4f04e18f0f8b27fa8366122fee58e09b5eaf8fa50164edc0919f23" exitCode=0 Jul 08 18:47:47.183320 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:47.183253 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f8tt8" event={"ID":"34fb2988-24da-4856-89ff-161e7ed86c63","Type":"ContainerDied","Data":"de4c53733e4f04e18f0f8b27fa8366122fee58e09b5eaf8fa50164edc0919f23"} Jul 08 18:47:47.183320 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:47.183292 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f8tt8" event={"ID":"34fb2988-24da-4856-89ff-161e7ed86c63","Type":"ContainerStarted","Data":"3b28706a83aecaad685a9edf5407a94894771cefa9801db46ad91b9576f72935"} Jul 08 18:47:48.190273 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:48.190241 2559 generic.go:356] "Generic (PLEG): container finished" podID="e75d9e1a-840c-417a-a19f-e7eb51aa6d37" containerID="968fc9f9e574409eadc43587bf6714847b7f9968c97aa64145403ec06d42c3d7" exitCode=0 Jul 08 18:47:48.190674 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:48.190320 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vlh9b" event={"ID":"e75d9e1a-840c-417a-a19f-e7eb51aa6d37","Type":"ContainerDied","Data":"968fc9f9e574409eadc43587bf6714847b7f9968c97aa64145403ec06d42c3d7"} Jul 08 18:47:48.192647 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:48.192620 2559 generic.go:356] "Generic (PLEG): container finished" podID="6fe8b5ae-c089-4636-a79e-f2a4241f443f" containerID="0eeb3412684bd54e0fafe880a47dd57495853f0b95cb3603691ce25b12fc591b" exitCode=0 Jul 08 18:47:48.192745 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:48.192696 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vlvcm" event={"ID":"6fe8b5ae-c089-4636-a79e-f2a4241f443f","Type":"ContainerDied","Data":"0eeb3412684bd54e0fafe880a47dd57495853f0b95cb3603691ce25b12fc591b"} Jul 08 18:47:48.194443 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:48.194420 2559 generic.go:356] "Generic (PLEG): container finished" podID="34fb2988-24da-4856-89ff-161e7ed86c63" containerID="a17d30f2b04b11f4ae9dddf657dc0ff194ee2f1defa5af5c71ab9eb5d0787491" exitCode=0 Jul 08 18:47:48.194589 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:48.194493 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f8tt8" event={"ID":"34fb2988-24da-4856-89ff-161e7ed86c63","Type":"ContainerDied","Data":"a17d30f2b04b11f4ae9dddf657dc0ff194ee2f1defa5af5c71ab9eb5d0787491"} Jul 08 18:47:48.435905 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:48.435880 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759phldm" Jul 08 18:47:48.520127 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:48.520091 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/048eec7c-f6e5-4f32-9085-6d1c85683bd4-bundle\") pod \"048eec7c-f6e5-4f32-9085-6d1c85683bd4\" (UID: \"048eec7c-f6e5-4f32-9085-6d1c85683bd4\") " Jul 08 18:47:48.520127 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:48.520137 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/048eec7c-f6e5-4f32-9085-6d1c85683bd4-util\") pod \"048eec7c-f6e5-4f32-9085-6d1c85683bd4\" (UID: \"048eec7c-f6e5-4f32-9085-6d1c85683bd4\") " Jul 08 18:47:48.520348 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:48.520156 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8pwgj\" (UniqueName: \"kubernetes.io/projected/048eec7c-f6e5-4f32-9085-6d1c85683bd4-kube-api-access-8pwgj\") pod \"048eec7c-f6e5-4f32-9085-6d1c85683bd4\" (UID: \"048eec7c-f6e5-4f32-9085-6d1c85683bd4\") " Jul 08 18:47:48.520601 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:48.520572 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/048eec7c-f6e5-4f32-9085-6d1c85683bd4-bundle" (OuterVolumeSpecName: "bundle") pod "048eec7c-f6e5-4f32-9085-6d1c85683bd4" (UID: "048eec7c-f6e5-4f32-9085-6d1c85683bd4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:47:48.522392 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:48.522366 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/048eec7c-f6e5-4f32-9085-6d1c85683bd4-kube-api-access-8pwgj" (OuterVolumeSpecName: "kube-api-access-8pwgj") pod "048eec7c-f6e5-4f32-9085-6d1c85683bd4" (UID: "048eec7c-f6e5-4f32-9085-6d1c85683bd4"). InnerVolumeSpecName "kube-api-access-8pwgj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:47:48.525836 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:48.525815 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/048eec7c-f6e5-4f32-9085-6d1c85683bd4-util" (OuterVolumeSpecName: "util") pod "048eec7c-f6e5-4f32-9085-6d1c85683bd4" (UID: "048eec7c-f6e5-4f32-9085-6d1c85683bd4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:47:48.621268 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:48.621239 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/048eec7c-f6e5-4f32-9085-6d1c85683bd4-bundle\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:47:48.621268 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:48.621268 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/048eec7c-f6e5-4f32-9085-6d1c85683bd4-util\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:47:48.621461 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:48.621283 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8pwgj\" (UniqueName: \"kubernetes.io/projected/048eec7c-f6e5-4f32-9085-6d1c85683bd4-kube-api-access-8pwgj\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:47:49.201206 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:49.201175 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759phldm" event={"ID":"048eec7c-f6e5-4f32-9085-6d1c85683bd4","Type":"ContainerDied","Data":"b73ca7447d6dd7613a6714dc6199d589fc8262c4d5673f34a564c2bf6ee465c3"} Jul 08 18:47:49.201206 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:49.201205 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759phldm" Jul 08 18:47:49.201680 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:49.201223 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b73ca7447d6dd7613a6714dc6199d589fc8262c4d5673f34a564c2bf6ee465c3" Jul 08 18:47:49.203289 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:49.203262 2559 generic.go:356] "Generic (PLEG): container finished" podID="e75d9e1a-840c-417a-a19f-e7eb51aa6d37" containerID="0861d9272347feee32ec98cf32671904a6625e96e4df9b4002e0310add8eafae" exitCode=0 Jul 08 18:47:49.203420 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:49.203339 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vlh9b" event={"ID":"e75d9e1a-840c-417a-a19f-e7eb51aa6d37","Type":"ContainerDied","Data":"0861d9272347feee32ec98cf32671904a6625e96e4df9b4002e0310add8eafae"} Jul 08 18:47:49.205459 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:49.205435 2559 generic.go:356] "Generic (PLEG): container finished" podID="34fb2988-24da-4856-89ff-161e7ed86c63" containerID="07c5a901ea9c383d09f1e39573c9eecbf932e9c438bab252e997dd56ea106d4f" exitCode=0 Jul 08 18:47:49.205599 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:49.205483 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f8tt8" event={"ID":"34fb2988-24da-4856-89ff-161e7ed86c63","Type":"ContainerDied","Data":"07c5a901ea9c383d09f1e39573c9eecbf932e9c438bab252e997dd56ea106d4f"} Jul 08 18:47:49.341361 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:49.341334 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vlvcm" Jul 08 18:47:49.428004 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:49.427970 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jl4dv\" (UniqueName: \"kubernetes.io/projected/6fe8b5ae-c089-4636-a79e-f2a4241f443f-kube-api-access-jl4dv\") pod \"6fe8b5ae-c089-4636-a79e-f2a4241f443f\" (UID: \"6fe8b5ae-c089-4636-a79e-f2a4241f443f\") " Jul 08 18:47:49.428185 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:49.428023 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6fe8b5ae-c089-4636-a79e-f2a4241f443f-util\") pod \"6fe8b5ae-c089-4636-a79e-f2a4241f443f\" (UID: \"6fe8b5ae-c089-4636-a79e-f2a4241f443f\") " Jul 08 18:47:49.428229 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:49.428195 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6fe8b5ae-c089-4636-a79e-f2a4241f443f-bundle\") pod \"6fe8b5ae-c089-4636-a79e-f2a4241f443f\" (UID: \"6fe8b5ae-c089-4636-a79e-f2a4241f443f\") " Jul 08 18:47:49.428731 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:49.428705 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fe8b5ae-c089-4636-a79e-f2a4241f443f-bundle" (OuterVolumeSpecName: "bundle") pod "6fe8b5ae-c089-4636-a79e-f2a4241f443f" (UID: "6fe8b5ae-c089-4636-a79e-f2a4241f443f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:47:49.430042 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:49.430021 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fe8b5ae-c089-4636-a79e-f2a4241f443f-kube-api-access-jl4dv" (OuterVolumeSpecName: "kube-api-access-jl4dv") pod "6fe8b5ae-c089-4636-a79e-f2a4241f443f" (UID: "6fe8b5ae-c089-4636-a79e-f2a4241f443f"). InnerVolumeSpecName "kube-api-access-jl4dv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:47:49.435548 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:49.435522 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fe8b5ae-c089-4636-a79e-f2a4241f443f-util" (OuterVolumeSpecName: "util") pod "6fe8b5ae-c089-4636-a79e-f2a4241f443f" (UID: "6fe8b5ae-c089-4636-a79e-f2a4241f443f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:47:49.529957 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:49.529873 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6fe8b5ae-c089-4636-a79e-f2a4241f443f-bundle\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:47:49.529957 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:49.529900 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jl4dv\" (UniqueName: \"kubernetes.io/projected/6fe8b5ae-c089-4636-a79e-f2a4241f443f-kube-api-access-jl4dv\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:47:49.529957 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:49.529910 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6fe8b5ae-c089-4636-a79e-f2a4241f443f-util\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:47:50.212420 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:50.212392 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vlvcm" Jul 08 18:47:50.212882 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:50.212421 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vlvcm" event={"ID":"6fe8b5ae-c089-4636-a79e-f2a4241f443f","Type":"ContainerDied","Data":"e2a5605d1fefa56783560cd6a74c08fcf177ad29934b9744a1e05ef5ef6ced70"} Jul 08 18:47:50.212882 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:50.212466 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e2a5605d1fefa56783560cd6a74c08fcf177ad29934b9744a1e05ef5ef6ced70" Jul 08 18:47:50.372172 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:50.372148 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f8tt8" Jul 08 18:47:50.374212 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:50.374194 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vlh9b" Jul 08 18:47:50.439610 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:50.439577 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lg4th\" (UniqueName: \"kubernetes.io/projected/e75d9e1a-840c-417a-a19f-e7eb51aa6d37-kube-api-access-lg4th\") pod \"e75d9e1a-840c-417a-a19f-e7eb51aa6d37\" (UID: \"e75d9e1a-840c-417a-a19f-e7eb51aa6d37\") " Jul 08 18:47:50.439799 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:50.439640 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xpfq9\" (UniqueName: \"kubernetes.io/projected/34fb2988-24da-4856-89ff-161e7ed86c63-kube-api-access-xpfq9\") pod \"34fb2988-24da-4856-89ff-161e7ed86c63\" (UID: \"34fb2988-24da-4856-89ff-161e7ed86c63\") " Jul 08 18:47:50.439799 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:50.439677 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/34fb2988-24da-4856-89ff-161e7ed86c63-util\") pod \"34fb2988-24da-4856-89ff-161e7ed86c63\" (UID: \"34fb2988-24da-4856-89ff-161e7ed86c63\") " Jul 08 18:47:50.439799 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:50.439749 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e75d9e1a-840c-417a-a19f-e7eb51aa6d37-util\") pod \"e75d9e1a-840c-417a-a19f-e7eb51aa6d37\" (UID: \"e75d9e1a-840c-417a-a19f-e7eb51aa6d37\") " Jul 08 18:47:50.439974 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:50.439805 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e75d9e1a-840c-417a-a19f-e7eb51aa6d37-bundle\") pod \"e75d9e1a-840c-417a-a19f-e7eb51aa6d37\" (UID: \"e75d9e1a-840c-417a-a19f-e7eb51aa6d37\") " Jul 08 18:47:50.439974 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:50.439829 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/34fb2988-24da-4856-89ff-161e7ed86c63-bundle\") pod \"34fb2988-24da-4856-89ff-161e7ed86c63\" (UID: \"34fb2988-24da-4856-89ff-161e7ed86c63\") " Jul 08 18:47:50.440588 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:50.440553 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e75d9e1a-840c-417a-a19f-e7eb51aa6d37-bundle" (OuterVolumeSpecName: "bundle") pod "e75d9e1a-840c-417a-a19f-e7eb51aa6d37" (UID: "e75d9e1a-840c-417a-a19f-e7eb51aa6d37"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:47:50.440917 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:50.440895 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34fb2988-24da-4856-89ff-161e7ed86c63-bundle" (OuterVolumeSpecName: "bundle") pod "34fb2988-24da-4856-89ff-161e7ed86c63" (UID: "34fb2988-24da-4856-89ff-161e7ed86c63"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:47:50.442355 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:50.442268 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e75d9e1a-840c-417a-a19f-e7eb51aa6d37-kube-api-access-lg4th" (OuterVolumeSpecName: "kube-api-access-lg4th") pod "e75d9e1a-840c-417a-a19f-e7eb51aa6d37" (UID: "e75d9e1a-840c-417a-a19f-e7eb51aa6d37"). InnerVolumeSpecName "kube-api-access-lg4th". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:47:50.442355 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:50.442308 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34fb2988-24da-4856-89ff-161e7ed86c63-kube-api-access-xpfq9" (OuterVolumeSpecName: "kube-api-access-xpfq9") pod "34fb2988-24da-4856-89ff-161e7ed86c63" (UID: "34fb2988-24da-4856-89ff-161e7ed86c63"). InnerVolumeSpecName "kube-api-access-xpfq9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:47:50.446032 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:50.446003 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34fb2988-24da-4856-89ff-161e7ed86c63-util" (OuterVolumeSpecName: "util") pod "34fb2988-24da-4856-89ff-161e7ed86c63" (UID: "34fb2988-24da-4856-89ff-161e7ed86c63"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:47:50.446139 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:50.446104 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e75d9e1a-840c-417a-a19f-e7eb51aa6d37-util" (OuterVolumeSpecName: "util") pod "e75d9e1a-840c-417a-a19f-e7eb51aa6d37" (UID: "e75d9e1a-840c-417a-a19f-e7eb51aa6d37"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:47:50.541540 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:50.541437 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xpfq9\" (UniqueName: \"kubernetes.io/projected/34fb2988-24da-4856-89ff-161e7ed86c63-kube-api-access-xpfq9\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:47:50.541540 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:50.541482 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/34fb2988-24da-4856-89ff-161e7ed86c63-util\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:47:50.541540 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:50.541493 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e75d9e1a-840c-417a-a19f-e7eb51aa6d37-util\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:47:50.541540 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:50.541501 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e75d9e1a-840c-417a-a19f-e7eb51aa6d37-bundle\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:47:50.541540 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:50.541509 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/34fb2988-24da-4856-89ff-161e7ed86c63-bundle\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:47:50.541540 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:50.541517 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lg4th\" (UniqueName: \"kubernetes.io/projected/e75d9e1a-840c-417a-a19f-e7eb51aa6d37-kube-api-access-lg4th\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:47:51.220216 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:51.220186 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vlh9b" Jul 08 18:47:51.220216 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:51.220208 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vlh9b" event={"ID":"e75d9e1a-840c-417a-a19f-e7eb51aa6d37","Type":"ContainerDied","Data":"f3db995f4ecb7779e2acc32c3a2477a56b74447632b5e02ed9f98865517dafdf"} Jul 08 18:47:51.220673 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:51.220260 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f3db995f4ecb7779e2acc32c3a2477a56b74447632b5e02ed9f98865517dafdf" Jul 08 18:47:51.222075 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:51.222050 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f8tt8" event={"ID":"34fb2988-24da-4856-89ff-161e7ed86c63","Type":"ContainerDied","Data":"3b28706a83aecaad685a9edf5407a94894771cefa9801db46ad91b9576f72935"} Jul 08 18:47:51.222191 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:51.222079 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f8tt8" Jul 08 18:47:51.222191 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:51.222090 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3b28706a83aecaad685a9edf5407a94894771cefa9801db46ad91b9576f72935" Jul 08 18:47:53.707228 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.707173 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-56984b4675-zpj7r"] Jul 08 18:47:53.707907 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.707785 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34fb2988-24da-4856-89ff-161e7ed86c63" containerName="pull" Jul 08 18:47:53.707907 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.707807 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="34fb2988-24da-4856-89ff-161e7ed86c63" containerName="pull" Jul 08 18:47:53.707907 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.707823 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34fb2988-24da-4856-89ff-161e7ed86c63" containerName="util" Jul 08 18:47:53.707907 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.707831 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="34fb2988-24da-4856-89ff-161e7ed86c63" containerName="util" Jul 08 18:47:53.707907 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.707842 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34fb2988-24da-4856-89ff-161e7ed86c63" containerName="extract" Jul 08 18:47:53.707907 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.707851 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="34fb2988-24da-4856-89ff-161e7ed86c63" containerName="extract" Jul 08 18:47:53.707907 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.707864 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="048eec7c-f6e5-4f32-9085-6d1c85683bd4" containerName="pull" Jul 08 18:47:53.707907 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.707872 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="048eec7c-f6e5-4f32-9085-6d1c85683bd4" containerName="pull" Jul 08 18:47:53.707907 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.707889 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="048eec7c-f6e5-4f32-9085-6d1c85683bd4" containerName="util" Jul 08 18:47:53.707907 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.707897 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="048eec7c-f6e5-4f32-9085-6d1c85683bd4" containerName="util" Jul 08 18:47:53.707907 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.707909 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fe8b5ae-c089-4636-a79e-f2a4241f443f" containerName="pull" Jul 08 18:47:53.707907 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.707917 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fe8b5ae-c089-4636-a79e-f2a4241f443f" containerName="pull" Jul 08 18:47:53.708558 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.707927 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fe8b5ae-c089-4636-a79e-f2a4241f443f" containerName="extract" Jul 08 18:47:53.708558 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.707934 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fe8b5ae-c089-4636-a79e-f2a4241f443f" containerName="extract" Jul 08 18:47:53.708558 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.707957 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="048eec7c-f6e5-4f32-9085-6d1c85683bd4" containerName="extract" Jul 08 18:47:53.708558 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.707965 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="048eec7c-f6e5-4f32-9085-6d1c85683bd4" containerName="extract" Jul 08 18:47:53.708558 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.707975 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e75d9e1a-840c-417a-a19f-e7eb51aa6d37" containerName="pull" Jul 08 18:47:53.708558 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.707983 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e75d9e1a-840c-417a-a19f-e7eb51aa6d37" containerName="pull" Jul 08 18:47:53.708558 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.707997 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e75d9e1a-840c-417a-a19f-e7eb51aa6d37" containerName="util" Jul 08 18:47:53.708558 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.708005 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e75d9e1a-840c-417a-a19f-e7eb51aa6d37" containerName="util" Jul 08 18:47:53.708558 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.708016 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e75d9e1a-840c-417a-a19f-e7eb51aa6d37" containerName="extract" Jul 08 18:47:53.708558 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.708023 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e75d9e1a-840c-417a-a19f-e7eb51aa6d37" containerName="extract" Jul 08 18:47:53.708558 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.708036 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fe8b5ae-c089-4636-a79e-f2a4241f443f" containerName="util" Jul 08 18:47:53.708558 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.708044 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fe8b5ae-c089-4636-a79e-f2a4241f443f" containerName="util" Jul 08 18:47:53.708558 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.708148 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="048eec7c-f6e5-4f32-9085-6d1c85683bd4" containerName="extract" Jul 08 18:47:53.708558 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.708160 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fe8b5ae-c089-4636-a79e-f2a4241f443f" containerName="extract" Jul 08 18:47:53.708558 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.708168 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="34fb2988-24da-4856-89ff-161e7ed86c63" containerName="extract" Jul 08 18:47:53.708558 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.708175 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e75d9e1a-840c-417a-a19f-e7eb51aa6d37" containerName="extract" Jul 08 18:47:53.713038 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.713011 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56984b4675-zpj7r" Jul 08 18:47:53.723592 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.723567 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56984b4675-zpj7r"] Jul 08 18:47:53.773098 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.773067 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42bceb24-9f7a-4d9f-8368-510f67655caf-oauth-serving-cert\") pod \"console-56984b4675-zpj7r\" (UID: \"42bceb24-9f7a-4d9f-8368-510f67655caf\") " pod="openshift-console/console-56984b4675-zpj7r" Jul 08 18:47:53.773098 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.773107 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42bceb24-9f7a-4d9f-8368-510f67655caf-service-ca\") pod \"console-56984b4675-zpj7r\" (UID: \"42bceb24-9f7a-4d9f-8368-510f67655caf\") " pod="openshift-console/console-56984b4675-zpj7r" Jul 08 18:47:53.773340 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.773180 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42bceb24-9f7a-4d9f-8368-510f67655caf-trusted-ca-bundle\") pod \"console-56984b4675-zpj7r\" (UID: \"42bceb24-9f7a-4d9f-8368-510f67655caf\") " pod="openshift-console/console-56984b4675-zpj7r" Jul 08 18:47:53.773340 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.773198 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7zcb\" (UniqueName: \"kubernetes.io/projected/42bceb24-9f7a-4d9f-8368-510f67655caf-kube-api-access-q7zcb\") pod \"console-56984b4675-zpj7r\" (UID: \"42bceb24-9f7a-4d9f-8368-510f67655caf\") " pod="openshift-console/console-56984b4675-zpj7r" Jul 08 18:47:53.773340 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.773220 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42bceb24-9f7a-4d9f-8368-510f67655caf-console-serving-cert\") pod \"console-56984b4675-zpj7r\" (UID: \"42bceb24-9f7a-4d9f-8368-510f67655caf\") " pod="openshift-console/console-56984b4675-zpj7r" Jul 08 18:47:53.773340 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.773242 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42bceb24-9f7a-4d9f-8368-510f67655caf-console-oauth-config\") pod \"console-56984b4675-zpj7r\" (UID: \"42bceb24-9f7a-4d9f-8368-510f67655caf\") " pod="openshift-console/console-56984b4675-zpj7r" Jul 08 18:47:53.773340 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.773269 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42bceb24-9f7a-4d9f-8368-510f67655caf-console-config\") pod \"console-56984b4675-zpj7r\" (UID: \"42bceb24-9f7a-4d9f-8368-510f67655caf\") " pod="openshift-console/console-56984b4675-zpj7r" Jul 08 18:47:53.874065 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.874029 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42bceb24-9f7a-4d9f-8368-510f67655caf-console-config\") pod \"console-56984b4675-zpj7r\" (UID: \"42bceb24-9f7a-4d9f-8368-510f67655caf\") " pod="openshift-console/console-56984b4675-zpj7r" Jul 08 18:47:53.874065 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.874070 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42bceb24-9f7a-4d9f-8368-510f67655caf-oauth-serving-cert\") pod \"console-56984b4675-zpj7r\" (UID: \"42bceb24-9f7a-4d9f-8368-510f67655caf\") " pod="openshift-console/console-56984b4675-zpj7r" Jul 08 18:47:53.874294 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.874099 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42bceb24-9f7a-4d9f-8368-510f67655caf-service-ca\") pod \"console-56984b4675-zpj7r\" (UID: \"42bceb24-9f7a-4d9f-8368-510f67655caf\") " pod="openshift-console/console-56984b4675-zpj7r" Jul 08 18:47:53.874294 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.874178 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42bceb24-9f7a-4d9f-8368-510f67655caf-trusted-ca-bundle\") pod \"console-56984b4675-zpj7r\" (UID: \"42bceb24-9f7a-4d9f-8368-510f67655caf\") " pod="openshift-console/console-56984b4675-zpj7r" Jul 08 18:47:53.874294 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.874208 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7zcb\" (UniqueName: \"kubernetes.io/projected/42bceb24-9f7a-4d9f-8368-510f67655caf-kube-api-access-q7zcb\") pod \"console-56984b4675-zpj7r\" (UID: \"42bceb24-9f7a-4d9f-8368-510f67655caf\") " pod="openshift-console/console-56984b4675-zpj7r" Jul 08 18:47:53.874294 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.874239 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42bceb24-9f7a-4d9f-8368-510f67655caf-console-serving-cert\") pod \"console-56984b4675-zpj7r\" (UID: \"42bceb24-9f7a-4d9f-8368-510f67655caf\") " pod="openshift-console/console-56984b4675-zpj7r" Jul 08 18:47:53.874294 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.874276 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42bceb24-9f7a-4d9f-8368-510f67655caf-console-oauth-config\") pod \"console-56984b4675-zpj7r\" (UID: \"42bceb24-9f7a-4d9f-8368-510f67655caf\") " pod="openshift-console/console-56984b4675-zpj7r" Jul 08 18:47:53.874864 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.874836 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42bceb24-9f7a-4d9f-8368-510f67655caf-console-config\") pod \"console-56984b4675-zpj7r\" (UID: \"42bceb24-9f7a-4d9f-8368-510f67655caf\") " pod="openshift-console/console-56984b4675-zpj7r" Jul 08 18:47:53.875008 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.874928 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42bceb24-9f7a-4d9f-8368-510f67655caf-oauth-serving-cert\") pod \"console-56984b4675-zpj7r\" (UID: \"42bceb24-9f7a-4d9f-8368-510f67655caf\") " pod="openshift-console/console-56984b4675-zpj7r" Jul 08 18:47:53.875008 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.874952 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42bceb24-9f7a-4d9f-8368-510f67655caf-service-ca\") pod \"console-56984b4675-zpj7r\" (UID: \"42bceb24-9f7a-4d9f-8368-510f67655caf\") " pod="openshift-console/console-56984b4675-zpj7r" Jul 08 18:47:53.875161 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.875144 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42bceb24-9f7a-4d9f-8368-510f67655caf-trusted-ca-bundle\") pod \"console-56984b4675-zpj7r\" (UID: \"42bceb24-9f7a-4d9f-8368-510f67655caf\") " pod="openshift-console/console-56984b4675-zpj7r" Jul 08 18:47:53.876678 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.876654 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42bceb24-9f7a-4d9f-8368-510f67655caf-console-oauth-config\") pod \"console-56984b4675-zpj7r\" (UID: \"42bceb24-9f7a-4d9f-8368-510f67655caf\") " pod="openshift-console/console-56984b4675-zpj7r" Jul 08 18:47:53.876756 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.876735 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42bceb24-9f7a-4d9f-8368-510f67655caf-console-serving-cert\") pod \"console-56984b4675-zpj7r\" (UID: \"42bceb24-9f7a-4d9f-8368-510f67655caf\") " pod="openshift-console/console-56984b4675-zpj7r" Jul 08 18:47:53.884482 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:53.884460 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7zcb\" (UniqueName: \"kubernetes.io/projected/42bceb24-9f7a-4d9f-8368-510f67655caf-kube-api-access-q7zcb\") pod \"console-56984b4675-zpj7r\" (UID: \"42bceb24-9f7a-4d9f-8368-510f67655caf\") " pod="openshift-console/console-56984b4675-zpj7r" Jul 08 18:47:54.023180 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:54.023085 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56984b4675-zpj7r" Jul 08 18:47:54.170877 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:54.170842 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56984b4675-zpj7r"] Jul 08 18:47:54.243848 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:54.243800 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56984b4675-zpj7r" event={"ID":"42bceb24-9f7a-4d9f-8368-510f67655caf","Type":"ContainerStarted","Data":"378e3bcf971581fade390f5f30bde021aa84371bf26bdcb51e51f9e5b9cbc1a4"} Jul 08 18:47:54.243848 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:54.243868 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56984b4675-zpj7r" event={"ID":"42bceb24-9f7a-4d9f-8368-510f67655caf","Type":"ContainerStarted","Data":"c2048886afffce263f21dcfe5d1b5f0d6857a3d894828f3ef92d96994ade8cce"} Jul 08 18:47:54.264524 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:47:54.264472 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-56984b4675-zpj7r" podStartSLOduration=1.264456497 podStartE2EDuration="1.264456497s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:47:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:47:54.263566577 +0000 UTC m=+517.499864678" watchObservedRunningTime="2026-07-08 18:47:54.264456497 +0000 UTC m=+517.500754587" Jul 08 18:48:04.024179 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:04.024122 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56984b4675-zpj7r" Jul 08 18:48:04.024698 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:04.024231 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-56984b4675-zpj7r" Jul 08 18:48:04.029099 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:04.029074 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-56984b4675-zpj7r" Jul 08 18:48:04.304120 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:04.304035 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-56984b4675-zpj7r" Jul 08 18:48:04.395835 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:04.393067 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7dc8c55599-t9v7x"] Jul 08 18:48:13.188951 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:13.188871 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cn2t"] Jul 08 18:48:13.192382 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:13.192360 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cn2t" Jul 08 18:48:13.195433 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:13.195407 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 18:48:13.195546 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:13.195419 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 18:48:13.195591 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:13.195545 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-qw22r\"" Jul 08 18:48:13.210834 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:13.210808 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cn2t"] Jul 08 18:48:13.267632 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:13.267591 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5n7j\" (UniqueName: \"kubernetes.io/projected/5d416660-162e-4f1e-a05e-f9e1dbee4e7b-kube-api-access-n5n7j\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9cn2t\" (UID: \"5d416660-162e-4f1e-a05e-f9e1dbee4e7b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cn2t" Jul 08 18:48:13.267810 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:13.267663 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5d416660-162e-4f1e-a05e-f9e1dbee4e7b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9cn2t\" (UID: \"5d416660-162e-4f1e-a05e-f9e1dbee4e7b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cn2t" Jul 08 18:48:13.368973 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:13.368940 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5n7j\" (UniqueName: \"kubernetes.io/projected/5d416660-162e-4f1e-a05e-f9e1dbee4e7b-kube-api-access-n5n7j\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9cn2t\" (UID: \"5d416660-162e-4f1e-a05e-f9e1dbee4e7b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cn2t" Jul 08 18:48:13.369123 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:13.369000 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5d416660-162e-4f1e-a05e-f9e1dbee4e7b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9cn2t\" (UID: \"5d416660-162e-4f1e-a05e-f9e1dbee4e7b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cn2t" Jul 08 18:48:13.369396 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:13.369376 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5d416660-162e-4f1e-a05e-f9e1dbee4e7b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9cn2t\" (UID: \"5d416660-162e-4f1e-a05e-f9e1dbee4e7b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cn2t" Jul 08 18:48:13.378692 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:13.378663 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5n7j\" (UniqueName: \"kubernetes.io/projected/5d416660-162e-4f1e-a05e-f9e1dbee4e7b-kube-api-access-n5n7j\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9cn2t\" (UID: \"5d416660-162e-4f1e-a05e-f9e1dbee4e7b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cn2t" Jul 08 18:48:13.503472 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:13.503374 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cn2t" Jul 08 18:48:13.647391 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:13.647364 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cn2t"] Jul 08 18:48:13.649878 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:48:13.649847 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d416660_162e_4f1e_a05e_f9e1dbee4e7b.slice/crio-fb44c00404def313143d262305f6ce98888bd9cb18a8d060afa47361de5dbe97 WatchSource:0}: Error finding container fb44c00404def313143d262305f6ce98888bd9cb18a8d060afa47361de5dbe97: Status 404 returned error can't find the container with id fb44c00404def313143d262305f6ce98888bd9cb18a8d060afa47361de5dbe97 Jul 08 18:48:14.369236 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:14.369196 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cn2t" event={"ID":"5d416660-162e-4f1e-a05e-f9e1dbee4e7b","Type":"ContainerStarted","Data":"fb44c00404def313143d262305f6ce98888bd9cb18a8d060afa47361de5dbe97"} Jul 08 18:48:21.417728 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:21.417686 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cn2t" event={"ID":"5d416660-162e-4f1e-a05e-f9e1dbee4e7b","Type":"ContainerStarted","Data":"910a6768c2380994cc3f0863f2af729f3737b2cfdd6723ac63228c6e1873dce4"} Jul 08 18:48:21.418172 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:21.417860 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cn2t" Jul 08 18:48:21.440920 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:21.440866 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cn2t" podStartSLOduration=1.212057818 podStartE2EDuration="8.440849235s" totalImagesPullingTime="7.228791417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:48:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:48:21.437627433 +0000 UTC m=+544.673925523" watchObservedRunningTime="2026-07-08 18:48:21.440849235 +0000 UTC m=+544.677147377" Jul 08 18:48:29.424571 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:29.424525 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7dc8c55599-t9v7x" podUID="dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a" containerName="console" containerID="cri-o://a9e52255429cbe181873b4a7968f0ebc8976032b1dabe76c5a7f437d445b858b" gracePeriod=15 Jul 08 18:48:29.679294 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:29.679236 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dc8c55599-t9v7x" Jul 08 18:48:29.724392 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:29.724355 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vll64\" (UniqueName: \"kubernetes.io/projected/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a-kube-api-access-vll64\") pod \"dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a\" (UID: \"dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a\") " Jul 08 18:48:29.724579 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:29.724416 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a-service-ca\") pod \"dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a\" (UID: \"dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a\") " Jul 08 18:48:29.724579 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:29.724441 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a-oauth-serving-cert\") pod \"dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a\" (UID: \"dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a\") " Jul 08 18:48:29.724579 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:29.724486 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a-trusted-ca-bundle\") pod \"dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a\" (UID: \"dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a\") " Jul 08 18:48:29.724579 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:29.724577 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a-console-serving-cert\") pod \"dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a\" (UID: \"dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a\") " Jul 08 18:48:29.724842 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:29.724609 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a-console-config\") pod \"dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a\" (UID: \"dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a\") " Jul 08 18:48:29.724900 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:29.724841 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a-console-oauth-config\") pod \"dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a\" (UID: \"dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a\") " Jul 08 18:48:29.725183 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:29.725105 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a-service-ca" (OuterVolumeSpecName: "service-ca") pod "dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a" (UID: "dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:48:29.725183 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:29.725161 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a-console-config" (OuterVolumeSpecName: "console-config") pod "dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a" (UID: "dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:48:29.725183 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:29.725170 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a" (UID: "dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:48:29.725404 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:29.725113 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a" (UID: "dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:48:29.725650 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:29.725567 2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a-console-config\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:48:29.725650 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:29.725590 2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a-service-ca\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:48:29.725650 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:29.725603 2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a-oauth-serving-cert\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:48:29.725650 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:29.725616 2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a-trusted-ca-bundle\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:48:29.727700 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:29.727675 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a" (UID: "dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:48:29.727700 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:29.727692 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a" (UID: "dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:48:29.727892 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:29.727679 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a-kube-api-access-vll64" (OuterVolumeSpecName: "kube-api-access-vll64") pod "dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a" (UID: "dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a"). InnerVolumeSpecName "kube-api-access-vll64". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:48:29.826441 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:29.826400 2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a-console-serving-cert\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:48:29.826441 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:29.826438 2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a-console-oauth-config\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:48:29.826654 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:29.826453 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vll64\" (UniqueName: \"kubernetes.io/projected/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a-kube-api-access-vll64\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:48:30.473575 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:30.473537 2559 generic.go:356] "Generic (PLEG): container finished" podID="dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a" containerID="a9e52255429cbe181873b4a7968f0ebc8976032b1dabe76c5a7f437d445b858b" exitCode=0 Jul 08 18:48:30.474063 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:30.473579 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dc8c55599-t9v7x" Jul 08 18:48:30.474063 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:30.473613 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dc8c55599-t9v7x" event={"ID":"dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a","Type":"ContainerDied","Data":"a9e52255429cbe181873b4a7968f0ebc8976032b1dabe76c5a7f437d445b858b"} Jul 08 18:48:30.474063 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:30.473665 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dc8c55599-t9v7x" event={"ID":"dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a","Type":"ContainerDied","Data":"f37851cf4478b2e2a97bf94f91686f0f45c51ad8c4326dbd8566885d3c18a748"} Jul 08 18:48:30.474063 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:30.473684 2559 scope.go:117] "RemoveContainer" containerID="a9e52255429cbe181873b4a7968f0ebc8976032b1dabe76c5a7f437d445b858b" Jul 08 18:48:30.484034 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:30.484017 2559 scope.go:117] "RemoveContainer" containerID="a9e52255429cbe181873b4a7968f0ebc8976032b1dabe76c5a7f437d445b858b" Jul 08 18:48:30.484352 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:48:30.484332 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9e52255429cbe181873b4a7968f0ebc8976032b1dabe76c5a7f437d445b858b\": container with ID starting with a9e52255429cbe181873b4a7968f0ebc8976032b1dabe76c5a7f437d445b858b not found: ID does not exist" containerID="a9e52255429cbe181873b4a7968f0ebc8976032b1dabe76c5a7f437d445b858b" Jul 08 18:48:30.484438 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:30.484361 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9e52255429cbe181873b4a7968f0ebc8976032b1dabe76c5a7f437d445b858b"} err="failed to get container status \"a9e52255429cbe181873b4a7968f0ebc8976032b1dabe76c5a7f437d445b858b\": rpc error: code = NotFound desc = could not find container \"a9e52255429cbe181873b4a7968f0ebc8976032b1dabe76c5a7f437d445b858b\": container with ID starting with a9e52255429cbe181873b4a7968f0ebc8976032b1dabe76c5a7f437d445b858b not found: ID does not exist" Jul 08 18:48:30.497138 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:30.497109 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7dc8c55599-t9v7x"] Jul 08 18:48:30.502114 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:30.502089 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7dc8c55599-t9v7x"] Jul 08 18:48:31.687990 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:31.687961 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a" path="/var/lib/kubelet/pods/dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a/volumes" Jul 08 18:48:32.426179 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:32.426140 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cn2t" Jul 08 18:48:34.136997 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:34.136956 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cn2t"] Jul 08 18:48:34.137449 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:34.137136 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cn2t" podUID="5d416660-162e-4f1e-a05e-f9e1dbee4e7b" containerName="manager" containerID="cri-o://910a6768c2380994cc3f0863f2af729f3737b2cfdd6723ac63228c6e1873dce4" gracePeriod=2 Jul 08 18:48:34.145744 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:34.145537 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cn2t"] Jul 08 18:48:34.186445 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:34.186411 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q6s8f"] Jul 08 18:48:34.187050 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:34.187031 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a" containerName="console" Jul 08 18:48:34.187134 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:34.187052 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a" containerName="console" Jul 08 18:48:34.187134 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:34.187062 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d416660-162e-4f1e-a05e-f9e1dbee4e7b" containerName="manager" Jul 08 18:48:34.187134 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:34.187067 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d416660-162e-4f1e-a05e-f9e1dbee4e7b" containerName="manager" Jul 08 18:48:34.187230 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:34.187145 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d416660-162e-4f1e-a05e-f9e1dbee4e7b" containerName="manager" Jul 08 18:48:34.187230 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:34.187154 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbc9c6d7-beb1-41aa-8dc3-5a0857e8a33a" containerName="console" Jul 08 18:48:34.190253 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:34.190234 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q6s8f" Jul 08 18:48:34.197851 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:48:34.197814 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9cn2t\" is forbidden: User \"system:node:ip-10-0-143-154.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-154.ec2.internal' and this object" podUID="5d416660-162e-4f1e-a05e-f9e1dbee4e7b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cn2t" Jul 08 18:48:34.210416 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:34.210386 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q6s8f"] Jul 08 18:48:34.269064 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:34.269027 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j467m\" (UniqueName: \"kubernetes.io/projected/aa1ebd6b-85d6-488e-98be-0fd932981137-kube-api-access-j467m\") pod \"kuadrant-operator-controller-manager-8684c9ff49-q6s8f\" (UID: \"aa1ebd6b-85d6-488e-98be-0fd932981137\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q6s8f" Jul 08 18:48:34.269199 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:34.269137 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa1ebd6b-85d6-488e-98be-0fd932981137-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-q6s8f\" (UID: \"aa1ebd6b-85d6-488e-98be-0fd932981137\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q6s8f" Jul 08 18:48:34.370532 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:34.370500 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j467m\" (UniqueName: \"kubernetes.io/projected/aa1ebd6b-85d6-488e-98be-0fd932981137-kube-api-access-j467m\") pod \"kuadrant-operator-controller-manager-8684c9ff49-q6s8f\" (UID: \"aa1ebd6b-85d6-488e-98be-0fd932981137\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q6s8f" Jul 08 18:48:34.370694 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:34.370570 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa1ebd6b-85d6-488e-98be-0fd932981137-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-q6s8f\" (UID: \"aa1ebd6b-85d6-488e-98be-0fd932981137\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q6s8f" Jul 08 18:48:34.371142 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:34.371118 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa1ebd6b-85d6-488e-98be-0fd932981137-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-q6s8f\" (UID: \"aa1ebd6b-85d6-488e-98be-0fd932981137\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q6s8f" Jul 08 18:48:34.383507 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:34.383480 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j467m\" (UniqueName: \"kubernetes.io/projected/aa1ebd6b-85d6-488e-98be-0fd932981137-kube-api-access-j467m\") pod \"kuadrant-operator-controller-manager-8684c9ff49-q6s8f\" (UID: \"aa1ebd6b-85d6-488e-98be-0fd932981137\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q6s8f" Jul 08 18:48:34.386730 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:34.386707 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cn2t" Jul 08 18:48:34.389606 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:48:34.389540 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9cn2t\" is forbidden: User \"system:node:ip-10-0-143-154.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-154.ec2.internal' and this object" podUID="5d416660-162e-4f1e-a05e-f9e1dbee4e7b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cn2t" Jul 08 18:48:34.470932 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:34.470893 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5n7j\" (UniqueName: \"kubernetes.io/projected/5d416660-162e-4f1e-a05e-f9e1dbee4e7b-kube-api-access-n5n7j\") pod \"5d416660-162e-4f1e-a05e-f9e1dbee4e7b\" (UID: \"5d416660-162e-4f1e-a05e-f9e1dbee4e7b\") " Jul 08 18:48:34.471136 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:34.471009 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5d416660-162e-4f1e-a05e-f9e1dbee4e7b-extensions-socket-volume\") pod \"5d416660-162e-4f1e-a05e-f9e1dbee4e7b\" (UID: \"5d416660-162e-4f1e-a05e-f9e1dbee4e7b\") " Jul 08 18:48:34.471581 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:34.471549 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d416660-162e-4f1e-a05e-f9e1dbee4e7b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5d416660-162e-4f1e-a05e-f9e1dbee4e7b" (UID: "5d416660-162e-4f1e-a05e-f9e1dbee4e7b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:48:34.473294 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:34.473267 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d416660-162e-4f1e-a05e-f9e1dbee4e7b-kube-api-access-n5n7j" (OuterVolumeSpecName: "kube-api-access-n5n7j") pod "5d416660-162e-4f1e-a05e-f9e1dbee4e7b" (UID: "5d416660-162e-4f1e-a05e-f9e1dbee4e7b"). InnerVolumeSpecName "kube-api-access-n5n7j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:48:34.498381 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:34.498353 2559 generic.go:356] "Generic (PLEG): container finished" podID="5d416660-162e-4f1e-a05e-f9e1dbee4e7b" containerID="910a6768c2380994cc3f0863f2af729f3737b2cfdd6723ac63228c6e1873dce4" exitCode=0 Jul 08 18:48:34.498523 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:34.498396 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cn2t" Jul 08 18:48:34.498523 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:34.498468 2559 scope.go:117] "RemoveContainer" containerID="910a6768c2380994cc3f0863f2af729f3737b2cfdd6723ac63228c6e1873dce4" Jul 08 18:48:34.502169 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:48:34.502138 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9cn2t\" is forbidden: User \"system:node:ip-10-0-143-154.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-154.ec2.internal' and this object" podUID="5d416660-162e-4f1e-a05e-f9e1dbee4e7b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cn2t" Jul 08 18:48:34.508577 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:34.508559 2559 scope.go:117] "RemoveContainer" containerID="910a6768c2380994cc3f0863f2af729f3737b2cfdd6723ac63228c6e1873dce4" Jul 08 18:48:34.508861 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:48:34.508845 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"910a6768c2380994cc3f0863f2af729f3737b2cfdd6723ac63228c6e1873dce4\": container with ID starting with 910a6768c2380994cc3f0863f2af729f3737b2cfdd6723ac63228c6e1873dce4 not found: ID does not exist" containerID="910a6768c2380994cc3f0863f2af729f3737b2cfdd6723ac63228c6e1873dce4" Jul 08 18:48:34.508909 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:34.508871 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"910a6768c2380994cc3f0863f2af729f3737b2cfdd6723ac63228c6e1873dce4"} err="failed to get container status \"910a6768c2380994cc3f0863f2af729f3737b2cfdd6723ac63228c6e1873dce4\": rpc error: code = NotFound desc = could not find container \"910a6768c2380994cc3f0863f2af729f3737b2cfdd6723ac63228c6e1873dce4\": container with ID starting with 910a6768c2380994cc3f0863f2af729f3737b2cfdd6723ac63228c6e1873dce4 not found: ID does not exist" Jul 08 18:48:34.509110 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:48:34.509090 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9cn2t\" is forbidden: User \"system:node:ip-10-0-143-154.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-154.ec2.internal' and this object" podUID="5d416660-162e-4f1e-a05e-f9e1dbee4e7b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cn2t" Jul 08 18:48:34.517347 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:34.517331 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q6s8f" Jul 08 18:48:34.572409 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:34.572374 2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5d416660-162e-4f1e-a05e-f9e1dbee4e7b-extensions-socket-volume\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:48:34.572542 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:34.572410 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5n7j\" (UniqueName: \"kubernetes.io/projected/5d416660-162e-4f1e-a05e-f9e1dbee4e7b-kube-api-access-n5n7j\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:48:34.666903 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:34.666697 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q6s8f"] Jul 08 18:48:34.669974 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:48:34.669950 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa1ebd6b_85d6_488e_98be_0fd932981137.slice/crio-a0168bc42936490c9ad52b16e21cd0b71043aa994deb2ec42232e89aded022ff WatchSource:0}: Error finding container a0168bc42936490c9ad52b16e21cd0b71043aa994deb2ec42232e89aded022ff: Status 404 returned error can't find the container with id a0168bc42936490c9ad52b16e21cd0b71043aa994deb2ec42232e89aded022ff Jul 08 18:48:35.506129 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:35.506090 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q6s8f" event={"ID":"aa1ebd6b-85d6-488e-98be-0fd932981137","Type":"ContainerStarted","Data":"9da56cef40d1fd84f8b4ce47bbc5786b43d2cb244b2a5ac727dcbb8eac32f486"} Jul 08 18:48:35.506620 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:35.506152 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q6s8f" event={"ID":"aa1ebd6b-85d6-488e-98be-0fd932981137","Type":"ContainerStarted","Data":"a0168bc42936490c9ad52b16e21cd0b71043aa994deb2ec42232e89aded022ff"} Jul 08 18:48:35.506620 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:35.506181 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q6s8f" Jul 08 18:48:35.508935 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:48:35.508904 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9cn2t\" is forbidden: User \"system:node:ip-10-0-143-154.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-154.ec2.internal' and this object" podUID="5d416660-162e-4f1e-a05e-f9e1dbee4e7b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cn2t" Jul 08 18:48:35.548431 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:35.548379 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q6s8f" podStartSLOduration=1.548363031 podStartE2EDuration="1.548363031s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:48:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:48:35.545888349 +0000 UTC m=+558.782186440" watchObservedRunningTime="2026-07-08 18:48:35.548363031 +0000 UTC m=+558.784661190" Jul 08 18:48:35.688736 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:35.688699 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d416660-162e-4f1e-a05e-f9e1dbee4e7b" path="/var/lib/kubelet/pods/5d416660-162e-4f1e-a05e-f9e1dbee4e7b/volumes" Jul 08 18:48:46.515080 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:46.515042 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q6s8f" Jul 08 18:48:51.005122 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:51.005080 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q6s8f"] Jul 08 18:48:51.005589 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:51.005345 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q6s8f" podUID="aa1ebd6b-85d6-488e-98be-0fd932981137" containerName="manager" containerID="cri-o://9da56cef40d1fd84f8b4ce47bbc5786b43d2cb244b2a5ac727dcbb8eac32f486" gracePeriod=10 Jul 08 18:48:51.258398 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:51.258336 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q6s8f" Jul 08 18:48:51.288123 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:51.288087 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wt499"] Jul 08 18:48:51.288480 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:51.288468 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa1ebd6b-85d6-488e-98be-0fd932981137" containerName="manager" Jul 08 18:48:51.288540 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:51.288481 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa1ebd6b-85d6-488e-98be-0fd932981137" containerName="manager" Jul 08 18:48:51.288588 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:51.288558 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa1ebd6b-85d6-488e-98be-0fd932981137" containerName="manager" Jul 08 18:48:51.291346 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:51.291267 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wt499" Jul 08 18:48:51.314239 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:51.314209 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wt499"] Jul 08 18:48:51.433857 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:51.433822 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa1ebd6b-85d6-488e-98be-0fd932981137-extensions-socket-volume\") pod \"aa1ebd6b-85d6-488e-98be-0fd932981137\" (UID: \"aa1ebd6b-85d6-488e-98be-0fd932981137\") " Jul 08 18:48:51.434029 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:51.433933 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j467m\" (UniqueName: \"kubernetes.io/projected/aa1ebd6b-85d6-488e-98be-0fd932981137-kube-api-access-j467m\") pod \"aa1ebd6b-85d6-488e-98be-0fd932981137\" (UID: \"aa1ebd6b-85d6-488e-98be-0fd932981137\") " Jul 08 18:48:51.434085 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:51.434069 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/219156f1-1ecb-43ff-b3df-042f5ba9ca77-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-wt499\" (UID: \"219156f1-1ecb-43ff-b3df-042f5ba9ca77\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wt499" Jul 08 18:48:51.434151 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:51.434135 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpf2s\" (UniqueName: \"kubernetes.io/projected/219156f1-1ecb-43ff-b3df-042f5ba9ca77-kube-api-access-lpf2s\") pod \"kuadrant-operator-controller-manager-86878c769f-wt499\" (UID: \"219156f1-1ecb-43ff-b3df-042f5ba9ca77\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wt499" Jul 08 18:48:51.434287 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:51.434249 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa1ebd6b-85d6-488e-98be-0fd932981137-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "aa1ebd6b-85d6-488e-98be-0fd932981137" (UID: "aa1ebd6b-85d6-488e-98be-0fd932981137"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:48:51.435949 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:51.435929 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa1ebd6b-85d6-488e-98be-0fd932981137-kube-api-access-j467m" (OuterVolumeSpecName: "kube-api-access-j467m") pod "aa1ebd6b-85d6-488e-98be-0fd932981137" (UID: "aa1ebd6b-85d6-488e-98be-0fd932981137"). InnerVolumeSpecName "kube-api-access-j467m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:48:51.534744 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:51.534643 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpf2s\" (UniqueName: \"kubernetes.io/projected/219156f1-1ecb-43ff-b3df-042f5ba9ca77-kube-api-access-lpf2s\") pod \"kuadrant-operator-controller-manager-86878c769f-wt499\" (UID: \"219156f1-1ecb-43ff-b3df-042f5ba9ca77\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wt499" Jul 08 18:48:51.534744 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:51.534714 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/219156f1-1ecb-43ff-b3df-042f5ba9ca77-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-wt499\" (UID: \"219156f1-1ecb-43ff-b3df-042f5ba9ca77\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wt499" Jul 08 18:48:51.535000 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:51.534751 2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa1ebd6b-85d6-488e-98be-0fd932981137-extensions-socket-volume\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:48:51.535000 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:51.534761 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j467m\" (UniqueName: \"kubernetes.io/projected/aa1ebd6b-85d6-488e-98be-0fd932981137-kube-api-access-j467m\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:48:51.535122 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:51.535100 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/219156f1-1ecb-43ff-b3df-042f5ba9ca77-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-wt499\" (UID: \"219156f1-1ecb-43ff-b3df-042f5ba9ca77\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wt499" Jul 08 18:48:51.544228 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:51.544203 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpf2s\" (UniqueName: \"kubernetes.io/projected/219156f1-1ecb-43ff-b3df-042f5ba9ca77-kube-api-access-lpf2s\") pod \"kuadrant-operator-controller-manager-86878c769f-wt499\" (UID: \"219156f1-1ecb-43ff-b3df-042f5ba9ca77\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wt499" Jul 08 18:48:51.601929 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:51.601896 2559 generic.go:356] "Generic (PLEG): container finished" podID="aa1ebd6b-85d6-488e-98be-0fd932981137" containerID="9da56cef40d1fd84f8b4ce47bbc5786b43d2cb244b2a5ac727dcbb8eac32f486" exitCode=0 Jul 08 18:48:51.602102 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:51.601934 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wt499" Jul 08 18:48:51.602102 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:51.601945 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q6s8f" Jul 08 18:48:51.602102 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:51.601986 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q6s8f" event={"ID":"aa1ebd6b-85d6-488e-98be-0fd932981137","Type":"ContainerDied","Data":"9da56cef40d1fd84f8b4ce47bbc5786b43d2cb244b2a5ac727dcbb8eac32f486"} Jul 08 18:48:51.602102 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:51.602041 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q6s8f" event={"ID":"aa1ebd6b-85d6-488e-98be-0fd932981137","Type":"ContainerDied","Data":"a0168bc42936490c9ad52b16e21cd0b71043aa994deb2ec42232e89aded022ff"} Jul 08 18:48:51.602102 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:51.602064 2559 scope.go:117] "RemoveContainer" containerID="9da56cef40d1fd84f8b4ce47bbc5786b43d2cb244b2a5ac727dcbb8eac32f486" Jul 08 18:48:51.616835 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:51.616818 2559 scope.go:117] "RemoveContainer" containerID="9da56cef40d1fd84f8b4ce47bbc5786b43d2cb244b2a5ac727dcbb8eac32f486" Jul 08 18:48:51.617113 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:48:51.617092 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9da56cef40d1fd84f8b4ce47bbc5786b43d2cb244b2a5ac727dcbb8eac32f486\": container with ID starting with 9da56cef40d1fd84f8b4ce47bbc5786b43d2cb244b2a5ac727dcbb8eac32f486 not found: ID does not exist" containerID="9da56cef40d1fd84f8b4ce47bbc5786b43d2cb244b2a5ac727dcbb8eac32f486" Jul 08 18:48:51.617196 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:51.617120 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9da56cef40d1fd84f8b4ce47bbc5786b43d2cb244b2a5ac727dcbb8eac32f486"} err="failed to get container status \"9da56cef40d1fd84f8b4ce47bbc5786b43d2cb244b2a5ac727dcbb8eac32f486\": rpc error: code = NotFound desc = could not find container \"9da56cef40d1fd84f8b4ce47bbc5786b43d2cb244b2a5ac727dcbb8eac32f486\": container with ID starting with 9da56cef40d1fd84f8b4ce47bbc5786b43d2cb244b2a5ac727dcbb8eac32f486 not found: ID does not exist" Jul 08 18:48:51.628284 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:51.628254 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q6s8f"] Jul 08 18:48:51.641312 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:51.641283 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q6s8f"] Jul 08 18:48:51.688547 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:51.688517 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa1ebd6b-85d6-488e-98be-0fd932981137" path="/var/lib/kubelet/pods/aa1ebd6b-85d6-488e-98be-0fd932981137/volumes" Jul 08 18:48:51.748685 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:51.748664 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wt499"] Jul 08 18:48:51.750851 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:48:51.750826 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod219156f1_1ecb_43ff_b3df_042f5ba9ca77.slice/crio-4b94e5e76094655877a99a4c833891f5e880380ed7d3dcf0b0998fc075464274 WatchSource:0}: Error finding container 4b94e5e76094655877a99a4c833891f5e880380ed7d3dcf0b0998fc075464274: Status 404 returned error can't find the container with id 4b94e5e76094655877a99a4c833891f5e880380ed7d3dcf0b0998fc075464274 Jul 08 18:48:52.610738 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:52.610703 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wt499" event={"ID":"219156f1-1ecb-43ff-b3df-042f5ba9ca77","Type":"ContainerStarted","Data":"7eda3f439b9bb45b515c768ad995b329ab785939182a47e9e5449e25525b0b19"} Jul 08 18:48:52.611317 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:52.610758 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wt499" event={"ID":"219156f1-1ecb-43ff-b3df-042f5ba9ca77","Type":"ContainerStarted","Data":"4b94e5e76094655877a99a4c833891f5e880380ed7d3dcf0b0998fc075464274"} Jul 08 18:48:52.611317 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:52.610816 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wt499" Jul 08 18:48:52.635696 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:48:52.635639 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wt499" podStartSLOduration=1.635624747 podStartE2EDuration="1.635624747s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:48:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:48:52.632150477 +0000 UTC m=+575.868448568" watchObservedRunningTime="2026-07-08 18:48:52.635624747 +0000 UTC m=+575.871922837" Jul 08 18:49:03.619518 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:03.619485 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wt499" Jul 08 18:49:17.214533 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:17.214504 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zzt4h_d94a3566-fcf3-47a1-b176-a33b71c3826d/console-operator/1.log" Jul 08 18:49:17.220984 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:17.220962 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rj4xk_56014c3a-384a-4775-96e5-290968b196b1/ovn-acl-logging/0.log" Jul 08 18:49:17.674612 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:17.674585 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zzt4h_d94a3566-fcf3-47a1-b176-a33b71c3826d/console-operator/1.log" Jul 08 18:49:17.680890 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:17.680867 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rj4xk_56014c3a-384a-4775-96e5-290968b196b1/ovn-acl-logging/0.log" Jul 08 18:49:25.266369 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:25.266329 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-6tljr"] Jul 08 18:49:25.271334 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:25.271308 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6tljr" Jul 08 18:49:25.273969 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:25.273945 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-9gnmc\"" Jul 08 18:49:25.277082 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:25.277059 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-6tljr"] Jul 08 18:49:25.372544 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:25.372504 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twhhv\" (UniqueName: \"kubernetes.io/projected/f3f2c24a-a324-4828-85b7-e36b413cf6a8-kube-api-access-twhhv\") pod \"authorino-58cd955675-6tljr\" (UID: \"f3f2c24a-a324-4828-85b7-e36b413cf6a8\") " pod="kuadrant-system/authorino-58cd955675-6tljr" Jul 08 18:49:25.473922 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:25.473886 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twhhv\" (UniqueName: \"kubernetes.io/projected/f3f2c24a-a324-4828-85b7-e36b413cf6a8-kube-api-access-twhhv\") pod \"authorino-58cd955675-6tljr\" (UID: \"f3f2c24a-a324-4828-85b7-e36b413cf6a8\") " pod="kuadrant-system/authorino-58cd955675-6tljr" Jul 08 18:49:25.482731 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:25.482684 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-twhhv\" (UniqueName: \"kubernetes.io/projected/f3f2c24a-a324-4828-85b7-e36b413cf6a8-kube-api-access-twhhv\") pod \"authorino-58cd955675-6tljr\" (UID: \"f3f2c24a-a324-4828-85b7-e36b413cf6a8\") " pod="kuadrant-system/authorino-58cd955675-6tljr" Jul 08 18:49:25.578876 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:25.578764 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-l2mj6"] Jul 08 18:49:25.582219 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:25.582189 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6tljr" Jul 08 18:49:25.582588 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:25.582568 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-l2mj6" Jul 08 18:49:25.590345 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:25.590317 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-l2mj6"] Jul 08 18:49:25.726337 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:25.726312 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-6tljr"] Jul 08 18:49:25.728002 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:49:25.727963 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf3f2c24a_a324_4828_85b7_e36b413cf6a8.slice/crio-bcab8fb00978500eeafb5e12414530de518aae226b1d69db9597de6a2153280e WatchSource:0}: Error finding container bcab8fb00978500eeafb5e12414530de518aae226b1d69db9597de6a2153280e: Status 404 returned error can't find the container with id bcab8fb00978500eeafb5e12414530de518aae226b1d69db9597de6a2153280e Jul 08 18:49:25.776721 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:25.776684 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wccth\" (UniqueName: \"kubernetes.io/projected/01d0d168-da2a-4064-8d1a-b9c2fa3f2097-kube-api-access-wccth\") pod \"authorino-75d556bd77-l2mj6\" (UID: \"01d0d168-da2a-4064-8d1a-b9c2fa3f2097\") " pod="kuadrant-system/authorino-75d556bd77-l2mj6" Jul 08 18:49:25.807901 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:25.807863 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6tljr" event={"ID":"f3f2c24a-a324-4828-85b7-e36b413cf6a8","Type":"ContainerStarted","Data":"bcab8fb00978500eeafb5e12414530de518aae226b1d69db9597de6a2153280e"} Jul 08 18:49:25.877795 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:25.877733 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wccth\" (UniqueName: \"kubernetes.io/projected/01d0d168-da2a-4064-8d1a-b9c2fa3f2097-kube-api-access-wccth\") pod \"authorino-75d556bd77-l2mj6\" (UID: \"01d0d168-da2a-4064-8d1a-b9c2fa3f2097\") " pod="kuadrant-system/authorino-75d556bd77-l2mj6" Jul 08 18:49:25.885361 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:25.885332 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wccth\" (UniqueName: \"kubernetes.io/projected/01d0d168-da2a-4064-8d1a-b9c2fa3f2097-kube-api-access-wccth\") pod \"authorino-75d556bd77-l2mj6\" (UID: \"01d0d168-da2a-4064-8d1a-b9c2fa3f2097\") " pod="kuadrant-system/authorino-75d556bd77-l2mj6" Jul 08 18:49:25.911018 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:25.910983 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-l2mj6" Jul 08 18:49:26.046855 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:26.046829 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-l2mj6"] Jul 08 18:49:26.049870 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:49:26.049833 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod01d0d168_da2a_4064_8d1a_b9c2fa3f2097.slice/crio-9ea16f04b4998bbaab29415d15fd12adcfdfea5ae92c75c45dc9fa207f6422c8 WatchSource:0}: Error finding container 9ea16f04b4998bbaab29415d15fd12adcfdfea5ae92c75c45dc9fa207f6422c8: Status 404 returned error can't find the container with id 9ea16f04b4998bbaab29415d15fd12adcfdfea5ae92c75c45dc9fa207f6422c8 Jul 08 18:49:26.821407 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:26.821335 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-l2mj6" event={"ID":"01d0d168-da2a-4064-8d1a-b9c2fa3f2097","Type":"ContainerStarted","Data":"9ea16f04b4998bbaab29415d15fd12adcfdfea5ae92c75c45dc9fa207f6422c8"} Jul 08 18:49:29.845312 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:29.845277 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-l2mj6" event={"ID":"01d0d168-da2a-4064-8d1a-b9c2fa3f2097","Type":"ContainerStarted","Data":"9439dbdc6bfefa43d69908e248ef32537381637c35a22bf4af292296023fa7e6"} Jul 08 18:49:29.846827 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:29.846798 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6tljr" event={"ID":"f3f2c24a-a324-4828-85b7-e36b413cf6a8","Type":"ContainerStarted","Data":"42c69ed6d02330636f929b32d65fc7d21322d2e3f85e5285def513bfd111b29c"} Jul 08 18:49:29.860734 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:29.860682 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-l2mj6" podStartSLOduration=1.5528839840000002 podStartE2EDuration="4.86066734s" totalImagesPullingTime="3.307783356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:49:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:49:29.860313076 +0000 UTC m=+613.096611166" watchObservedRunningTime="2026-07-08 18:49:29.86066734 +0000 UTC m=+613.096965430" Jul 08 18:49:29.876799 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:29.876735 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-6tljr" podStartSLOduration=1.258552879 podStartE2EDuration="4.876716702s" totalImagesPullingTime="3.618163823s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:49:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:49:29.874971453 +0000 UTC m=+613.111269540" watchObservedRunningTime="2026-07-08 18:49:29.876716702 +0000 UTC m=+613.113014792" Jul 08 18:49:29.900628 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:29.900590 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-6tljr"] Jul 08 18:49:31.858725 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:31.858685 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-6tljr" podUID="f3f2c24a-a324-4828-85b7-e36b413cf6a8" containerName="authorino" containerID="cri-o://42c69ed6d02330636f929b32d65fc7d21322d2e3f85e5285def513bfd111b29c" gracePeriod=30 Jul 08 18:49:32.117689 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:32.117667 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6tljr" Jul 08 18:49:32.145340 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:32.145297 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-twhhv\" (UniqueName: \"kubernetes.io/projected/f3f2c24a-a324-4828-85b7-e36b413cf6a8-kube-api-access-twhhv\") pod \"f3f2c24a-a324-4828-85b7-e36b413cf6a8\" (UID: \"f3f2c24a-a324-4828-85b7-e36b413cf6a8\") " Jul 08 18:49:32.147526 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:32.147496 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3f2c24a-a324-4828-85b7-e36b413cf6a8-kube-api-access-twhhv" (OuterVolumeSpecName: "kube-api-access-twhhv") pod "f3f2c24a-a324-4828-85b7-e36b413cf6a8" (UID: "f3f2c24a-a324-4828-85b7-e36b413cf6a8"). InnerVolumeSpecName "kube-api-access-twhhv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:49:32.247091 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:32.247058 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twhhv\" (UniqueName: \"kubernetes.io/projected/f3f2c24a-a324-4828-85b7-e36b413cf6a8-kube-api-access-twhhv\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:49:32.865588 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:32.865552 2559 generic.go:356] "Generic (PLEG): container finished" podID="f3f2c24a-a324-4828-85b7-e36b413cf6a8" containerID="42c69ed6d02330636f929b32d65fc7d21322d2e3f85e5285def513bfd111b29c" exitCode=0 Jul 08 18:49:32.865588 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:32.865591 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6tljr" Jul 08 18:49:32.866102 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:32.865632 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6tljr" event={"ID":"f3f2c24a-a324-4828-85b7-e36b413cf6a8","Type":"ContainerDied","Data":"42c69ed6d02330636f929b32d65fc7d21322d2e3f85e5285def513bfd111b29c"} Jul 08 18:49:32.866102 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:32.865675 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6tljr" event={"ID":"f3f2c24a-a324-4828-85b7-e36b413cf6a8","Type":"ContainerDied","Data":"bcab8fb00978500eeafb5e12414530de518aae226b1d69db9597de6a2153280e"} Jul 08 18:49:32.866102 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:32.865691 2559 scope.go:117] "RemoveContainer" containerID="42c69ed6d02330636f929b32d65fc7d21322d2e3f85e5285def513bfd111b29c" Jul 08 18:49:32.875354 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:32.875337 2559 scope.go:117] "RemoveContainer" containerID="42c69ed6d02330636f929b32d65fc7d21322d2e3f85e5285def513bfd111b29c" Jul 08 18:49:32.875654 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:49:32.875633 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42c69ed6d02330636f929b32d65fc7d21322d2e3f85e5285def513bfd111b29c\": container with ID starting with 42c69ed6d02330636f929b32d65fc7d21322d2e3f85e5285def513bfd111b29c not found: ID does not exist" containerID="42c69ed6d02330636f929b32d65fc7d21322d2e3f85e5285def513bfd111b29c" Jul 08 18:49:32.875705 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:32.875663 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42c69ed6d02330636f929b32d65fc7d21322d2e3f85e5285def513bfd111b29c"} err="failed to get container status \"42c69ed6d02330636f929b32d65fc7d21322d2e3f85e5285def513bfd111b29c\": rpc error: code = NotFound desc = could not find container \"42c69ed6d02330636f929b32d65fc7d21322d2e3f85e5285def513bfd111b29c\": container with ID starting with 42c69ed6d02330636f929b32d65fc7d21322d2e3f85e5285def513bfd111b29c not found: ID does not exist" Jul 08 18:49:32.888111 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:32.888090 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-6tljr"] Jul 08 18:49:32.892415 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:32.892395 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-6tljr"] Jul 08 18:49:33.688738 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:33.688702 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3f2c24a-a324-4828-85b7-e36b413cf6a8" path="/var/lib/kubelet/pods/f3f2c24a-a324-4828-85b7-e36b413cf6a8/volumes" Jul 08 18:49:44.209364 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:44.209318 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-bzp22"] Jul 08 18:49:44.215110 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:44.209902 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3f2c24a-a324-4828-85b7-e36b413cf6a8" containerName="authorino" Jul 08 18:49:44.215110 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:44.209918 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3f2c24a-a324-4828-85b7-e36b413cf6a8" containerName="authorino" Jul 08 18:49:44.215110 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:44.210009 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3f2c24a-a324-4828-85b7-e36b413cf6a8" containerName="authorino" Jul 08 18:49:44.216029 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:44.216011 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bzp22" Jul 08 18:49:44.220468 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:44.220442 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 18:49:44.220468 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:44.220464 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 08 18:49:44.220706 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:44.220571 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-kw5xt\"" Jul 08 18:49:44.220793 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:44.220720 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 18:49:44.224199 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:44.224177 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-bzp22"] Jul 08 18:49:44.353444 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:44.353414 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t4mg\" (UniqueName: \"kubernetes.io/projected/8b936c13-06a1-4f83-ae72-8d1fde896eab-kube-api-access-5t4mg\") pod \"postgres-69fb4594cf-bzp22\" (UID: \"8b936c13-06a1-4f83-ae72-8d1fde896eab\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bzp22" Jul 08 18:49:44.353621 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:44.353472 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/8b936c13-06a1-4f83-ae72-8d1fde896eab-data\") pod \"postgres-69fb4594cf-bzp22\" (UID: \"8b936c13-06a1-4f83-ae72-8d1fde896eab\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bzp22" Jul 08 18:49:44.454589 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:44.454548 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/8b936c13-06a1-4f83-ae72-8d1fde896eab-data\") pod \"postgres-69fb4594cf-bzp22\" (UID: \"8b936c13-06a1-4f83-ae72-8d1fde896eab\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bzp22" Jul 08 18:49:44.454823 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:44.454651 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t4mg\" (UniqueName: \"kubernetes.io/projected/8b936c13-06a1-4f83-ae72-8d1fde896eab-kube-api-access-5t4mg\") pod \"postgres-69fb4594cf-bzp22\" (UID: \"8b936c13-06a1-4f83-ae72-8d1fde896eab\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bzp22" Jul 08 18:49:44.454980 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:44.454960 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/8b936c13-06a1-4f83-ae72-8d1fde896eab-data\") pod \"postgres-69fb4594cf-bzp22\" (UID: \"8b936c13-06a1-4f83-ae72-8d1fde896eab\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bzp22" Jul 08 18:49:44.463506 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:44.463455 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t4mg\" (UniqueName: \"kubernetes.io/projected/8b936c13-06a1-4f83-ae72-8d1fde896eab-kube-api-access-5t4mg\") pod \"postgres-69fb4594cf-bzp22\" (UID: \"8b936c13-06a1-4f83-ae72-8d1fde896eab\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bzp22" Jul 08 18:49:44.527548 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:44.527511 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bzp22" Jul 08 18:49:44.667373 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:44.667344 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-bzp22"] Jul 08 18:49:44.933157 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:44.933122 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bzp22" event={"ID":"8b936c13-06a1-4f83-ae72-8d1fde896eab","Type":"ContainerStarted","Data":"af52265bdbd91fcbfec8b4d0541be9c78a8a1e23632228386479e524a06859a4"} Jul 08 18:49:57.006199 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:57.006155 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bzp22" event={"ID":"8b936c13-06a1-4f83-ae72-8d1fde896eab","Type":"ContainerStarted","Data":"89d1e2e3d903b3a4844f0827e873f3bf715cdc8a1aa5a5a39f5861834c3732f7"} Jul 08 18:49:57.006717 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:57.006305 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bzp22" Jul 08 18:49:57.026635 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:49:57.026585 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bzp22" podStartSLOduration=1.544611927 podStartE2EDuration="13.026568198s" totalImagesPullingTime="11.481956271s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:49:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:49:57.023224694 +0000 UTC m=+640.259522786" watchObservedRunningTime="2026-07-08 18:49:57.026568198 +0000 UTC m=+640.262866287" Jul 08 18:50:03.042802 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:03.042755 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bzp22" Jul 08 18:50:03.665194 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:03.665161 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wndmt"] Jul 08 18:50:03.669031 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:03.669009 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-wndmt" Jul 08 18:50:03.678000 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:03.677971 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wndmt"] Jul 08 18:50:03.835981 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:03.835930 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfxlb\" (UniqueName: \"kubernetes.io/projected/5bbf57dc-1a2a-4572-8ca5-00f29c0b847b-kube-api-access-cfxlb\") pod \"authorino-5d4b67c446-wndmt\" (UID: \"5bbf57dc-1a2a-4572-8ca5-00f29c0b847b\") " pod="kuadrant-system/authorino-5d4b67c446-wndmt" Jul 08 18:50:03.880024 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:03.879992 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wndmt"] Jul 08 18:50:03.880342 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:50:03.880323 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-cfxlb], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-wndmt" podUID="5bbf57dc-1a2a-4572-8ca5-00f29c0b847b" Jul 08 18:50:03.908012 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:03.907982 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5ccbc9b875-wdprt"] Jul 08 18:50:03.912029 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:03.912008 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5ccbc9b875-wdprt" Jul 08 18:50:03.914862 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:03.914832 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 08 18:50:03.921304 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:03.921281 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5ccbc9b875-wdprt"] Jul 08 18:50:03.937347 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:03.937323 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfxlb\" (UniqueName: \"kubernetes.io/projected/5bbf57dc-1a2a-4572-8ca5-00f29c0b847b-kube-api-access-cfxlb\") pod \"authorino-5d4b67c446-wndmt\" (UID: \"5bbf57dc-1a2a-4572-8ca5-00f29c0b847b\") " pod="kuadrant-system/authorino-5d4b67c446-wndmt" Jul 08 18:50:03.945362 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:03.945336 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfxlb\" (UniqueName: \"kubernetes.io/projected/5bbf57dc-1a2a-4572-8ca5-00f29c0b847b-kube-api-access-cfxlb\") pod \"authorino-5d4b67c446-wndmt\" (UID: \"5bbf57dc-1a2a-4572-8ca5-00f29c0b847b\") " pod="kuadrant-system/authorino-5d4b67c446-wndmt" Jul 08 18:50:04.038207 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:04.038169 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2c82473b-6248-42e7-8cbe-65645822dd48-tls-cert\") pod \"authorino-5ccbc9b875-wdprt\" (UID: \"2c82473b-6248-42e7-8cbe-65645822dd48\") " pod="kuadrant-system/authorino-5ccbc9b875-wdprt" Jul 08 18:50:04.038384 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:04.038212 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwg99\" (UniqueName: \"kubernetes.io/projected/2c82473b-6248-42e7-8cbe-65645822dd48-kube-api-access-mwg99\") pod \"authorino-5ccbc9b875-wdprt\" (UID: \"2c82473b-6248-42e7-8cbe-65645822dd48\") " pod="kuadrant-system/authorino-5ccbc9b875-wdprt" Jul 08 18:50:04.048604 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:04.048576 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-wndmt" Jul 08 18:50:04.056122 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:04.056102 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-wndmt" Jul 08 18:50:04.139483 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:04.139448 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2c82473b-6248-42e7-8cbe-65645822dd48-tls-cert\") pod \"authorino-5ccbc9b875-wdprt\" (UID: \"2c82473b-6248-42e7-8cbe-65645822dd48\") " pod="kuadrant-system/authorino-5ccbc9b875-wdprt" Jul 08 18:50:04.139483 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:04.139493 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwg99\" (UniqueName: \"kubernetes.io/projected/2c82473b-6248-42e7-8cbe-65645822dd48-kube-api-access-mwg99\") pod \"authorino-5ccbc9b875-wdprt\" (UID: \"2c82473b-6248-42e7-8cbe-65645822dd48\") " pod="kuadrant-system/authorino-5ccbc9b875-wdprt" Jul 08 18:50:04.141947 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:04.141928 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2c82473b-6248-42e7-8cbe-65645822dd48-tls-cert\") pod \"authorino-5ccbc9b875-wdprt\" (UID: \"2c82473b-6248-42e7-8cbe-65645822dd48\") " pod="kuadrant-system/authorino-5ccbc9b875-wdprt" Jul 08 18:50:04.149251 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:04.149223 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwg99\" (UniqueName: \"kubernetes.io/projected/2c82473b-6248-42e7-8cbe-65645822dd48-kube-api-access-mwg99\") pod \"authorino-5ccbc9b875-wdprt\" (UID: \"2c82473b-6248-42e7-8cbe-65645822dd48\") " pod="kuadrant-system/authorino-5ccbc9b875-wdprt" Jul 08 18:50:04.221695 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:04.221606 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5ccbc9b875-wdprt" Jul 08 18:50:04.241094 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:04.240991 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cfxlb\" (UniqueName: \"kubernetes.io/projected/5bbf57dc-1a2a-4572-8ca5-00f29c0b847b-kube-api-access-cfxlb\") pod \"5bbf57dc-1a2a-4572-8ca5-00f29c0b847b\" (UID: \"5bbf57dc-1a2a-4572-8ca5-00f29c0b847b\") " Jul 08 18:50:04.243034 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:04.243002 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bbf57dc-1a2a-4572-8ca5-00f29c0b847b-kube-api-access-cfxlb" (OuterVolumeSpecName: "kube-api-access-cfxlb") pod "5bbf57dc-1a2a-4572-8ca5-00f29c0b847b" (UID: "5bbf57dc-1a2a-4572-8ca5-00f29c0b847b"). InnerVolumeSpecName "kube-api-access-cfxlb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:50:04.342913 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:04.342883 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cfxlb\" (UniqueName: \"kubernetes.io/projected/5bbf57dc-1a2a-4572-8ca5-00f29c0b847b-kube-api-access-cfxlb\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:50:04.359258 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:04.359233 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5ccbc9b875-wdprt"] Jul 08 18:50:04.361749 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:50:04.361717 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2c82473b_6248_42e7_8cbe_65645822dd48.slice/crio-f83be676981a07b4d639294ca7b2e106b608b5b7eba3c01c426539bb4cc83531 WatchSource:0}: Error finding container f83be676981a07b4d639294ca7b2e106b608b5b7eba3c01c426539bb4cc83531: Status 404 returned error can't find the container with id f83be676981a07b4d639294ca7b2e106b608b5b7eba3c01c426539bb4cc83531 Jul 08 18:50:04.363149 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:04.363133 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 18:50:05.056468 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:05.056372 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5ccbc9b875-wdprt" event={"ID":"2c82473b-6248-42e7-8cbe-65645822dd48","Type":"ContainerStarted","Data":"a08fb0097f47dda5b0f4c6ddd63dcd9a3bcb253bdceb241680f324af6b59c038"} Jul 08 18:50:05.056468 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:05.056426 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5ccbc9b875-wdprt" event={"ID":"2c82473b-6248-42e7-8cbe-65645822dd48","Type":"ContainerStarted","Data":"f83be676981a07b4d639294ca7b2e106b608b5b7eba3c01c426539bb4cc83531"} Jul 08 18:50:05.056905 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:05.056471 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-wndmt" Jul 08 18:50:05.076334 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:05.076270 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5ccbc9b875-wdprt" podStartSLOduration=1.719350923 podStartE2EDuration="2.076255577s" totalImagesPullingTime="356.904654ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:50:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:50:05.073724989 +0000 UTC m=+648.310023069" watchObservedRunningTime="2026-07-08 18:50:05.076255577 +0000 UTC m=+648.312553667" Jul 08 18:50:05.105666 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:05.105630 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-l2mj6"] Jul 08 18:50:05.106206 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:05.106158 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-l2mj6" podUID="01d0d168-da2a-4064-8d1a-b9c2fa3f2097" containerName="authorino" containerID="cri-o://9439dbdc6bfefa43d69908e248ef32537381637c35a22bf4af292296023fa7e6" gracePeriod=30 Jul 08 18:50:05.111922 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:05.111889 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wndmt"] Jul 08 18:50:05.116053 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:05.116028 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wndmt"] Jul 08 18:50:05.364945 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:05.364918 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-l2mj6" Jul 08 18:50:05.556002 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:05.555967 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wccth\" (UniqueName: \"kubernetes.io/projected/01d0d168-da2a-4064-8d1a-b9c2fa3f2097-kube-api-access-wccth\") pod \"01d0d168-da2a-4064-8d1a-b9c2fa3f2097\" (UID: \"01d0d168-da2a-4064-8d1a-b9c2fa3f2097\") " Jul 08 18:50:05.558030 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:05.558005 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01d0d168-da2a-4064-8d1a-b9c2fa3f2097-kube-api-access-wccth" (OuterVolumeSpecName: "kube-api-access-wccth") pod "01d0d168-da2a-4064-8d1a-b9c2fa3f2097" (UID: "01d0d168-da2a-4064-8d1a-b9c2fa3f2097"). InnerVolumeSpecName "kube-api-access-wccth". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:50:05.657126 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:05.657095 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wccth\" (UniqueName: \"kubernetes.io/projected/01d0d168-da2a-4064-8d1a-b9c2fa3f2097-kube-api-access-wccth\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:50:05.687928 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:05.687900 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5bbf57dc-1a2a-4572-8ca5-00f29c0b847b" path="/var/lib/kubelet/pods/5bbf57dc-1a2a-4572-8ca5-00f29c0b847b/volumes" Jul 08 18:50:06.063358 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:06.063263 2559 generic.go:356] "Generic (PLEG): container finished" podID="01d0d168-da2a-4064-8d1a-b9c2fa3f2097" containerID="9439dbdc6bfefa43d69908e248ef32537381637c35a22bf4af292296023fa7e6" exitCode=0 Jul 08 18:50:06.063358 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:06.063309 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-l2mj6" Jul 08 18:50:06.063358 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:06.063345 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-l2mj6" event={"ID":"01d0d168-da2a-4064-8d1a-b9c2fa3f2097","Type":"ContainerDied","Data":"9439dbdc6bfefa43d69908e248ef32537381637c35a22bf4af292296023fa7e6"} Jul 08 18:50:06.063935 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:06.063381 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-l2mj6" event={"ID":"01d0d168-da2a-4064-8d1a-b9c2fa3f2097","Type":"ContainerDied","Data":"9ea16f04b4998bbaab29415d15fd12adcfdfea5ae92c75c45dc9fa207f6422c8"} Jul 08 18:50:06.063935 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:06.063398 2559 scope.go:117] "RemoveContainer" containerID="9439dbdc6bfefa43d69908e248ef32537381637c35a22bf4af292296023fa7e6" Jul 08 18:50:06.072387 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:06.072370 2559 scope.go:117] "RemoveContainer" containerID="9439dbdc6bfefa43d69908e248ef32537381637c35a22bf4af292296023fa7e6" Jul 08 18:50:06.072659 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:50:06.072638 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9439dbdc6bfefa43d69908e248ef32537381637c35a22bf4af292296023fa7e6\": container with ID starting with 9439dbdc6bfefa43d69908e248ef32537381637c35a22bf4af292296023fa7e6 not found: ID does not exist" containerID="9439dbdc6bfefa43d69908e248ef32537381637c35a22bf4af292296023fa7e6" Jul 08 18:50:06.072703 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:06.072669 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9439dbdc6bfefa43d69908e248ef32537381637c35a22bf4af292296023fa7e6"} err="failed to get container status \"9439dbdc6bfefa43d69908e248ef32537381637c35a22bf4af292296023fa7e6\": rpc error: code = NotFound desc = could not find container \"9439dbdc6bfefa43d69908e248ef32537381637c35a22bf4af292296023fa7e6\": container with ID starting with 9439dbdc6bfefa43d69908e248ef32537381637c35a22bf4af292296023fa7e6 not found: ID does not exist" Jul 08 18:50:06.080740 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:06.080713 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-l2mj6"] Jul 08 18:50:06.082963 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:06.082946 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-l2mj6"] Jul 08 18:50:07.687721 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:50:07.687688 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01d0d168-da2a-4064-8d1a-b9c2fa3f2097" path="/var/lib/kubelet/pods/01d0d168-da2a-4064-8d1a-b9c2fa3f2097/volumes" Jul 08 18:51:29.151365 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:29.151317 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-6fj55"] Jul 08 18:51:29.151897 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:29.151842 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01d0d168-da2a-4064-8d1a-b9c2fa3f2097" containerName="authorino" Jul 08 18:51:29.151897 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:29.151859 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="01d0d168-da2a-4064-8d1a-b9c2fa3f2097" containerName="authorino" Jul 08 18:51:29.151981 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:29.151953 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="01d0d168-da2a-4064-8d1a-b9c2fa3f2097" containerName="authorino" Jul 08 18:51:29.155134 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:29.155117 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6fj55" Jul 08 18:51:29.160419 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:29.160397 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 08 18:51:29.160617 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:29.160604 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-gwjg6\"" Jul 08 18:51:29.162076 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:29.161782 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 08 18:51:29.162076 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:29.161813 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 08 18:51:29.169989 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:29.169969 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-6fj55"] Jul 08 18:51:29.271546 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:29.271493 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70becc73-a4a0-428c-9d76-2138b4e726e2-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6fj55\" (UID: \"70becc73-a4a0-428c-9d76-2138b4e726e2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6fj55" Jul 08 18:51:29.271743 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:29.271575 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70becc73-a4a0-428c-9d76-2138b4e726e2-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6fj55\" (UID: \"70becc73-a4a0-428c-9d76-2138b4e726e2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6fj55" Jul 08 18:51:29.271743 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:29.271681 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70becc73-a4a0-428c-9d76-2138b4e726e2-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6fj55\" (UID: \"70becc73-a4a0-428c-9d76-2138b4e726e2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6fj55" Jul 08 18:51:29.271743 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:29.271705 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70becc73-a4a0-428c-9d76-2138b4e726e2-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6fj55\" (UID: \"70becc73-a4a0-428c-9d76-2138b4e726e2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6fj55" Jul 08 18:51:29.271743 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:29.271731 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6qtt\" (UniqueName: \"kubernetes.io/projected/70becc73-a4a0-428c-9d76-2138b4e726e2-kube-api-access-r6qtt\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6fj55\" (UID: \"70becc73-a4a0-428c-9d76-2138b4e726e2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6fj55" Jul 08 18:51:29.271942 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:29.271846 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70becc73-a4a0-428c-9d76-2138b4e726e2-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6fj55\" (UID: \"70becc73-a4a0-428c-9d76-2138b4e726e2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6fj55" Jul 08 18:51:29.373410 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:29.373376 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70becc73-a4a0-428c-9d76-2138b4e726e2-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6fj55\" (UID: \"70becc73-a4a0-428c-9d76-2138b4e726e2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6fj55" Jul 08 18:51:29.373410 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:29.373412 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70becc73-a4a0-428c-9d76-2138b4e726e2-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6fj55\" (UID: \"70becc73-a4a0-428c-9d76-2138b4e726e2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6fj55" Jul 08 18:51:29.373667 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:29.373429 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6qtt\" (UniqueName: \"kubernetes.io/projected/70becc73-a4a0-428c-9d76-2138b4e726e2-kube-api-access-r6qtt\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6fj55\" (UID: \"70becc73-a4a0-428c-9d76-2138b4e726e2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6fj55" Jul 08 18:51:29.373667 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:29.373476 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70becc73-a4a0-428c-9d76-2138b4e726e2-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6fj55\" (UID: \"70becc73-a4a0-428c-9d76-2138b4e726e2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6fj55" Jul 08 18:51:29.373667 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:29.373524 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70becc73-a4a0-428c-9d76-2138b4e726e2-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6fj55\" (UID: \"70becc73-a4a0-428c-9d76-2138b4e726e2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6fj55" Jul 08 18:51:29.373667 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:29.373573 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70becc73-a4a0-428c-9d76-2138b4e726e2-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6fj55\" (UID: \"70becc73-a4a0-428c-9d76-2138b4e726e2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6fj55" Jul 08 18:51:29.373921 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:29.373878 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70becc73-a4a0-428c-9d76-2138b4e726e2-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6fj55\" (UID: \"70becc73-a4a0-428c-9d76-2138b4e726e2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6fj55" Jul 08 18:51:29.373921 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:29.373897 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70becc73-a4a0-428c-9d76-2138b4e726e2-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6fj55\" (UID: \"70becc73-a4a0-428c-9d76-2138b4e726e2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6fj55" Jul 08 18:51:29.374015 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:29.373955 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70becc73-a4a0-428c-9d76-2138b4e726e2-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6fj55\" (UID: \"70becc73-a4a0-428c-9d76-2138b4e726e2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6fj55" Jul 08 18:51:29.375869 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:29.375847 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70becc73-a4a0-428c-9d76-2138b4e726e2-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6fj55\" (UID: \"70becc73-a4a0-428c-9d76-2138b4e726e2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6fj55" Jul 08 18:51:29.376058 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:29.376039 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70becc73-a4a0-428c-9d76-2138b4e726e2-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6fj55\" (UID: \"70becc73-a4a0-428c-9d76-2138b4e726e2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6fj55" Jul 08 18:51:29.382613 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:29.382586 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6qtt\" (UniqueName: \"kubernetes.io/projected/70becc73-a4a0-428c-9d76-2138b4e726e2-kube-api-access-r6qtt\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6fj55\" (UID: \"70becc73-a4a0-428c-9d76-2138b4e726e2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6fj55" Jul 08 18:51:29.465710 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:29.465616 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6fj55" Jul 08 18:51:29.614723 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:29.614516 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-6fj55"] Jul 08 18:51:29.617652 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:51:29.617627 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70becc73_a4a0_428c_9d76_2138b4e726e2.slice/crio-d6d9b9938eb7abce7c51c2ef153526abfde59912af921517f0ee85df81324f44 WatchSource:0}: Error finding container d6d9b9938eb7abce7c51c2ef153526abfde59912af921517f0ee85df81324f44: Status 404 returned error can't find the container with id d6d9b9938eb7abce7c51c2ef153526abfde59912af921517f0ee85df81324f44 Jul 08 18:51:30.548928 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:30.548885 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6fj55" event={"ID":"70becc73-a4a0-428c-9d76-2138b4e726e2","Type":"ContainerStarted","Data":"d6d9b9938eb7abce7c51c2ef153526abfde59912af921517f0ee85df81324f44"} Jul 08 18:51:31.557569 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:31.557532 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6fj55" event={"ID":"70becc73-a4a0-428c-9d76-2138b4e726e2","Type":"ContainerStarted","Data":"3edd28a9a11d8bdf5df61e03f90cdfd8492fb0c74ed8eb75a2e8cd1ec7aaa376"} Jul 08 18:51:31.583891 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:31.583831 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6fj55" podStartSLOduration=1.420959369 podStartE2EDuration="2.583811032s" totalImagesPullingTime="1.162851663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:51:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:51:31.58300519 +0000 UTC m=+734.819303281" watchObservedRunningTime="2026-07-08 18:51:31.583811032 +0000 UTC m=+734.820109126" Jul 08 18:51:39.465867 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:39.465829 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6fj55" Jul 08 18:51:39.466265 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:39.465906 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6fj55" Jul 08 18:51:39.478526 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:39.478490 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6fj55" Jul 08 18:51:39.616943 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:39.616911 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6fj55" Jul 08 18:51:53.773984 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:53.773947 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8"] Jul 08 18:51:53.778154 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:53.778117 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8" Jul 08 18:51:53.781139 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:53.781115 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 08 18:51:53.789367 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:53.789342 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8"] Jul 08 18:51:53.819527 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:53.819491 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/600f4dca-4fb4-40b1-b2af-111478b50316-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8\" (UID: \"600f4dca-4fb4-40b1-b2af-111478b50316\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8" Jul 08 18:51:53.819723 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:53.819556 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9gbb\" (UniqueName: \"kubernetes.io/projected/600f4dca-4fb4-40b1-b2af-111478b50316-kube-api-access-z9gbb\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8\" (UID: \"600f4dca-4fb4-40b1-b2af-111478b50316\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8" Jul 08 18:51:53.819723 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:53.819609 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/600f4dca-4fb4-40b1-b2af-111478b50316-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8\" (UID: \"600f4dca-4fb4-40b1-b2af-111478b50316\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8" Jul 08 18:51:53.819723 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:53.819645 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/600f4dca-4fb4-40b1-b2af-111478b50316-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8\" (UID: \"600f4dca-4fb4-40b1-b2af-111478b50316\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8" Jul 08 18:51:53.819723 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:53.819695 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/600f4dca-4fb4-40b1-b2af-111478b50316-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8\" (UID: \"600f4dca-4fb4-40b1-b2af-111478b50316\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8" Jul 08 18:51:53.819928 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:53.819746 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/600f4dca-4fb4-40b1-b2af-111478b50316-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8\" (UID: \"600f4dca-4fb4-40b1-b2af-111478b50316\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8" Jul 08 18:51:53.921233 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:53.921189 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/600f4dca-4fb4-40b1-b2af-111478b50316-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8\" (UID: \"600f4dca-4fb4-40b1-b2af-111478b50316\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8" Jul 08 18:51:53.921438 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:53.921249 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/600f4dca-4fb4-40b1-b2af-111478b50316-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8\" (UID: \"600f4dca-4fb4-40b1-b2af-111478b50316\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8" Jul 08 18:51:53.921438 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:53.921281 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/600f4dca-4fb4-40b1-b2af-111478b50316-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8\" (UID: \"600f4dca-4fb4-40b1-b2af-111478b50316\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8" Jul 08 18:51:53.921438 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:53.921333 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/600f4dca-4fb4-40b1-b2af-111478b50316-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8\" (UID: \"600f4dca-4fb4-40b1-b2af-111478b50316\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8" Jul 08 18:51:53.921438 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:53.921399 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/600f4dca-4fb4-40b1-b2af-111478b50316-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8\" (UID: \"600f4dca-4fb4-40b1-b2af-111478b50316\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8" Jul 08 18:51:53.921671 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:53.921442 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9gbb\" (UniqueName: \"kubernetes.io/projected/600f4dca-4fb4-40b1-b2af-111478b50316-kube-api-access-z9gbb\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8\" (UID: \"600f4dca-4fb4-40b1-b2af-111478b50316\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8" Jul 08 18:51:53.921671 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:53.921630 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/600f4dca-4fb4-40b1-b2af-111478b50316-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8\" (UID: \"600f4dca-4fb4-40b1-b2af-111478b50316\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8" Jul 08 18:51:53.921798 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:53.921637 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/600f4dca-4fb4-40b1-b2af-111478b50316-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8\" (UID: \"600f4dca-4fb4-40b1-b2af-111478b50316\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8" Jul 08 18:51:53.921798 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:53.921715 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/600f4dca-4fb4-40b1-b2af-111478b50316-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8\" (UID: \"600f4dca-4fb4-40b1-b2af-111478b50316\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8" Jul 08 18:51:53.923750 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:53.923726 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/600f4dca-4fb4-40b1-b2af-111478b50316-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8\" (UID: \"600f4dca-4fb4-40b1-b2af-111478b50316\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8" Jul 08 18:51:53.924243 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:53.924219 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/600f4dca-4fb4-40b1-b2af-111478b50316-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8\" (UID: \"600f4dca-4fb4-40b1-b2af-111478b50316\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8" Jul 08 18:51:53.929634 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:53.929604 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9gbb\" (UniqueName: \"kubernetes.io/projected/600f4dca-4fb4-40b1-b2af-111478b50316-kube-api-access-z9gbb\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8\" (UID: \"600f4dca-4fb4-40b1-b2af-111478b50316\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8" Jul 08 18:51:54.090033 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:54.089946 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8" Jul 08 18:51:54.227304 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:54.227273 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8"] Jul 08 18:51:54.228714 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:51:54.228684 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod600f4dca_4fb4_40b1_b2af_111478b50316.slice/crio-5e7c93bda6fc453c63af924f837675c3fb1ff18acc47d81bca0172b130975a45 WatchSource:0}: Error finding container 5e7c93bda6fc453c63af924f837675c3fb1ff18acc47d81bca0172b130975a45: Status 404 returned error can't find the container with id 5e7c93bda6fc453c63af924f837675c3fb1ff18acc47d81bca0172b130975a45 Jul 08 18:51:54.690411 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:54.690308 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8" event={"ID":"600f4dca-4fb4-40b1-b2af-111478b50316","Type":"ContainerStarted","Data":"9ab12b93416af6a65e25dc03d03c80b8191e26e077bbf2dd2c5aeeb876ecb169"} Jul 08 18:51:54.690411 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:54.690373 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8" event={"ID":"600f4dca-4fb4-40b1-b2af-111478b50316","Type":"ContainerStarted","Data":"5e7c93bda6fc453c63af924f837675c3fb1ff18acc47d81bca0172b130975a45"} Jul 08 18:51:54.711085 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:51:54.711025 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8" podStartSLOduration=1.518743116 podStartE2EDuration="1.711009815s" totalImagesPullingTime="192.266699ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:51:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:51:54.708965083 +0000 UTC m=+757.945263175" watchObservedRunningTime="2026-07-08 18:51:54.711009815 +0000 UTC m=+757.947307904" Jul 08 18:52:04.090297 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:52:04.090251 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8" Jul 08 18:52:04.090870 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:52:04.090316 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8" Jul 08 18:52:04.102946 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:52:04.102916 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8" Jul 08 18:52:04.759116 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:52:04.759089 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8" Jul 08 18:53:21.984479 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:53:21.984441 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5578968cc4-srdrd"] Jul 08 18:53:21.988264 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:53:21.988245 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5578968cc4-srdrd" Jul 08 18:53:21.998441 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:53:21.998408 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5578968cc4-srdrd"] Jul 08 18:53:22.076753 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:53:22.076717 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8427523c-36f1-464f-888a-a8d3fade1d2a-tls-cert\") pod \"authorino-5578968cc4-srdrd\" (UID: \"8427523c-36f1-464f-888a-a8d3fade1d2a\") " pod="kuadrant-system/authorino-5578968cc4-srdrd" Jul 08 18:53:22.076753 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:53:22.076756 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxq7r\" (UniqueName: \"kubernetes.io/projected/8427523c-36f1-464f-888a-a8d3fade1d2a-kube-api-access-wxq7r\") pod \"authorino-5578968cc4-srdrd\" (UID: \"8427523c-36f1-464f-888a-a8d3fade1d2a\") " pod="kuadrant-system/authorino-5578968cc4-srdrd" Jul 08 18:53:22.177356 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:53:22.177302 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8427523c-36f1-464f-888a-a8d3fade1d2a-tls-cert\") pod \"authorino-5578968cc4-srdrd\" (UID: \"8427523c-36f1-464f-888a-a8d3fade1d2a\") " pod="kuadrant-system/authorino-5578968cc4-srdrd" Jul 08 18:53:22.177356 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:53:22.177363 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxq7r\" (UniqueName: \"kubernetes.io/projected/8427523c-36f1-464f-888a-a8d3fade1d2a-kube-api-access-wxq7r\") pod \"authorino-5578968cc4-srdrd\" (UID: \"8427523c-36f1-464f-888a-a8d3fade1d2a\") " pod="kuadrant-system/authorino-5578968cc4-srdrd" Jul 08 18:53:22.179916 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:53:22.179889 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8427523c-36f1-464f-888a-a8d3fade1d2a-tls-cert\") pod \"authorino-5578968cc4-srdrd\" (UID: \"8427523c-36f1-464f-888a-a8d3fade1d2a\") " pod="kuadrant-system/authorino-5578968cc4-srdrd" Jul 08 18:53:22.186357 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:53:22.186335 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxq7r\" (UniqueName: \"kubernetes.io/projected/8427523c-36f1-464f-888a-a8d3fade1d2a-kube-api-access-wxq7r\") pod \"authorino-5578968cc4-srdrd\" (UID: \"8427523c-36f1-464f-888a-a8d3fade1d2a\") " pod="kuadrant-system/authorino-5578968cc4-srdrd" Jul 08 18:53:22.298533 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:53:22.298431 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5578968cc4-srdrd" Jul 08 18:53:22.452844 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:53:22.452819 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5578968cc4-srdrd"] Jul 08 18:53:22.455046 ip-10-0-143-154 kubenswrapper[2559]: W0708 18:53:22.455021 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8427523c_36f1_464f_888a_a8d3fade1d2a.slice/crio-af369dae4120166cb1f3cb501fe0e065b0adc28ad1f82c3f5a1ed09759ee2881 WatchSource:0}: Error finding container af369dae4120166cb1f3cb501fe0e065b0adc28ad1f82c3f5a1ed09759ee2881: Status 404 returned error can't find the container with id af369dae4120166cb1f3cb501fe0e065b0adc28ad1f82c3f5a1ed09759ee2881 Jul 08 18:53:23.227815 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:53:23.226892 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5578968cc4-srdrd" event={"ID":"8427523c-36f1-464f-888a-a8d3fade1d2a","Type":"ContainerStarted","Data":"9b33dace16a47940998872a07e6be1d508bc543e2323fde22e4c08537a19ff2c"} Jul 08 18:53:23.227815 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:53:23.226943 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5578968cc4-srdrd" event={"ID":"8427523c-36f1-464f-888a-a8d3fade1d2a","Type":"ContainerStarted","Data":"af369dae4120166cb1f3cb501fe0e065b0adc28ad1f82c3f5a1ed09759ee2881"} Jul 08 18:53:23.248217 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:53:23.248161 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5578968cc4-srdrd" podStartSLOduration=1.750842468 podStartE2EDuration="2.248145795s" totalImagesPullingTime="497.303327ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:53:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:53:23.246351374 +0000 UTC m=+846.482649457" watchObservedRunningTime="2026-07-08 18:53:23.248145795 +0000 UTC m=+846.484443885" Jul 08 18:53:23.283737 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:53:23.283698 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5ccbc9b875-wdprt"] Jul 08 18:53:23.283942 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:53:23.283901 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5ccbc9b875-wdprt" podUID="2c82473b-6248-42e7-8cbe-65645822dd48" containerName="authorino" containerID="cri-o://a08fb0097f47dda5b0f4c6ddd63dcd9a3bcb253bdceb241680f324af6b59c038" gracePeriod=30 Jul 08 18:53:23.548539 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:53:23.548509 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5ccbc9b875-wdprt" Jul 08 18:53:23.592008 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:53:23.591926 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2c82473b-6248-42e7-8cbe-65645822dd48-tls-cert\") pod \"2c82473b-6248-42e7-8cbe-65645822dd48\" (UID: \"2c82473b-6248-42e7-8cbe-65645822dd48\") " Jul 08 18:53:23.592203 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:53:23.592024 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mwg99\" (UniqueName: \"kubernetes.io/projected/2c82473b-6248-42e7-8cbe-65645822dd48-kube-api-access-mwg99\") pod \"2c82473b-6248-42e7-8cbe-65645822dd48\" (UID: \"2c82473b-6248-42e7-8cbe-65645822dd48\") " Jul 08 18:53:23.594084 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:53:23.594045 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c82473b-6248-42e7-8cbe-65645822dd48-kube-api-access-mwg99" (OuterVolumeSpecName: "kube-api-access-mwg99") pod "2c82473b-6248-42e7-8cbe-65645822dd48" (UID: "2c82473b-6248-42e7-8cbe-65645822dd48"). InnerVolumeSpecName "kube-api-access-mwg99". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:53:23.604034 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:53:23.603997 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c82473b-6248-42e7-8cbe-65645822dd48-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "2c82473b-6248-42e7-8cbe-65645822dd48" (UID: "2c82473b-6248-42e7-8cbe-65645822dd48"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:53:23.692860 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:53:23.692831 2559 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2c82473b-6248-42e7-8cbe-65645822dd48-tls-cert\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:53:23.692860 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:53:23.692859 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mwg99\" (UniqueName: \"kubernetes.io/projected/2c82473b-6248-42e7-8cbe-65645822dd48-kube-api-access-mwg99\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:53:24.240929 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:53:24.240888 2559 generic.go:356] "Generic (PLEG): container finished" podID="2c82473b-6248-42e7-8cbe-65645822dd48" containerID="a08fb0097f47dda5b0f4c6ddd63dcd9a3bcb253bdceb241680f324af6b59c038" exitCode=0 Jul 08 18:53:24.241377 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:53:24.240966 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5ccbc9b875-wdprt" Jul 08 18:53:24.241377 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:53:24.240967 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5ccbc9b875-wdprt" event={"ID":"2c82473b-6248-42e7-8cbe-65645822dd48","Type":"ContainerDied","Data":"a08fb0097f47dda5b0f4c6ddd63dcd9a3bcb253bdceb241680f324af6b59c038"} Jul 08 18:53:24.241377 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:53:24.241017 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5ccbc9b875-wdprt" event={"ID":"2c82473b-6248-42e7-8cbe-65645822dd48","Type":"ContainerDied","Data":"f83be676981a07b4d639294ca7b2e106b608b5b7eba3c01c426539bb4cc83531"} Jul 08 18:53:24.241377 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:53:24.241039 2559 scope.go:117] "RemoveContainer" containerID="a08fb0097f47dda5b0f4c6ddd63dcd9a3bcb253bdceb241680f324af6b59c038" Jul 08 18:53:24.252656 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:53:24.252637 2559 scope.go:117] "RemoveContainer" containerID="a08fb0097f47dda5b0f4c6ddd63dcd9a3bcb253bdceb241680f324af6b59c038" Jul 08 18:53:24.252940 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:53:24.252916 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a08fb0097f47dda5b0f4c6ddd63dcd9a3bcb253bdceb241680f324af6b59c038\": container with ID starting with a08fb0097f47dda5b0f4c6ddd63dcd9a3bcb253bdceb241680f324af6b59c038 not found: ID does not exist" containerID="a08fb0097f47dda5b0f4c6ddd63dcd9a3bcb253bdceb241680f324af6b59c038" Jul 08 18:53:24.253035 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:53:24.252947 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a08fb0097f47dda5b0f4c6ddd63dcd9a3bcb253bdceb241680f324af6b59c038"} err="failed to get container status \"a08fb0097f47dda5b0f4c6ddd63dcd9a3bcb253bdceb241680f324af6b59c038\": rpc error: code = NotFound desc = could not find container \"a08fb0097f47dda5b0f4c6ddd63dcd9a3bcb253bdceb241680f324af6b59c038\": container with ID starting with a08fb0097f47dda5b0f4c6ddd63dcd9a3bcb253bdceb241680f324af6b59c038 not found: ID does not exist" Jul 08 18:53:24.257729 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:53:24.257703 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5ccbc9b875-wdprt"] Jul 08 18:53:24.261889 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:53:24.261867 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5ccbc9b875-wdprt"] Jul 08 18:53:25.688564 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:53:25.688526 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c82473b-6248-42e7-8cbe-65645822dd48" path="/var/lib/kubelet/pods/2c82473b-6248-42e7-8cbe-65645822dd48/volumes" Jul 08 18:54:17.255044 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:54:17.254973 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zzt4h_d94a3566-fcf3-47a1-b176-a33b71c3826d/console-operator/1.log" Jul 08 18:54:17.261821 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:54:17.261798 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rj4xk_56014c3a-384a-4775-96e5-290968b196b1/ovn-acl-logging/0.log" Jul 08 18:54:17.712817 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:54:17.712789 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zzt4h_d94a3566-fcf3-47a1-b176-a33b71c3826d/console-operator/1.log" Jul 08 18:54:17.719220 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:54:17.719201 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rj4xk_56014c3a-384a-4775-96e5-290968b196b1/ovn-acl-logging/0.log" Jul 08 18:59:17.299186 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:59:17.299103 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zzt4h_d94a3566-fcf3-47a1-b176-a33b71c3826d/console-operator/1.log" Jul 08 18:59:17.306369 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:59:17.306345 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rj4xk_56014c3a-384a-4775-96e5-290968b196b1/ovn-acl-logging/0.log" Jul 08 18:59:17.765744 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:59:17.765706 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zzt4h_d94a3566-fcf3-47a1-b176-a33b71c3826d/console-operator/1.log" Jul 08 18:59:17.772861 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:59:17.772835 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rj4xk_56014c3a-384a-4775-96e5-290968b196b1/ovn-acl-logging/0.log" Jul 08 18:59:28.934298 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:59:28.934260 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wt499"] Jul 08 18:59:28.934928 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:59:28.934479 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wt499" podUID="219156f1-1ecb-43ff-b3df-042f5ba9ca77" containerName="manager" containerID="cri-o://7eda3f439b9bb45b515c768ad995b329ab785939182a47e9e5449e25525b0b19" gracePeriod=10 Jul 08 18:59:31.585312 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:59:31.585281 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wt499" Jul 08 18:59:31.605501 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:59:31.605471 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lpf2s\" (UniqueName: \"kubernetes.io/projected/219156f1-1ecb-43ff-b3df-042f5ba9ca77-kube-api-access-lpf2s\") pod \"219156f1-1ecb-43ff-b3df-042f5ba9ca77\" (UID: \"219156f1-1ecb-43ff-b3df-042f5ba9ca77\") " Jul 08 18:59:31.605632 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:59:31.605541 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/219156f1-1ecb-43ff-b3df-042f5ba9ca77-extensions-socket-volume\") pod \"219156f1-1ecb-43ff-b3df-042f5ba9ca77\" (UID: \"219156f1-1ecb-43ff-b3df-042f5ba9ca77\") " Jul 08 18:59:31.605992 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:59:31.605960 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/219156f1-1ecb-43ff-b3df-042f5ba9ca77-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "219156f1-1ecb-43ff-b3df-042f5ba9ca77" (UID: "219156f1-1ecb-43ff-b3df-042f5ba9ca77"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:59:31.607583 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:59:31.607555 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/219156f1-1ecb-43ff-b3df-042f5ba9ca77-kube-api-access-lpf2s" (OuterVolumeSpecName: "kube-api-access-lpf2s") pod "219156f1-1ecb-43ff-b3df-042f5ba9ca77" (UID: "219156f1-1ecb-43ff-b3df-042f5ba9ca77"). InnerVolumeSpecName "kube-api-access-lpf2s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:59:31.706605 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:59:31.706577 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lpf2s\" (UniqueName: \"kubernetes.io/projected/219156f1-1ecb-43ff-b3df-042f5ba9ca77-kube-api-access-lpf2s\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:59:31.706605 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:59:31.706601 2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/219156f1-1ecb-43ff-b3df-042f5ba9ca77-extensions-socket-volume\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 18:59:32.446577 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:59:32.446537 2559 generic.go:356] "Generic (PLEG): container finished" podID="219156f1-1ecb-43ff-b3df-042f5ba9ca77" containerID="7eda3f439b9bb45b515c768ad995b329ab785939182a47e9e5449e25525b0b19" exitCode=0 Jul 08 18:59:32.446762 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:59:32.446598 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wt499" Jul 08 18:59:32.446762 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:59:32.446621 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wt499" event={"ID":"219156f1-1ecb-43ff-b3df-042f5ba9ca77","Type":"ContainerDied","Data":"7eda3f439b9bb45b515c768ad995b329ab785939182a47e9e5449e25525b0b19"} Jul 08 18:59:32.446762 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:59:32.446698 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wt499" event={"ID":"219156f1-1ecb-43ff-b3df-042f5ba9ca77","Type":"ContainerDied","Data":"4b94e5e76094655877a99a4c833891f5e880380ed7d3dcf0b0998fc075464274"} Jul 08 18:59:32.446762 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:59:32.446721 2559 scope.go:117] "RemoveContainer" containerID="7eda3f439b9bb45b515c768ad995b329ab785939182a47e9e5449e25525b0b19" Jul 08 18:59:32.456834 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:59:32.456613 2559 scope.go:117] "RemoveContainer" containerID="7eda3f439b9bb45b515c768ad995b329ab785939182a47e9e5449e25525b0b19" Jul 08 18:59:32.456895 ip-10-0-143-154 kubenswrapper[2559]: E0708 18:59:32.456867 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7eda3f439b9bb45b515c768ad995b329ab785939182a47e9e5449e25525b0b19\": container with ID starting with 7eda3f439b9bb45b515c768ad995b329ab785939182a47e9e5449e25525b0b19 not found: ID does not exist" containerID="7eda3f439b9bb45b515c768ad995b329ab785939182a47e9e5449e25525b0b19" Jul 08 18:59:32.456934 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:59:32.456891 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7eda3f439b9bb45b515c768ad995b329ab785939182a47e9e5449e25525b0b19"} err="failed to get container status \"7eda3f439b9bb45b515c768ad995b329ab785939182a47e9e5449e25525b0b19\": rpc error: code = NotFound desc = could not find container \"7eda3f439b9bb45b515c768ad995b329ab785939182a47e9e5449e25525b0b19\": container with ID starting with 7eda3f439b9bb45b515c768ad995b329ab785939182a47e9e5449e25525b0b19 not found: ID does not exist" Jul 08 18:59:32.467189 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:59:32.467167 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wt499"] Jul 08 18:59:32.471256 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:59:32.471234 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wt499"] Jul 08 18:59:33.688436 ip-10-0-143-154 kubenswrapper[2559]: I0708 18:59:33.688402 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="219156f1-1ecb-43ff-b3df-042f5ba9ca77" path="/var/lib/kubelet/pods/219156f1-1ecb-43ff-b3df-042f5ba9ca77/volumes" Jul 08 19:00:00.150107 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:00:00.150062 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725620-xcnwt"] Jul 08 19:00:00.152556 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:00:00.150509 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c82473b-6248-42e7-8cbe-65645822dd48" containerName="authorino" Jul 08 19:00:00.152556 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:00:00.150522 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c82473b-6248-42e7-8cbe-65645822dd48" containerName="authorino" Jul 08 19:00:00.152556 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:00:00.150533 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="219156f1-1ecb-43ff-b3df-042f5ba9ca77" containerName="manager" Jul 08 19:00:00.152556 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:00:00.150538 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="219156f1-1ecb-43ff-b3df-042f5ba9ca77" containerName="manager" Jul 08 19:00:00.152556 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:00:00.150599 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="219156f1-1ecb-43ff-b3df-042f5ba9ca77" containerName="manager" Jul 08 19:00:00.152556 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:00:00.150608 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c82473b-6248-42e7-8cbe-65645822dd48" containerName="authorino" Jul 08 19:00:00.153650 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:00:00.153634 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725620-xcnwt" Jul 08 19:00:00.156608 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:00:00.156589 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-kxbw6\"" Jul 08 19:00:00.161944 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:00:00.161918 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725620-xcnwt"] Jul 08 19:00:00.269128 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:00:00.269084 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pthrr\" (UniqueName: \"kubernetes.io/projected/4e94da38-4c98-445e-92d0-c3bf1806a938-kube-api-access-pthrr\") pod \"maas-api-key-cleanup-29725620-xcnwt\" (UID: \"4e94da38-4c98-445e-92d0-c3bf1806a938\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725620-xcnwt" Jul 08 19:00:00.370634 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:00:00.370598 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pthrr\" (UniqueName: \"kubernetes.io/projected/4e94da38-4c98-445e-92d0-c3bf1806a938-kube-api-access-pthrr\") pod \"maas-api-key-cleanup-29725620-xcnwt\" (UID: \"4e94da38-4c98-445e-92d0-c3bf1806a938\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725620-xcnwt" Jul 08 19:00:00.379513 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:00:00.379467 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pthrr\" (UniqueName: \"kubernetes.io/projected/4e94da38-4c98-445e-92d0-c3bf1806a938-kube-api-access-pthrr\") pod \"maas-api-key-cleanup-29725620-xcnwt\" (UID: \"4e94da38-4c98-445e-92d0-c3bf1806a938\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725620-xcnwt" Jul 08 19:00:00.466114 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:00:00.466014 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725620-xcnwt" Jul 08 19:00:00.599915 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:00:00.599884 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725620-xcnwt"] Jul 08 19:00:00.601870 ip-10-0-143-154 kubenswrapper[2559]: W0708 19:00:00.601840 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e94da38_4c98_445e_92d0_c3bf1806a938.slice/crio-256244d21390a6cfe1a7c3624bee5e85e2d0d032babbb3545d84b100f0c42237 WatchSource:0}: Error finding container 256244d21390a6cfe1a7c3624bee5e85e2d0d032babbb3545d84b100f0c42237: Status 404 returned error can't find the container with id 256244d21390a6cfe1a7c3624bee5e85e2d0d032babbb3545d84b100f0c42237 Jul 08 19:00:00.603921 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:00:00.603902 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 19:00:01.612891 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:00:01.612850 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725620-xcnwt" event={"ID":"4e94da38-4c98-445e-92d0-c3bf1806a938","Type":"ContainerStarted","Data":"256244d21390a6cfe1a7c3624bee5e85e2d0d032babbb3545d84b100f0c42237"} Jul 08 19:00:06.646267 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:00:06.646230 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725620-xcnwt" event={"ID":"4e94da38-4c98-445e-92d0-c3bf1806a938","Type":"ContainerStarted","Data":"8f0f0b1783b775b35378d58b7c85fc83be6b7d2c7c2ccc2fd9366446e3cab136"} Jul 08 19:00:06.666751 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:00:06.666704 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725620-xcnwt" podStartSLOduration=1.818289391 podStartE2EDuration="6.66668944s" totalImagesPullingTime="4.848400049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:00:06.664962963 +0000 UTC m=+1249.901261054" watchObservedRunningTime="2026-07-08 19:00:06.66668944 +0000 UTC m=+1249.902987530" Jul 08 19:00:30.136473 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:00:30.136431 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8h5ns"] Jul 08 19:00:30.144107 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:00:30.144088 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8h5ns" Jul 08 19:00:30.146880 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:00:30.146858 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-qw22r\"" Jul 08 19:00:30.153886 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:00:30.153860 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8h5ns"] Jul 08 19:00:30.260640 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:00:30.260605 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kknqw\" (UniqueName: \"kubernetes.io/projected/05d9a127-a2e3-4f54-a423-dc30f5144ee7-kube-api-access-kknqw\") pod \"kuadrant-operator-controller-manager-86878c769f-8h5ns\" (UID: \"05d9a127-a2e3-4f54-a423-dc30f5144ee7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8h5ns" Jul 08 19:00:30.260640 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:00:30.260643 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/05d9a127-a2e3-4f54-a423-dc30f5144ee7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8h5ns\" (UID: \"05d9a127-a2e3-4f54-a423-dc30f5144ee7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8h5ns" Jul 08 19:00:30.362018 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:00:30.361980 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kknqw\" (UniqueName: \"kubernetes.io/projected/05d9a127-a2e3-4f54-a423-dc30f5144ee7-kube-api-access-kknqw\") pod \"kuadrant-operator-controller-manager-86878c769f-8h5ns\" (UID: \"05d9a127-a2e3-4f54-a423-dc30f5144ee7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8h5ns" Jul 08 19:00:30.362194 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:00:30.362024 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/05d9a127-a2e3-4f54-a423-dc30f5144ee7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8h5ns\" (UID: \"05d9a127-a2e3-4f54-a423-dc30f5144ee7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8h5ns" Jul 08 19:00:30.362472 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:00:30.362453 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/05d9a127-a2e3-4f54-a423-dc30f5144ee7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8h5ns\" (UID: \"05d9a127-a2e3-4f54-a423-dc30f5144ee7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8h5ns" Jul 08 19:00:30.371990 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:00:30.371961 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kknqw\" (UniqueName: \"kubernetes.io/projected/05d9a127-a2e3-4f54-a423-dc30f5144ee7-kube-api-access-kknqw\") pod \"kuadrant-operator-controller-manager-86878c769f-8h5ns\" (UID: \"05d9a127-a2e3-4f54-a423-dc30f5144ee7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8h5ns" Jul 08 19:00:30.455719 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:00:30.455630 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8h5ns" Jul 08 19:00:30.619381 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:00:30.619356 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8h5ns"] Jul 08 19:00:30.621091 ip-10-0-143-154 kubenswrapper[2559]: W0708 19:00:30.621065 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05d9a127_a2e3_4f54_a423_dc30f5144ee7.slice/crio-80ac614be7c7d827b593c03efd1781c8e681b086a5817eaf646b4fa71e524904 WatchSource:0}: Error finding container 80ac614be7c7d827b593c03efd1781c8e681b086a5817eaf646b4fa71e524904: Status 404 returned error can't find the container with id 80ac614be7c7d827b593c03efd1781c8e681b086a5817eaf646b4fa71e524904 Jul 08 19:00:30.782588 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:00:30.782498 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8h5ns" event={"ID":"05d9a127-a2e3-4f54-a423-dc30f5144ee7","Type":"ContainerStarted","Data":"9416285273aaab42049dc24d402b5901d05d4190163a992529f432ef2e3a20c0"} Jul 08 19:00:30.782588 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:00:30.782546 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8h5ns" event={"ID":"05d9a127-a2e3-4f54-a423-dc30f5144ee7","Type":"ContainerStarted","Data":"80ac614be7c7d827b593c03efd1781c8e681b086a5817eaf646b4fa71e524904"} Jul 08 19:00:30.782588 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:00:30.782590 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8h5ns" Jul 08 19:00:30.805317 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:00:30.805263 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8h5ns" podStartSLOduration=0.805247683 podStartE2EDuration="805.247683ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:00:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:00:30.803705361 +0000 UTC m=+1274.040003451" watchObservedRunningTime="2026-07-08 19:00:30.805247683 +0000 UTC m=+1274.041545773" Jul 08 19:00:41.791121 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:00:41.791011 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8h5ns" Jul 08 19:01:00.011705 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:01:00.011668 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725620-xcnwt"] Jul 08 19:01:00.012235 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:01:00.011884 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725620-xcnwt" podUID="4e94da38-4c98-445e-92d0-c3bf1806a938" containerName="cleanup" containerID="cri-o://8f0f0b1783b775b35378d58b7c85fc83be6b7d2c7c2ccc2fd9366446e3cab136" gracePeriod=30 Jul 08 19:01:30.168736 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:01:30.168698 2559 generic.go:356] "Generic (PLEG): container finished" podID="4e94da38-4c98-445e-92d0-c3bf1806a938" containerID="8f0f0b1783b775b35378d58b7c85fc83be6b7d2c7c2ccc2fd9366446e3cab136" exitCode=137 Jul 08 19:01:30.169166 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:01:30.168785 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725620-xcnwt" event={"ID":"4e94da38-4c98-445e-92d0-c3bf1806a938","Type":"ContainerDied","Data":"8f0f0b1783b775b35378d58b7c85fc83be6b7d2c7c2ccc2fd9366446e3cab136"} Jul 08 19:01:30.203080 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:01:30.203057 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725620-xcnwt" Jul 08 19:01:30.343854 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:01:30.343718 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pthrr\" (UniqueName: \"kubernetes.io/projected/4e94da38-4c98-445e-92d0-c3bf1806a938-kube-api-access-pthrr\") pod \"4e94da38-4c98-445e-92d0-c3bf1806a938\" (UID: \"4e94da38-4c98-445e-92d0-c3bf1806a938\") " Jul 08 19:01:30.345962 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:01:30.345924 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e94da38-4c98-445e-92d0-c3bf1806a938-kube-api-access-pthrr" (OuterVolumeSpecName: "kube-api-access-pthrr") pod "4e94da38-4c98-445e-92d0-c3bf1806a938" (UID: "4e94da38-4c98-445e-92d0-c3bf1806a938"). InnerVolumeSpecName "kube-api-access-pthrr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:01:30.445050 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:01:30.445018 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pthrr\" (UniqueName: \"kubernetes.io/projected/4e94da38-4c98-445e-92d0-c3bf1806a938-kube-api-access-pthrr\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 19:01:31.175688 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:01:31.175650 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725620-xcnwt" Jul 08 19:01:31.176151 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:01:31.175681 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725620-xcnwt" event={"ID":"4e94da38-4c98-445e-92d0-c3bf1806a938","Type":"ContainerDied","Data":"256244d21390a6cfe1a7c3624bee5e85e2d0d032babbb3545d84b100f0c42237"} Jul 08 19:01:31.176151 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:01:31.175746 2559 scope.go:117] "RemoveContainer" containerID="8f0f0b1783b775b35378d58b7c85fc83be6b7d2c7c2ccc2fd9366446e3cab136" Jul 08 19:01:31.199322 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:01:31.199288 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725620-xcnwt"] Jul 08 19:01:31.204027 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:01:31.203999 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725620-xcnwt"] Jul 08 19:01:31.687962 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:01:31.687927 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e94da38-4c98-445e-92d0-c3bf1806a938" path="/var/lib/kubelet/pods/4e94da38-4c98-445e-92d0-c3bf1806a938/volumes" Jul 08 19:04:17.342517 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:04:17.342489 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zzt4h_d94a3566-fcf3-47a1-b176-a33b71c3826d/console-operator/1.log" Jul 08 19:04:17.349036 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:04:17.349010 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rj4xk_56014c3a-384a-4775-96e5-290968b196b1/ovn-acl-logging/0.log" Jul 08 19:04:17.805053 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:04:17.805028 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zzt4h_d94a3566-fcf3-47a1-b176-a33b71c3826d/console-operator/1.log" Jul 08 19:04:17.811710 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:04:17.811688 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rj4xk_56014c3a-384a-4775-96e5-290968b196b1/ovn-acl-logging/0.log" Jul 08 19:06:52.054671 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:52.054638 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-574a6850-847b47f577-x2v5v"] Jul 08 19:06:52.055135 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:52.055122 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e94da38-4c98-445e-92d0-c3bf1806a938" containerName="cleanup" Jul 08 19:06:52.055184 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:52.055138 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e94da38-4c98-445e-92d0-c3bf1806a938" containerName="cleanup" Jul 08 19:06:52.055226 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:52.055212 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e94da38-4c98-445e-92d0-c3bf1806a938" containerName="cleanup" Jul 08 19:06:52.058012 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:52.057990 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-574a6850-847b47f577-x2v5v" Jul 08 19:06:52.060564 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:52.060540 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-574a6850-serving-cert\"" Jul 08 19:06:52.060699 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:52.060598 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-kxbw6\"" Jul 08 19:06:52.069308 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:52.069273 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-574a6850-847b47f577-x2v5v"] Jul 08 19:06:52.191499 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:52.191454 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80334dbf-1798-44a4-8316-028cae1cd128-openshift-service-ca\") pod \"maas-api-e2e-ait-574a6850-847b47f577-x2v5v\" (UID: \"80334dbf-1798-44a4-8316-028cae1cd128\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-574a6850-847b47f577-x2v5v" Jul 08 19:06:52.191708 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:52.191590 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80334dbf-1798-44a4-8316-028cae1cd128-maas-api-tls\") pod \"maas-api-e2e-ait-574a6850-847b47f577-x2v5v\" (UID: \"80334dbf-1798-44a4-8316-028cae1cd128\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-574a6850-847b47f577-x2v5v" Jul 08 19:06:52.191708 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:52.191624 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5b7j\" (UniqueName: \"kubernetes.io/projected/80334dbf-1798-44a4-8316-028cae1cd128-kube-api-access-t5b7j\") pod \"maas-api-e2e-ait-574a6850-847b47f577-x2v5v\" (UID: \"80334dbf-1798-44a4-8316-028cae1cd128\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-574a6850-847b47f577-x2v5v" Jul 08 19:06:52.292656 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:52.292613 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80334dbf-1798-44a4-8316-028cae1cd128-maas-api-tls\") pod \"maas-api-e2e-ait-574a6850-847b47f577-x2v5v\" (UID: \"80334dbf-1798-44a4-8316-028cae1cd128\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-574a6850-847b47f577-x2v5v" Jul 08 19:06:52.292867 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:52.292664 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5b7j\" (UniqueName: \"kubernetes.io/projected/80334dbf-1798-44a4-8316-028cae1cd128-kube-api-access-t5b7j\") pod \"maas-api-e2e-ait-574a6850-847b47f577-x2v5v\" (UID: \"80334dbf-1798-44a4-8316-028cae1cd128\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-574a6850-847b47f577-x2v5v" Jul 08 19:06:52.292867 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:52.292757 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80334dbf-1798-44a4-8316-028cae1cd128-openshift-service-ca\") pod \"maas-api-e2e-ait-574a6850-847b47f577-x2v5v\" (UID: \"80334dbf-1798-44a4-8316-028cae1cd128\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-574a6850-847b47f577-x2v5v" Jul 08 19:06:52.292867 ip-10-0-143-154 kubenswrapper[2559]: E0708 19:06:52.292759 2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-574a6850-serving-cert: secret "maas-api-e2e-ait-574a6850-serving-cert" not found Jul 08 19:06:52.293012 ip-10-0-143-154 kubenswrapper[2559]: E0708 19:06:52.292934 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80334dbf-1798-44a4-8316-028cae1cd128-maas-api-tls podName:80334dbf-1798-44a4-8316-028cae1cd128 nodeName:}" failed. No retries permitted until 2026-07-08 19:06:52.792909284 +0000 UTC m=+1656.029207369 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/80334dbf-1798-44a4-8316-028cae1cd128-maas-api-tls") pod "maas-api-e2e-ait-574a6850-847b47f577-x2v5v" (UID: "80334dbf-1798-44a4-8316-028cae1cd128") : secret "maas-api-e2e-ait-574a6850-serving-cert" not found Jul 08 19:06:52.293501 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:52.293476 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80334dbf-1798-44a4-8316-028cae1cd128-openshift-service-ca\") pod \"maas-api-e2e-ait-574a6850-847b47f577-x2v5v\" (UID: \"80334dbf-1798-44a4-8316-028cae1cd128\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-574a6850-847b47f577-x2v5v" Jul 08 19:06:52.304444 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:52.304420 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5b7j\" (UniqueName: \"kubernetes.io/projected/80334dbf-1798-44a4-8316-028cae1cd128-kube-api-access-t5b7j\") pod \"maas-api-e2e-ait-574a6850-847b47f577-x2v5v\" (UID: \"80334dbf-1798-44a4-8316-028cae1cd128\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-574a6850-847b47f577-x2v5v" Jul 08 19:06:52.797887 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:52.797843 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80334dbf-1798-44a4-8316-028cae1cd128-maas-api-tls\") pod \"maas-api-e2e-ait-574a6850-847b47f577-x2v5v\" (UID: \"80334dbf-1798-44a4-8316-028cae1cd128\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-574a6850-847b47f577-x2v5v" Jul 08 19:06:52.800356 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:52.800328 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80334dbf-1798-44a4-8316-028cae1cd128-maas-api-tls\") pod \"maas-api-e2e-ait-574a6850-847b47f577-x2v5v\" (UID: \"80334dbf-1798-44a4-8316-028cae1cd128\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-574a6850-847b47f577-x2v5v" Jul 08 19:06:52.969979 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:52.969938 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-574a6850-847b47f577-x2v5v" Jul 08 19:06:53.130853 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:53.130822 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-574a6850-847b47f577-x2v5v"] Jul 08 19:06:53.133016 ip-10-0-143-154 kubenswrapper[2559]: W0708 19:06:53.132980 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80334dbf_1798_44a4_8316_028cae1cd128.slice/crio-f9d4141fe8beb8f44549012ddaaa1c310745ca7ecd697341ce798601cad214bd WatchSource:0}: Error finding container f9d4141fe8beb8f44549012ddaaa1c310745ca7ecd697341ce798601cad214bd: Status 404 returned error can't find the container with id f9d4141fe8beb8f44549012ddaaa1c310745ca7ecd697341ce798601cad214bd Jul 08 19:06:53.134474 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:53.134454 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 19:06:54.058443 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:54.058397 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-574a6850-847b47f577-x2v5v" event={"ID":"80334dbf-1798-44a4-8316-028cae1cd128","Type":"ContainerStarted","Data":"f9d4141fe8beb8f44549012ddaaa1c310745ca7ecd697341ce798601cad214bd"} Jul 08 19:06:56.072547 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:56.072511 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-574a6850-847b47f577-x2v5v" event={"ID":"80334dbf-1798-44a4-8316-028cae1cd128","Type":"ContainerStarted","Data":"12b59e84ba3809a9815e0b392bf861cf1c28f2be7ea39926fff4b2ad3f326365"} Jul 08 19:06:56.072968 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:56.072624 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-574a6850-847b47f577-x2v5v" Jul 08 19:06:56.090993 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:56.090947 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-574a6850-847b47f577-x2v5v" podStartSLOduration=2.058417443 podStartE2EDuration="4.090931507s" totalImagesPullingTime="2.032514064s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:06:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:06:56.090032554 +0000 UTC m=+1659.326330648" watchObservedRunningTime="2026-07-08 19:06:56.090931507 +0000 UTC m=+1659.327229596" Jul 08 19:06:57.465096 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:57.465061 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-574a6850-847b47f577-x2v5v"] Jul 08 19:06:58.084434 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:58.084376 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-574a6850-847b47f577-x2v5v" podUID="80334dbf-1798-44a4-8316-028cae1cd128" containerName="maas-api" containerID="cri-o://12b59e84ba3809a9815e0b392bf861cf1c28f2be7ea39926fff4b2ad3f326365" gracePeriod=30 Jul 08 19:06:58.347664 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:58.347598 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-574a6850-847b47f577-x2v5v" Jul 08 19:06:58.466633 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:58.466594 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80334dbf-1798-44a4-8316-028cae1cd128-openshift-service-ca\") pod \"80334dbf-1798-44a4-8316-028cae1cd128\" (UID: \"80334dbf-1798-44a4-8316-028cae1cd128\") " Jul 08 19:06:58.467137 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:58.466641 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80334dbf-1798-44a4-8316-028cae1cd128-maas-api-tls\") pod \"80334dbf-1798-44a4-8316-028cae1cd128\" (UID: \"80334dbf-1798-44a4-8316-028cae1cd128\") " Jul 08 19:06:58.467137 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:58.466672 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t5b7j\" (UniqueName: \"kubernetes.io/projected/80334dbf-1798-44a4-8316-028cae1cd128-kube-api-access-t5b7j\") pod \"80334dbf-1798-44a4-8316-028cae1cd128\" (UID: \"80334dbf-1798-44a4-8316-028cae1cd128\") " Jul 08 19:06:58.467137 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:58.466982 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80334dbf-1798-44a4-8316-028cae1cd128-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "80334dbf-1798-44a4-8316-028cae1cd128" (UID: "80334dbf-1798-44a4-8316-028cae1cd128"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:06:58.468761 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:58.468736 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80334dbf-1798-44a4-8316-028cae1cd128-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "80334dbf-1798-44a4-8316-028cae1cd128" (UID: "80334dbf-1798-44a4-8316-028cae1cd128"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:06:58.468875 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:58.468851 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80334dbf-1798-44a4-8316-028cae1cd128-kube-api-access-t5b7j" (OuterVolumeSpecName: "kube-api-access-t5b7j") pod "80334dbf-1798-44a4-8316-028cae1cd128" (UID: "80334dbf-1798-44a4-8316-028cae1cd128"). InnerVolumeSpecName "kube-api-access-t5b7j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:06:58.567465 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:58.567409 2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80334dbf-1798-44a4-8316-028cae1cd128-openshift-service-ca\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 19:06:58.567465 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:58.567455 2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80334dbf-1798-44a4-8316-028cae1cd128-maas-api-tls\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 19:06:58.567465 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:58.567472 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t5b7j\" (UniqueName: \"kubernetes.io/projected/80334dbf-1798-44a4-8316-028cae1cd128-kube-api-access-t5b7j\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 19:06:59.092085 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:59.092001 2559 generic.go:356] "Generic (PLEG): container finished" podID="80334dbf-1798-44a4-8316-028cae1cd128" containerID="12b59e84ba3809a9815e0b392bf861cf1c28f2be7ea39926fff4b2ad3f326365" exitCode=0 Jul 08 19:06:59.092085 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:59.092050 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-574a6850-847b47f577-x2v5v" Jul 08 19:06:59.092270 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:59.092090 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-574a6850-847b47f577-x2v5v" event={"ID":"80334dbf-1798-44a4-8316-028cae1cd128","Type":"ContainerDied","Data":"12b59e84ba3809a9815e0b392bf861cf1c28f2be7ea39926fff4b2ad3f326365"} Jul 08 19:06:59.092270 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:59.092146 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-574a6850-847b47f577-x2v5v" event={"ID":"80334dbf-1798-44a4-8316-028cae1cd128","Type":"ContainerDied","Data":"f9d4141fe8beb8f44549012ddaaa1c310745ca7ecd697341ce798601cad214bd"} Jul 08 19:06:59.092270 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:59.092175 2559 scope.go:117] "RemoveContainer" containerID="12b59e84ba3809a9815e0b392bf861cf1c28f2be7ea39926fff4b2ad3f326365" Jul 08 19:06:59.101879 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:59.101857 2559 scope.go:117] "RemoveContainer" containerID="12b59e84ba3809a9815e0b392bf861cf1c28f2be7ea39926fff4b2ad3f326365" Jul 08 19:06:59.102150 ip-10-0-143-154 kubenswrapper[2559]: E0708 19:06:59.102130 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12b59e84ba3809a9815e0b392bf861cf1c28f2be7ea39926fff4b2ad3f326365\": container with ID starting with 12b59e84ba3809a9815e0b392bf861cf1c28f2be7ea39926fff4b2ad3f326365 not found: ID does not exist" containerID="12b59e84ba3809a9815e0b392bf861cf1c28f2be7ea39926fff4b2ad3f326365" Jul 08 19:06:59.102213 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:59.102160 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12b59e84ba3809a9815e0b392bf861cf1c28f2be7ea39926fff4b2ad3f326365"} err="failed to get container status \"12b59e84ba3809a9815e0b392bf861cf1c28f2be7ea39926fff4b2ad3f326365\": rpc error: code = NotFound desc = could not find container \"12b59e84ba3809a9815e0b392bf861cf1c28f2be7ea39926fff4b2ad3f326365\": container with ID starting with 12b59e84ba3809a9815e0b392bf861cf1c28f2be7ea39926fff4b2ad3f326365 not found: ID does not exist" Jul 08 19:06:59.115978 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:59.115947 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-574a6850-847b47f577-x2v5v"] Jul 08 19:06:59.120165 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:59.120142 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-574a6850-847b47f577-x2v5v"] Jul 08 19:06:59.688999 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:06:59.688964 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80334dbf-1798-44a4-8316-028cae1cd128" path="/var/lib/kubelet/pods/80334dbf-1798-44a4-8316-028cae1cd128/volumes" Jul 08 19:07:24.178682 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:24.178645 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d50245fc-697cd9d855-zxggb"] Jul 08 19:07:24.179159 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:24.179141 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80334dbf-1798-44a4-8316-028cae1cd128" containerName="maas-api" Jul 08 19:07:24.179205 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:24.179161 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="80334dbf-1798-44a4-8316-028cae1cd128" containerName="maas-api" Jul 08 19:07:24.179240 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:24.179226 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="80334dbf-1798-44a4-8316-028cae1cd128" containerName="maas-api" Jul 08 19:07:24.184566 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:24.184548 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d50245fc-697cd9d855-zxggb" Jul 08 19:07:24.187429 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:24.187405 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-d50245fc-serving-cert\"" Jul 08 19:07:24.187557 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:24.187493 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-kxbw6\"" Jul 08 19:07:24.200567 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:24.200545 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d50245fc-697cd9d855-zxggb"] Jul 08 19:07:24.329970 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:24.329928 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f1f5942-3a3f-43fe-aa8d-5b7e6808674a-openshift-service-ca\") pod \"maas-api-e2e-derive-d50245fc-697cd9d855-zxggb\" (UID: \"5f1f5942-3a3f-43fe-aa8d-5b7e6808674a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d50245fc-697cd9d855-zxggb" Jul 08 19:07:24.330164 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:24.330007 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f1f5942-3a3f-43fe-aa8d-5b7e6808674a-maas-api-tls\") pod \"maas-api-e2e-derive-d50245fc-697cd9d855-zxggb\" (UID: \"5f1f5942-3a3f-43fe-aa8d-5b7e6808674a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d50245fc-697cd9d855-zxggb" Jul 08 19:07:24.330164 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:24.330032 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k55z7\" (UniqueName: \"kubernetes.io/projected/5f1f5942-3a3f-43fe-aa8d-5b7e6808674a-kube-api-access-k55z7\") pod \"maas-api-e2e-derive-d50245fc-697cd9d855-zxggb\" (UID: \"5f1f5942-3a3f-43fe-aa8d-5b7e6808674a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d50245fc-697cd9d855-zxggb" Jul 08 19:07:24.431278 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:24.431181 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f1f5942-3a3f-43fe-aa8d-5b7e6808674a-openshift-service-ca\") pod \"maas-api-e2e-derive-d50245fc-697cd9d855-zxggb\" (UID: \"5f1f5942-3a3f-43fe-aa8d-5b7e6808674a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d50245fc-697cd9d855-zxggb" Jul 08 19:07:24.431469 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:24.431351 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f1f5942-3a3f-43fe-aa8d-5b7e6808674a-maas-api-tls\") pod \"maas-api-e2e-derive-d50245fc-697cd9d855-zxggb\" (UID: \"5f1f5942-3a3f-43fe-aa8d-5b7e6808674a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d50245fc-697cd9d855-zxggb" Jul 08 19:07:24.431469 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:24.431412 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k55z7\" (UniqueName: \"kubernetes.io/projected/5f1f5942-3a3f-43fe-aa8d-5b7e6808674a-kube-api-access-k55z7\") pod \"maas-api-e2e-derive-d50245fc-697cd9d855-zxggb\" (UID: \"5f1f5942-3a3f-43fe-aa8d-5b7e6808674a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d50245fc-697cd9d855-zxggb" Jul 08 19:07:24.431577 ip-10-0-143-154 kubenswrapper[2559]: E0708 19:07:24.431490 2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-d50245fc-serving-cert: secret "maas-api-e2e-derive-d50245fc-serving-cert" not found Jul 08 19:07:24.431577 ip-10-0-143-154 kubenswrapper[2559]: E0708 19:07:24.431562 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f1f5942-3a3f-43fe-aa8d-5b7e6808674a-maas-api-tls podName:5f1f5942-3a3f-43fe-aa8d-5b7e6808674a nodeName:}" failed. No retries permitted until 2026-07-08 19:07:24.931542976 +0000 UTC m=+1688.167841045 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5f1f5942-3a3f-43fe-aa8d-5b7e6808674a-maas-api-tls") pod "maas-api-e2e-derive-d50245fc-697cd9d855-zxggb" (UID: "5f1f5942-3a3f-43fe-aa8d-5b7e6808674a") : secret "maas-api-e2e-derive-d50245fc-serving-cert" not found Jul 08 19:07:24.431922 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:24.431903 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f1f5942-3a3f-43fe-aa8d-5b7e6808674a-openshift-service-ca\") pod \"maas-api-e2e-derive-d50245fc-697cd9d855-zxggb\" (UID: \"5f1f5942-3a3f-43fe-aa8d-5b7e6808674a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d50245fc-697cd9d855-zxggb" Jul 08 19:07:24.448211 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:24.448183 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k55z7\" (UniqueName: \"kubernetes.io/projected/5f1f5942-3a3f-43fe-aa8d-5b7e6808674a-kube-api-access-k55z7\") pod \"maas-api-e2e-derive-d50245fc-697cd9d855-zxggb\" (UID: \"5f1f5942-3a3f-43fe-aa8d-5b7e6808674a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d50245fc-697cd9d855-zxggb" Jul 08 19:07:24.936340 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:24.936301 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f1f5942-3a3f-43fe-aa8d-5b7e6808674a-maas-api-tls\") pod \"maas-api-e2e-derive-d50245fc-697cd9d855-zxggb\" (UID: \"5f1f5942-3a3f-43fe-aa8d-5b7e6808674a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d50245fc-697cd9d855-zxggb" Jul 08 19:07:24.938777 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:24.938747 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f1f5942-3a3f-43fe-aa8d-5b7e6808674a-maas-api-tls\") pod \"maas-api-e2e-derive-d50245fc-697cd9d855-zxggb\" (UID: \"5f1f5942-3a3f-43fe-aa8d-5b7e6808674a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d50245fc-697cd9d855-zxggb" Jul 08 19:07:25.094833 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:25.094790 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d50245fc-697cd9d855-zxggb" Jul 08 19:07:25.250788 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:25.250744 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d50245fc-697cd9d855-zxggb"] Jul 08 19:07:25.252738 ip-10-0-143-154 kubenswrapper[2559]: W0708 19:07:25.252714 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f1f5942_3a3f_43fe_aa8d_5b7e6808674a.slice/crio-d8c609b30435ee3e08fea6449ca12d78a27de271cf85954b4cd228038bab4188 WatchSource:0}: Error finding container d8c609b30435ee3e08fea6449ca12d78a27de271cf85954b4cd228038bab4188: Status 404 returned error can't find the container with id d8c609b30435ee3e08fea6449ca12d78a27de271cf85954b4cd228038bab4188 Jul 08 19:07:26.251510 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:26.251461 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d50245fc-697cd9d855-zxggb" event={"ID":"5f1f5942-3a3f-43fe-aa8d-5b7e6808674a","Type":"ContainerStarted","Data":"0c08405b57f50901e315360aa872ee20e27bc1fe87d9d5dbe41fd058c02da7c8"} Jul 08 19:07:26.251993 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:26.251525 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d50245fc-697cd9d855-zxggb" event={"ID":"5f1f5942-3a3f-43fe-aa8d-5b7e6808674a","Type":"ContainerStarted","Data":"d8c609b30435ee3e08fea6449ca12d78a27de271cf85954b4cd228038bab4188"} Jul 08 19:07:26.251993 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:26.251675 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d50245fc-697cd9d855-zxggb" Jul 08 19:07:26.273463 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:26.273410 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d50245fc-697cd9d855-zxggb" podStartSLOduration=1.849636597 podStartE2EDuration="2.273395147s" totalImagesPullingTime="423.75855ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:07:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:07:26.272421162 +0000 UTC m=+1689.508719254" watchObservedRunningTime="2026-07-08 19:07:26.273395147 +0000 UTC m=+1689.509693238" Jul 08 19:07:29.063899 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:29.063861 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d50245fc-697cd9d855-zxggb"] Jul 08 19:07:29.064326 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:29.064067 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d50245fc-697cd9d855-zxggb" podUID="5f1f5942-3a3f-43fe-aa8d-5b7e6808674a" containerName="maas-api" containerID="cri-o://0c08405b57f50901e315360aa872ee20e27bc1fe87d9d5dbe41fd058c02da7c8" gracePeriod=30 Jul 08 19:07:29.271986 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:29.271956 2559 generic.go:356] "Generic (PLEG): container finished" podID="5f1f5942-3a3f-43fe-aa8d-5b7e6808674a" containerID="0c08405b57f50901e315360aa872ee20e27bc1fe87d9d5dbe41fd058c02da7c8" exitCode=0 Jul 08 19:07:29.272144 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:29.272008 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d50245fc-697cd9d855-zxggb" event={"ID":"5f1f5942-3a3f-43fe-aa8d-5b7e6808674a","Type":"ContainerDied","Data":"0c08405b57f50901e315360aa872ee20e27bc1fe87d9d5dbe41fd058c02da7c8"} Jul 08 19:07:29.320124 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:29.320062 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d50245fc-697cd9d855-zxggb" Jul 08 19:07:29.482263 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:29.482232 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f1f5942-3a3f-43fe-aa8d-5b7e6808674a-maas-api-tls\") pod \"5f1f5942-3a3f-43fe-aa8d-5b7e6808674a\" (UID: \"5f1f5942-3a3f-43fe-aa8d-5b7e6808674a\") " Jul 08 19:07:29.482474 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:29.482351 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f1f5942-3a3f-43fe-aa8d-5b7e6808674a-openshift-service-ca\") pod \"5f1f5942-3a3f-43fe-aa8d-5b7e6808674a\" (UID: \"5f1f5942-3a3f-43fe-aa8d-5b7e6808674a\") " Jul 08 19:07:29.482474 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:29.482426 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k55z7\" (UniqueName: \"kubernetes.io/projected/5f1f5942-3a3f-43fe-aa8d-5b7e6808674a-kube-api-access-k55z7\") pod \"5f1f5942-3a3f-43fe-aa8d-5b7e6808674a\" (UID: \"5f1f5942-3a3f-43fe-aa8d-5b7e6808674a\") " Jul 08 19:07:29.482780 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:29.482735 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f1f5942-3a3f-43fe-aa8d-5b7e6808674a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5f1f5942-3a3f-43fe-aa8d-5b7e6808674a" (UID: "5f1f5942-3a3f-43fe-aa8d-5b7e6808674a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:07:29.484402 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:29.484375 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f1f5942-3a3f-43fe-aa8d-5b7e6808674a-kube-api-access-k55z7" (OuterVolumeSpecName: "kube-api-access-k55z7") pod "5f1f5942-3a3f-43fe-aa8d-5b7e6808674a" (UID: "5f1f5942-3a3f-43fe-aa8d-5b7e6808674a"). InnerVolumeSpecName "kube-api-access-k55z7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:07:29.484498 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:29.484429 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f1f5942-3a3f-43fe-aa8d-5b7e6808674a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5f1f5942-3a3f-43fe-aa8d-5b7e6808674a" (UID: "5f1f5942-3a3f-43fe-aa8d-5b7e6808674a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:07:29.583680 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:29.583594 2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f1f5942-3a3f-43fe-aa8d-5b7e6808674a-maas-api-tls\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 19:07:29.583680 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:29.583623 2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f1f5942-3a3f-43fe-aa8d-5b7e6808674a-openshift-service-ca\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 19:07:29.583680 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:29.583635 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k55z7\" (UniqueName: \"kubernetes.io/projected/5f1f5942-3a3f-43fe-aa8d-5b7e6808674a-kube-api-access-k55z7\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 19:07:30.279196 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:30.279166 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d50245fc-697cd9d855-zxggb" Jul 08 19:07:30.279196 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:30.279182 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d50245fc-697cd9d855-zxggb" event={"ID":"5f1f5942-3a3f-43fe-aa8d-5b7e6808674a","Type":"ContainerDied","Data":"d8c609b30435ee3e08fea6449ca12d78a27de271cf85954b4cd228038bab4188"} Jul 08 19:07:30.279708 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:30.279248 2559 scope.go:117] "RemoveContainer" containerID="0c08405b57f50901e315360aa872ee20e27bc1fe87d9d5dbe41fd058c02da7c8" Jul 08 19:07:30.305980 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:30.305952 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d50245fc-697cd9d855-zxggb"] Jul 08 19:07:30.310123 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:30.310095 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d50245fc-697cd9d855-zxggb"] Jul 08 19:07:31.688015 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:07:31.687979 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f1f5942-3a3f-43fe-aa8d-5b7e6808674a" path="/var/lib/kubelet/pods/5f1f5942-3a3f-43fe-aa8d-5b7e6808674a/volumes" Jul 08 19:09:17.384492 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:09:17.384462 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zzt4h_d94a3566-fcf3-47a1-b176-a33b71c3826d/console-operator/1.log" Jul 08 19:09:17.390857 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:09:17.390833 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rj4xk_56014c3a-384a-4775-96e5-290968b196b1/ovn-acl-logging/0.log" Jul 08 19:09:17.848700 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:09:17.848615 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zzt4h_d94a3566-fcf3-47a1-b176-a33b71c3826d/console-operator/1.log" Jul 08 19:09:17.855666 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:09:17.855645 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rj4xk_56014c3a-384a-4775-96e5-290968b196b1/ovn-acl-logging/0.log" Jul 08 19:13:03.206565 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:13:03.206530 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-521fd5-6dcd86744b-j4lgw"] Jul 08 19:13:03.206997 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:13:03.206979 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f1f5942-3a3f-43fe-aa8d-5b7e6808674a" containerName="maas-api" Jul 08 19:13:03.206997 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:13:03.206991 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f1f5942-3a3f-43fe-aa8d-5b7e6808674a" containerName="maas-api" Jul 08 19:13:03.207073 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:13:03.207055 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f1f5942-3a3f-43fe-aa8d-5b7e6808674a" containerName="maas-api" Jul 08 19:13:03.210162 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:13:03.210145 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-521fd5-6dcd86744b-j4lgw" Jul 08 19:13:03.212703 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:13:03.212677 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-521fd5-serving-cert\"" Jul 08 19:13:03.212858 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:13:03.212796 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-kxbw6\"" Jul 08 19:13:03.222596 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:13:03.222570 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-521fd5-6dcd86744b-j4lgw"] Jul 08 19:13:03.304150 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:13:03.304115 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6hkt\" (UniqueName: \"kubernetes.io/projected/d57ede72-c2d9-456e-b97f-91ac30b5024f-kube-api-access-n6hkt\") pod \"maas-api-e2e-inf-b-521fd5-6dcd86744b-j4lgw\" (UID: \"d57ede72-c2d9-456e-b97f-91ac30b5024f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-521fd5-6dcd86744b-j4lgw" Jul 08 19:13:03.304398 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:13:03.304160 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d57ede72-c2d9-456e-b97f-91ac30b5024f-maas-api-tls\") pod \"maas-api-e2e-inf-b-521fd5-6dcd86744b-j4lgw\" (UID: \"d57ede72-c2d9-456e-b97f-91ac30b5024f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-521fd5-6dcd86744b-j4lgw" Jul 08 19:13:03.304398 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:13:03.304216 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d57ede72-c2d9-456e-b97f-91ac30b5024f-openshift-service-ca\") pod \"maas-api-e2e-inf-b-521fd5-6dcd86744b-j4lgw\" (UID: \"d57ede72-c2d9-456e-b97f-91ac30b5024f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-521fd5-6dcd86744b-j4lgw" Jul 08 19:13:03.405606 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:13:03.405556 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d57ede72-c2d9-456e-b97f-91ac30b5024f-openshift-service-ca\") pod \"maas-api-e2e-inf-b-521fd5-6dcd86744b-j4lgw\" (UID: \"d57ede72-c2d9-456e-b97f-91ac30b5024f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-521fd5-6dcd86744b-j4lgw" Jul 08 19:13:03.405843 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:13:03.405678 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6hkt\" (UniqueName: \"kubernetes.io/projected/d57ede72-c2d9-456e-b97f-91ac30b5024f-kube-api-access-n6hkt\") pod \"maas-api-e2e-inf-b-521fd5-6dcd86744b-j4lgw\" (UID: \"d57ede72-c2d9-456e-b97f-91ac30b5024f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-521fd5-6dcd86744b-j4lgw" Jul 08 19:13:03.405843 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:13:03.405713 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d57ede72-c2d9-456e-b97f-91ac30b5024f-maas-api-tls\") pod \"maas-api-e2e-inf-b-521fd5-6dcd86744b-j4lgw\" (UID: \"d57ede72-c2d9-456e-b97f-91ac30b5024f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-521fd5-6dcd86744b-j4lgw" Jul 08 19:13:03.405932 ip-10-0-143-154 kubenswrapper[2559]: E0708 19:13:03.405860 2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-521fd5-serving-cert: secret "maas-api-e2e-inf-b-521fd5-serving-cert" not found Jul 08 19:13:03.405967 ip-10-0-143-154 kubenswrapper[2559]: E0708 19:13:03.405938 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d57ede72-c2d9-456e-b97f-91ac30b5024f-maas-api-tls podName:d57ede72-c2d9-456e-b97f-91ac30b5024f nodeName:}" failed. No retries permitted until 2026-07-08 19:13:03.905917746 +0000 UTC m=+2027.142215821 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d57ede72-c2d9-456e-b97f-91ac30b5024f-maas-api-tls") pod "maas-api-e2e-inf-b-521fd5-6dcd86744b-j4lgw" (UID: "d57ede72-c2d9-456e-b97f-91ac30b5024f") : secret "maas-api-e2e-inf-b-521fd5-serving-cert" not found Jul 08 19:13:03.406308 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:13:03.406287 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d57ede72-c2d9-456e-b97f-91ac30b5024f-openshift-service-ca\") pod \"maas-api-e2e-inf-b-521fd5-6dcd86744b-j4lgw\" (UID: \"d57ede72-c2d9-456e-b97f-91ac30b5024f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-521fd5-6dcd86744b-j4lgw" Jul 08 19:13:03.416740 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:13:03.416707 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6hkt\" (UniqueName: \"kubernetes.io/projected/d57ede72-c2d9-456e-b97f-91ac30b5024f-kube-api-access-n6hkt\") pod \"maas-api-e2e-inf-b-521fd5-6dcd86744b-j4lgw\" (UID: \"d57ede72-c2d9-456e-b97f-91ac30b5024f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-521fd5-6dcd86744b-j4lgw" Jul 08 19:13:03.910677 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:13:03.910640 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d57ede72-c2d9-456e-b97f-91ac30b5024f-maas-api-tls\") pod \"maas-api-e2e-inf-b-521fd5-6dcd86744b-j4lgw\" (UID: \"d57ede72-c2d9-456e-b97f-91ac30b5024f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-521fd5-6dcd86744b-j4lgw" Jul 08 19:13:03.913135 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:13:03.913103 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d57ede72-c2d9-456e-b97f-91ac30b5024f-maas-api-tls\") pod \"maas-api-e2e-inf-b-521fd5-6dcd86744b-j4lgw\" (UID: \"d57ede72-c2d9-456e-b97f-91ac30b5024f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-521fd5-6dcd86744b-j4lgw" Jul 08 19:13:04.122234 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:13:04.122201 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-521fd5-6dcd86744b-j4lgw" Jul 08 19:13:04.267459 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:13:04.267434 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-521fd5-6dcd86744b-j4lgw"] Jul 08 19:13:04.269236 ip-10-0-143-154 kubenswrapper[2559]: W0708 19:13:04.269207 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd57ede72_c2d9_456e_b97f_91ac30b5024f.slice/crio-ef00dea02dbe905ce7be5b65c67d73ba492a096e838c4c045828b30439ac1268 WatchSource:0}: Error finding container ef00dea02dbe905ce7be5b65c67d73ba492a096e838c4c045828b30439ac1268: Status 404 returned error can't find the container with id ef00dea02dbe905ce7be5b65c67d73ba492a096e838c4c045828b30439ac1268 Jul 08 19:13:04.270479 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:13:04.270463 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 19:13:05.258703 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:13:05.258656 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-521fd5-6dcd86744b-j4lgw" event={"ID":"d57ede72-c2d9-456e-b97f-91ac30b5024f","Type":"ContainerStarted","Data":"4347e4636bf2c8b34e7c22212895090b5d54309c46adef4dc5ea248ed59e8a3f"} Jul 08 19:13:05.258901 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:13:05.258806 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-521fd5-6dcd86744b-j4lgw" Jul 08 19:13:05.258901 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:13:05.258846 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-521fd5-6dcd86744b-j4lgw" event={"ID":"d57ede72-c2d9-456e-b97f-91ac30b5024f","Type":"ContainerStarted","Data":"ef00dea02dbe905ce7be5b65c67d73ba492a096e838c4c045828b30439ac1268"} Jul 08 19:13:05.295538 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:13:05.295482 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-521fd5-6dcd86744b-j4lgw" podStartSLOduration=1.957536116 podStartE2EDuration="2.295460163s" totalImagesPullingTime="337.924047ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:13:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:13:05.293082473 +0000 UTC m=+2028.529380562" watchObservedRunningTime="2026-07-08 19:13:05.295460163 +0000 UTC m=+2028.531758286" Jul 08 19:13:11.270963 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:13:11.270932 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-521fd5-6dcd86744b-j4lgw" Jul 08 19:14:17.425057 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:14:17.425029 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zzt4h_d94a3566-fcf3-47a1-b176-a33b71c3826d/console-operator/1.log" Jul 08 19:14:17.432098 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:14:17.432076 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rj4xk_56014c3a-384a-4775-96e5-290968b196b1/ovn-acl-logging/0.log" Jul 08 19:14:17.888375 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:14:17.888343 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zzt4h_d94a3566-fcf3-47a1-b176-a33b71c3826d/console-operator/1.log" Jul 08 19:14:17.894939 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:14:17.894910 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rj4xk_56014c3a-384a-4775-96e5-290968b196b1/ovn-acl-logging/0.log" Jul 08 19:14:51.566210 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:14:51.566176 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-521fd5-6dcd86744b-j4lgw"] Jul 08 19:14:51.566818 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:14:51.566399 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-521fd5-6dcd86744b-j4lgw" podUID="d57ede72-c2d9-456e-b97f-91ac30b5024f" containerName="maas-api" containerID="cri-o://4347e4636bf2c8b34e7c22212895090b5d54309c46adef4dc5ea248ed59e8a3f" gracePeriod=30 Jul 08 19:14:51.888744 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:14:51.888709 2559 generic.go:356] "Generic (PLEG): container finished" podID="d57ede72-c2d9-456e-b97f-91ac30b5024f" containerID="4347e4636bf2c8b34e7c22212895090b5d54309c46adef4dc5ea248ed59e8a3f" exitCode=0 Jul 08 19:14:51.888877 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:14:51.888806 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-521fd5-6dcd86744b-j4lgw" event={"ID":"d57ede72-c2d9-456e-b97f-91ac30b5024f","Type":"ContainerDied","Data":"4347e4636bf2c8b34e7c22212895090b5d54309c46adef4dc5ea248ed59e8a3f"} Jul 08 19:14:52.009919 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:14:52.009893 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-521fd5-6dcd86744b-j4lgw" Jul 08 19:14:52.075489 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:14:52.075454 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6hkt\" (UniqueName: \"kubernetes.io/projected/d57ede72-c2d9-456e-b97f-91ac30b5024f-kube-api-access-n6hkt\") pod \"d57ede72-c2d9-456e-b97f-91ac30b5024f\" (UID: \"d57ede72-c2d9-456e-b97f-91ac30b5024f\") " Jul 08 19:14:52.075657 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:14:52.075523 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d57ede72-c2d9-456e-b97f-91ac30b5024f-openshift-service-ca\") pod \"d57ede72-c2d9-456e-b97f-91ac30b5024f\" (UID: \"d57ede72-c2d9-456e-b97f-91ac30b5024f\") " Jul 08 19:14:52.075657 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:14:52.075593 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d57ede72-c2d9-456e-b97f-91ac30b5024f-maas-api-tls\") pod \"d57ede72-c2d9-456e-b97f-91ac30b5024f\" (UID: \"d57ede72-c2d9-456e-b97f-91ac30b5024f\") " Jul 08 19:14:52.076010 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:14:52.075977 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d57ede72-c2d9-456e-b97f-91ac30b5024f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d57ede72-c2d9-456e-b97f-91ac30b5024f" (UID: "d57ede72-c2d9-456e-b97f-91ac30b5024f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:14:52.077622 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:14:52.077596 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d57ede72-c2d9-456e-b97f-91ac30b5024f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d57ede72-c2d9-456e-b97f-91ac30b5024f" (UID: "d57ede72-c2d9-456e-b97f-91ac30b5024f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:14:52.077622 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:14:52.077613 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d57ede72-c2d9-456e-b97f-91ac30b5024f-kube-api-access-n6hkt" (OuterVolumeSpecName: "kube-api-access-n6hkt") pod "d57ede72-c2d9-456e-b97f-91ac30b5024f" (UID: "d57ede72-c2d9-456e-b97f-91ac30b5024f"). InnerVolumeSpecName "kube-api-access-n6hkt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:14:52.176544 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:14:52.176503 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6hkt\" (UniqueName: \"kubernetes.io/projected/d57ede72-c2d9-456e-b97f-91ac30b5024f-kube-api-access-n6hkt\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 19:14:52.176544 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:14:52.176545 2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d57ede72-c2d9-456e-b97f-91ac30b5024f-openshift-service-ca\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 19:14:52.176544 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:14:52.176556 2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d57ede72-c2d9-456e-b97f-91ac30b5024f-maas-api-tls\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 19:14:52.895185 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:14:52.895158 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-521fd5-6dcd86744b-j4lgw" Jul 08 19:14:52.895185 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:14:52.895169 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-521fd5-6dcd86744b-j4lgw" event={"ID":"d57ede72-c2d9-456e-b97f-91ac30b5024f","Type":"ContainerDied","Data":"ef00dea02dbe905ce7be5b65c67d73ba492a096e838c4c045828b30439ac1268"} Jul 08 19:14:52.895697 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:14:52.895222 2559 scope.go:117] "RemoveContainer" containerID="4347e4636bf2c8b34e7c22212895090b5d54309c46adef4dc5ea248ed59e8a3f" Jul 08 19:14:52.919755 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:14:52.919723 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-521fd5-6dcd86744b-j4lgw"] Jul 08 19:14:52.932969 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:14:52.926810 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-521fd5-6dcd86744b-j4lgw"] Jul 08 19:14:53.687682 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:14:53.687650 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d57ede72-c2d9-456e-b97f-91ac30b5024f" path="/var/lib/kubelet/pods/d57ede72-c2d9-456e-b97f-91ac30b5024f/volumes" Jul 08 19:15:00.134752 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:00.134717 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725635-9929s"] Jul 08 19:15:00.135226 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:00.135151 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d57ede72-c2d9-456e-b97f-91ac30b5024f" containerName="maas-api" Jul 08 19:15:00.135226 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:00.135163 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d57ede72-c2d9-456e-b97f-91ac30b5024f" containerName="maas-api" Jul 08 19:15:00.135308 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:00.135254 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d57ede72-c2d9-456e-b97f-91ac30b5024f" containerName="maas-api" Jul 08 19:15:00.139806 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:00.139760 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725635-9929s" Jul 08 19:15:00.143096 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:00.143075 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-kxbw6\"" Jul 08 19:15:00.151666 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:00.151642 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725635-9929s"] Jul 08 19:15:00.255957 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:00.255917 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62j8b\" (UniqueName: \"kubernetes.io/projected/55664b9e-537b-416d-ac85-79899d694d41-kube-api-access-62j8b\") pod \"maas-api-key-cleanup-29725635-9929s\" (UID: \"55664b9e-537b-416d-ac85-79899d694d41\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725635-9929s" Jul 08 19:15:00.357227 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:00.357183 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62j8b\" (UniqueName: \"kubernetes.io/projected/55664b9e-537b-416d-ac85-79899d694d41-kube-api-access-62j8b\") pod \"maas-api-key-cleanup-29725635-9929s\" (UID: \"55664b9e-537b-416d-ac85-79899d694d41\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725635-9929s" Jul 08 19:15:00.368956 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:00.368928 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-62j8b\" (UniqueName: \"kubernetes.io/projected/55664b9e-537b-416d-ac85-79899d694d41-kube-api-access-62j8b\") pod \"maas-api-key-cleanup-29725635-9929s\" (UID: \"55664b9e-537b-416d-ac85-79899d694d41\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725635-9929s" Jul 08 19:15:00.451405 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:00.451317 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725635-9929s" Jul 08 19:15:00.594166 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:00.594132 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725635-9929s"] Jul 08 19:15:00.595829 ip-10-0-143-154 kubenswrapper[2559]: W0708 19:15:00.595803 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55664b9e_537b_416d_ac85_79899d694d41.slice/crio-1105fe73b13ab9d331f12eeb04db17263c77c768a45129bf8936239ee7959e00 WatchSource:0}: Error finding container 1105fe73b13ab9d331f12eeb04db17263c77c768a45129bf8936239ee7959e00: Status 404 returned error can't find the container with id 1105fe73b13ab9d331f12eeb04db17263c77c768a45129bf8936239ee7959e00 Jul 08 19:15:00.943795 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:00.943743 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725635-9929s" event={"ID":"55664b9e-537b-416d-ac85-79899d694d41","Type":"ContainerStarted","Data":"1105fe73b13ab9d331f12eeb04db17263c77c768a45129bf8936239ee7959e00"} Jul 08 19:15:00.962602 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:00.962555 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725635-9929s" podStartSLOduration=0.962537677 podStartE2EDuration="962.537677ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:15:00.961459199 +0000 UTC m=+2144.197757289" watchObservedRunningTime="2026-07-08 19:15:00.962537677 +0000 UTC m=+2144.198835767" Jul 08 19:15:01.950874 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:01.950823 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725635-9929s" event={"ID":"55664b9e-537b-416d-ac85-79899d694d41","Type":"ContainerStarted","Data":"b0027ee4e9601f3411fa3bcef85c0e0f14fd280030e90529a2d780e5553d8213"} Jul 08 19:15:02.792530 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:02.792488 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-30f36b-79df68db65-ql9cm"] Jul 08 19:15:02.796077 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:02.796061 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-30f36b-79df68db65-ql9cm" Jul 08 19:15:02.799048 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:02.799028 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-30f36b-serving-cert\"" Jul 08 19:15:02.807036 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:02.807016 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-30f36b-79df68db65-ql9cm"] Jul 08 19:15:02.882279 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:02.882238 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b7b4258-9a57-4bb1-820e-2b087abefcac-maas-api-tls\") pod \"maas-api-e2e-api-a-30f36b-79df68db65-ql9cm\" (UID: \"7b7b4258-9a57-4bb1-820e-2b087abefcac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-30f36b-79df68db65-ql9cm" Jul 08 19:15:02.882454 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:02.882329 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b7b4258-9a57-4bb1-820e-2b087abefcac-openshift-service-ca\") pod \"maas-api-e2e-api-a-30f36b-79df68db65-ql9cm\" (UID: \"7b7b4258-9a57-4bb1-820e-2b087abefcac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-30f36b-79df68db65-ql9cm" Jul 08 19:15:02.882454 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:02.882374 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqgws\" (UniqueName: \"kubernetes.io/projected/7b7b4258-9a57-4bb1-820e-2b087abefcac-kube-api-access-wqgws\") pod \"maas-api-e2e-api-a-30f36b-79df68db65-ql9cm\" (UID: \"7b7b4258-9a57-4bb1-820e-2b087abefcac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-30f36b-79df68db65-ql9cm" Jul 08 19:15:02.983326 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:02.983288 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b7b4258-9a57-4bb1-820e-2b087abefcac-openshift-service-ca\") pod \"maas-api-e2e-api-a-30f36b-79df68db65-ql9cm\" (UID: \"7b7b4258-9a57-4bb1-820e-2b087abefcac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-30f36b-79df68db65-ql9cm" Jul 08 19:15:02.983861 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:02.983350 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqgws\" (UniqueName: \"kubernetes.io/projected/7b7b4258-9a57-4bb1-820e-2b087abefcac-kube-api-access-wqgws\") pod \"maas-api-e2e-api-a-30f36b-79df68db65-ql9cm\" (UID: \"7b7b4258-9a57-4bb1-820e-2b087abefcac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-30f36b-79df68db65-ql9cm" Jul 08 19:15:02.983861 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:02.983396 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b7b4258-9a57-4bb1-820e-2b087abefcac-maas-api-tls\") pod \"maas-api-e2e-api-a-30f36b-79df68db65-ql9cm\" (UID: \"7b7b4258-9a57-4bb1-820e-2b087abefcac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-30f36b-79df68db65-ql9cm" Jul 08 19:15:02.983861 ip-10-0-143-154 kubenswrapper[2559]: E0708 19:15:02.983506 2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-30f36b-serving-cert: secret "maas-api-e2e-api-a-30f36b-serving-cert" not found Jul 08 19:15:02.983861 ip-10-0-143-154 kubenswrapper[2559]: E0708 19:15:02.983566 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b7b4258-9a57-4bb1-820e-2b087abefcac-maas-api-tls podName:7b7b4258-9a57-4bb1-820e-2b087abefcac nodeName:}" failed. No retries permitted until 2026-07-08 19:15:03.483546643 +0000 UTC m=+2146.719844712 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7b7b4258-9a57-4bb1-820e-2b087abefcac-maas-api-tls") pod "maas-api-e2e-api-a-30f36b-79df68db65-ql9cm" (UID: "7b7b4258-9a57-4bb1-820e-2b087abefcac") : secret "maas-api-e2e-api-a-30f36b-serving-cert" not found Jul 08 19:15:02.984181 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:02.984153 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b7b4258-9a57-4bb1-820e-2b087abefcac-openshift-service-ca\") pod \"maas-api-e2e-api-a-30f36b-79df68db65-ql9cm\" (UID: \"7b7b4258-9a57-4bb1-820e-2b087abefcac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-30f36b-79df68db65-ql9cm" Jul 08 19:15:02.994093 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:02.994066 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqgws\" (UniqueName: \"kubernetes.io/projected/7b7b4258-9a57-4bb1-820e-2b087abefcac-kube-api-access-wqgws\") pod \"maas-api-e2e-api-a-30f36b-79df68db65-ql9cm\" (UID: \"7b7b4258-9a57-4bb1-820e-2b087abefcac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-30f36b-79df68db65-ql9cm" Jul 08 19:15:03.489724 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:03.489689 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b7b4258-9a57-4bb1-820e-2b087abefcac-maas-api-tls\") pod \"maas-api-e2e-api-a-30f36b-79df68db65-ql9cm\" (UID: \"7b7b4258-9a57-4bb1-820e-2b087abefcac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-30f36b-79df68db65-ql9cm" Jul 08 19:15:03.492170 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:03.492146 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b7b4258-9a57-4bb1-820e-2b087abefcac-maas-api-tls\") pod \"maas-api-e2e-api-a-30f36b-79df68db65-ql9cm\" (UID: \"7b7b4258-9a57-4bb1-820e-2b087abefcac\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-30f36b-79df68db65-ql9cm" Jul 08 19:15:03.708307 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:03.708271 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-30f36b-79df68db65-ql9cm" Jul 08 19:15:03.854077 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:03.854048 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-30f36b-79df68db65-ql9cm"] Jul 08 19:15:03.856325 ip-10-0-143-154 kubenswrapper[2559]: W0708 19:15:03.856301 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b7b4258_9a57_4bb1_820e_2b087abefcac.slice/crio-1beebf50a746541fe07d092e2e3c14419057d3f3ad8566f79fab149b841b6b5f WatchSource:0}: Error finding container 1beebf50a746541fe07d092e2e3c14419057d3f3ad8566f79fab149b841b6b5f: Status 404 returned error can't find the container with id 1beebf50a746541fe07d092e2e3c14419057d3f3ad8566f79fab149b841b6b5f Jul 08 19:15:03.963304 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:03.963271 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-30f36b-79df68db65-ql9cm" event={"ID":"7b7b4258-9a57-4bb1-820e-2b087abefcac","Type":"ContainerStarted","Data":"1beebf50a746541fe07d092e2e3c14419057d3f3ad8566f79fab149b841b6b5f"} Jul 08 19:15:04.970664 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:04.970631 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-30f36b-79df68db65-ql9cm" event={"ID":"7b7b4258-9a57-4bb1-820e-2b087abefcac","Type":"ContainerStarted","Data":"22458eb6a0b1c0ededfc883806ff3c6cab60cc77e36239700eac356dd59a79d5"} Jul 08 19:15:04.971098 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:04.970725 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-30f36b-79df68db65-ql9cm" Jul 08 19:15:04.991396 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:04.991350 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-30f36b-79df68db65-ql9cm" podStartSLOduration=2.4316159170000002 podStartE2EDuration="2.991335301s" totalImagesPullingTime="559.719384ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:15:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:15:04.988460939 +0000 UTC m=+2148.224759022" watchObservedRunningTime="2026-07-08 19:15:04.991335301 +0000 UTC m=+2148.227633392" Jul 08 19:15:10.986603 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:10.986502 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-30f36b-79df68db65-ql9cm" Jul 08 19:15:24.604900 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:24.604865 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-30f36b-79df68db65-ql9cm"] Jul 08 19:15:24.605538 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:24.605104 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-30f36b-79df68db65-ql9cm" podUID="7b7b4258-9a57-4bb1-820e-2b087abefcac" containerName="maas-api" containerID="cri-o://22458eb6a0b1c0ededfc883806ff3c6cab60cc77e36239700eac356dd59a79d5" gracePeriod=30 Jul 08 19:15:24.862410 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:24.862346 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-30f36b-79df68db65-ql9cm" Jul 08 19:15:24.898118 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:24.898086 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqgws\" (UniqueName: \"kubernetes.io/projected/7b7b4258-9a57-4bb1-820e-2b087abefcac-kube-api-access-wqgws\") pod \"7b7b4258-9a57-4bb1-820e-2b087abefcac\" (UID: \"7b7b4258-9a57-4bb1-820e-2b087abefcac\") " Jul 08 19:15:24.898327 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:24.898157 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b7b4258-9a57-4bb1-820e-2b087abefcac-maas-api-tls\") pod \"7b7b4258-9a57-4bb1-820e-2b087abefcac\" (UID: \"7b7b4258-9a57-4bb1-820e-2b087abefcac\") " Jul 08 19:15:24.898327 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:24.898229 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b7b4258-9a57-4bb1-820e-2b087abefcac-openshift-service-ca\") pod \"7b7b4258-9a57-4bb1-820e-2b087abefcac\" (UID: \"7b7b4258-9a57-4bb1-820e-2b087abefcac\") " Jul 08 19:15:24.898579 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:24.898541 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b7b4258-9a57-4bb1-820e-2b087abefcac-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7b7b4258-9a57-4bb1-820e-2b087abefcac" (UID: "7b7b4258-9a57-4bb1-820e-2b087abefcac"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:15:24.900285 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:24.900263 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b7b4258-9a57-4bb1-820e-2b087abefcac-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7b7b4258-9a57-4bb1-820e-2b087abefcac" (UID: "7b7b4258-9a57-4bb1-820e-2b087abefcac"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:15:24.900394 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:24.900378 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b7b4258-9a57-4bb1-820e-2b087abefcac-kube-api-access-wqgws" (OuterVolumeSpecName: "kube-api-access-wqgws") pod "7b7b4258-9a57-4bb1-820e-2b087abefcac" (UID: "7b7b4258-9a57-4bb1-820e-2b087abefcac"). InnerVolumeSpecName "kube-api-access-wqgws". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:15:24.999953 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:24.999916 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqgws\" (UniqueName: \"kubernetes.io/projected/7b7b4258-9a57-4bb1-820e-2b087abefcac-kube-api-access-wqgws\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 19:15:24.999953 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:24.999948 2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b7b4258-9a57-4bb1-820e-2b087abefcac-maas-api-tls\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 19:15:24.999953 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:24.999957 2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b7b4258-9a57-4bb1-820e-2b087abefcac-openshift-service-ca\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 19:15:25.095870 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:25.095833 2559 generic.go:356] "Generic (PLEG): container finished" podID="7b7b4258-9a57-4bb1-820e-2b087abefcac" containerID="22458eb6a0b1c0ededfc883806ff3c6cab60cc77e36239700eac356dd59a79d5" exitCode=0 Jul 08 19:15:25.096040 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:25.095881 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-30f36b-79df68db65-ql9cm" Jul 08 19:15:25.096040 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:25.095924 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-30f36b-79df68db65-ql9cm" event={"ID":"7b7b4258-9a57-4bb1-820e-2b087abefcac","Type":"ContainerDied","Data":"22458eb6a0b1c0ededfc883806ff3c6cab60cc77e36239700eac356dd59a79d5"} Jul 08 19:15:25.096040 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:25.095999 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-30f36b-79df68db65-ql9cm" event={"ID":"7b7b4258-9a57-4bb1-820e-2b087abefcac","Type":"ContainerDied","Data":"1beebf50a746541fe07d092e2e3c14419057d3f3ad8566f79fab149b841b6b5f"} Jul 08 19:15:25.096040 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:25.096022 2559 scope.go:117] "RemoveContainer" containerID="22458eb6a0b1c0ededfc883806ff3c6cab60cc77e36239700eac356dd59a79d5" Jul 08 19:15:25.112686 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:25.112547 2559 scope.go:117] "RemoveContainer" containerID="22458eb6a0b1c0ededfc883806ff3c6cab60cc77e36239700eac356dd59a79d5" Jul 08 19:15:25.113072 ip-10-0-143-154 kubenswrapper[2559]: E0708 19:15:25.113044 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22458eb6a0b1c0ededfc883806ff3c6cab60cc77e36239700eac356dd59a79d5\": container with ID starting with 22458eb6a0b1c0ededfc883806ff3c6cab60cc77e36239700eac356dd59a79d5 not found: ID does not exist" containerID="22458eb6a0b1c0ededfc883806ff3c6cab60cc77e36239700eac356dd59a79d5" Jul 08 19:15:25.113163 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:25.113085 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22458eb6a0b1c0ededfc883806ff3c6cab60cc77e36239700eac356dd59a79d5"} err="failed to get container status \"22458eb6a0b1c0ededfc883806ff3c6cab60cc77e36239700eac356dd59a79d5\": rpc error: code = NotFound desc = could not find container \"22458eb6a0b1c0ededfc883806ff3c6cab60cc77e36239700eac356dd59a79d5\": container with ID starting with 22458eb6a0b1c0ededfc883806ff3c6cab60cc77e36239700eac356dd59a79d5 not found: ID does not exist" Jul 08 19:15:25.130087 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:25.130050 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-30f36b-79df68db65-ql9cm"] Jul 08 19:15:25.132437 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:25.132414 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-30f36b-79df68db65-ql9cm"] Jul 08 19:15:25.688587 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:15:25.688551 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b7b4258-9a57-4bb1-820e-2b087abefcac" path="/var/lib/kubelet/pods/7b7b4258-9a57-4bb1-820e-2b087abefcac/volumes" Jul 08 19:16:00.026322 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:16:00.026281 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725635-9929s"] Jul 08 19:16:00.026926 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:16:00.026539 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725635-9929s" podUID="55664b9e-537b-416d-ac85-79899d694d41" containerName="cleanup" containerID="cri-o://b0027ee4e9601f3411fa3bcef85c0e0f14fd280030e90529a2d780e5553d8213" gracePeriod=30 Jul 08 19:16:30.224537 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:16:30.224507 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725635-9929s" Jul 08 19:16:30.344678 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:16:30.344583 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-62j8b\" (UniqueName: \"kubernetes.io/projected/55664b9e-537b-416d-ac85-79899d694d41-kube-api-access-62j8b\") pod \"55664b9e-537b-416d-ac85-79899d694d41\" (UID: \"55664b9e-537b-416d-ac85-79899d694d41\") " Jul 08 19:16:30.346694 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:16:30.346669 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55664b9e-537b-416d-ac85-79899d694d41-kube-api-access-62j8b" (OuterVolumeSpecName: "kube-api-access-62j8b") pod "55664b9e-537b-416d-ac85-79899d694d41" (UID: "55664b9e-537b-416d-ac85-79899d694d41"). InnerVolumeSpecName "kube-api-access-62j8b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:16:30.446075 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:16:30.446043 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-62j8b\" (UniqueName: \"kubernetes.io/projected/55664b9e-537b-416d-ac85-79899d694d41-kube-api-access-62j8b\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\"" Jul 08 19:16:30.486338 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:16:30.486303 2559 generic.go:356] "Generic (PLEG): container finished" podID="55664b9e-537b-416d-ac85-79899d694d41" containerID="b0027ee4e9601f3411fa3bcef85c0e0f14fd280030e90529a2d780e5553d8213" exitCode=137 Jul 08 19:16:30.486510 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:16:30.486346 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725635-9929s" Jul 08 19:16:30.486510 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:16:30.486388 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725635-9929s" event={"ID":"55664b9e-537b-416d-ac85-79899d694d41","Type":"ContainerDied","Data":"b0027ee4e9601f3411fa3bcef85c0e0f14fd280030e90529a2d780e5553d8213"} Jul 08 19:16:30.486510 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:16:30.486450 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725635-9929s" event={"ID":"55664b9e-537b-416d-ac85-79899d694d41","Type":"ContainerDied","Data":"1105fe73b13ab9d331f12eeb04db17263c77c768a45129bf8936239ee7959e00"} Jul 08 19:16:30.486510 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:16:30.486475 2559 scope.go:117] "RemoveContainer" containerID="b0027ee4e9601f3411fa3bcef85c0e0f14fd280030e90529a2d780e5553d8213" Jul 08 19:16:30.496215 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:16:30.496200 2559 scope.go:117] "RemoveContainer" containerID="b0027ee4e9601f3411fa3bcef85c0e0f14fd280030e90529a2d780e5553d8213" Jul 08 19:16:30.496453 ip-10-0-143-154 kubenswrapper[2559]: E0708 19:16:30.496435 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0027ee4e9601f3411fa3bcef85c0e0f14fd280030e90529a2d780e5553d8213\": container with ID starting with b0027ee4e9601f3411fa3bcef85c0e0f14fd280030e90529a2d780e5553d8213 not found: ID does not exist" containerID="b0027ee4e9601f3411fa3bcef85c0e0f14fd280030e90529a2d780e5553d8213" Jul 08 19:16:30.496505 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:16:30.496462 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0027ee4e9601f3411fa3bcef85c0e0f14fd280030e90529a2d780e5553d8213"} err="failed to get container status \"b0027ee4e9601f3411fa3bcef85c0e0f14fd280030e90529a2d780e5553d8213\": rpc error: code = NotFound desc = could not find container \"b0027ee4e9601f3411fa3bcef85c0e0f14fd280030e90529a2d780e5553d8213\": container with ID starting with b0027ee4e9601f3411fa3bcef85c0e0f14fd280030e90529a2d780e5553d8213 not found: ID does not exist" Jul 08 19:16:30.509100 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:16:30.509072 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725635-9929s"] Jul 08 19:16:30.513188 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:16:30.513166 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725635-9929s"] Jul 08 19:16:31.688305 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:16:31.688263 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55664b9e-537b-416d-ac85-79899d694d41" path="/var/lib/kubelet/pods/55664b9e-537b-416d-ac85-79899d694d41/volumes" Jul 08 19:19:08.516691 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:08.516659 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5578968cc4-srdrd_8427523c-36f1-464f-888a-a8d3fade1d2a/authorino/0.log" Jul 08 19:19:13.589176 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:13.589150 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-g7b8j_83674853-108f-4426-924b-38f003c0949a/manager/0.log" Jul 08 19:19:13.721852 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:13.721819 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-589jt_b1814dc5-9140-480f-9eb8-de0de2a37225/manager/0.log" Jul 08 19:19:13.996134 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:13.996106 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-56m64_393da8af-a671-4e2c-8ba2-da604d82cdd3/server/0.log" Jul 08 19:19:14.387627 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:14.387598 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c5cf545b9-n999r_d57edb35-475a-478b-b938-1b9c93165c16/copy-manifests/0.log" Jul 08 19:19:14.400128 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:14.400099 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c5cf545b9-n999r_d57edb35-475a-478b-b938-1b9c93165c16/manager/0.log" Jul 08 19:19:15.422099 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:15.422070 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759phldm_048eec7c-f6e5-4f32-9085-6d1c85683bd4/util/0.log" Jul 08 19:19:15.429313 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:15.429285 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759phldm_048eec7c-f6e5-4f32-9085-6d1c85683bd4/pull/0.log" Jul 08 19:19:15.435680 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:15.435657 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759phldm_048eec7c-f6e5-4f32-9085-6d1c85683bd4/extract/0.log" Jul 08 19:19:15.567885 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:15.567847 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vlh9b_e75d9e1a-840c-417a-a19f-e7eb51aa6d37/util/0.log" Jul 08 19:19:15.574297 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:15.574275 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vlh9b_e75d9e1a-840c-417a-a19f-e7eb51aa6d37/pull/0.log" Jul 08 19:19:15.580848 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:15.580823 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vlh9b_e75d9e1a-840c-417a-a19f-e7eb51aa6d37/extract/0.log" Jul 08 19:19:15.712745 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:15.712664 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vlvcm_6fe8b5ae-c089-4636-a79e-f2a4241f443f/util/0.log" Jul 08 19:19:15.718551 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:15.718527 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vlvcm_6fe8b5ae-c089-4636-a79e-f2a4241f443f/pull/0.log" Jul 08 19:19:15.724733 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:15.724712 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vlvcm_6fe8b5ae-c089-4636-a79e-f2a4241f443f/extract/0.log" Jul 08 19:19:15.854065 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:15.854030 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f8tt8_34fb2988-24da-4856-89ff-161e7ed86c63/pull/0.log" Jul 08 19:19:15.861907 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:15.861866 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f8tt8_34fb2988-24da-4856-89ff-161e7ed86c63/extract/0.log" Jul 08 19:19:15.870200 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:15.870175 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f8tt8_34fb2988-24da-4856-89ff-161e7ed86c63/util/0.log" Jul 08 19:19:16.004551 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:16.004470 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5578968cc4-srdrd_8427523c-36f1-464f-888a-a8d3fade1d2a/authorino/0.log" Jul 08 19:19:16.714795 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:16.714746 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-8h5ns_05d9a127-a2e3-4f54-a423-dc30f5144ee7/manager/0.log" Jul 08 19:19:17.381606 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:17.381578 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb6782447k_aa13ab45-55ef-49b7-bd0c-614c3d561a86/istio-proxy/0.log" Jul 08 19:19:17.467231 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:17.467204 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zzt4h_d94a3566-fcf3-47a1-b176-a33b71c3826d/console-operator/1.log" Jul 08 19:19:17.473871 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:17.473848 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rj4xk_56014c3a-384a-4775-96e5-290968b196b1/ovn-acl-logging/0.log" Jul 08 19:19:17.930039 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:17.930014 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zzt4h_d94a3566-fcf3-47a1-b176-a33b71c3826d/console-operator/1.log" Jul 08 19:19:17.936688 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:17.936667 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rj4xk_56014c3a-384a-4775-96e5-290968b196b1/ovn-acl-logging/0.log" Jul 08 19:19:18.822598 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:18.822565 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-6fj55_70becc73-a4a0-428c-9d76-2138b4e726e2/main/0.log" Jul 08 19:19:19.073134 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:19.073049 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6sfm8_600f4dca-4fb4-40b1-b2af-111478b50316/main/0.log" Jul 08 19:19:26.461137 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:26.461100 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-l9l9x_a0618904-8d08-404a-9a82-a0bb55351381/global-pull-secret-syncer/0.log" Jul 08 19:19:26.615813 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:26.615781 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-qklqp_d155f1e7-7109-445e-b884-7a3e49df96f8/konnectivity-agent/0.log" Jul 08 19:19:26.709395 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:26.709359 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-154.ec2.internal_3872d75f39f988d68af7ca611d18fcd5/haproxy/0.log" Jul 08 19:19:30.657985 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:30.657946 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759phldm_048eec7c-f6e5-4f32-9085-6d1c85683bd4/extract/0.log" Jul 08 19:19:30.712909 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:30.712872 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759phldm_048eec7c-f6e5-4f32-9085-6d1c85683bd4/util/0.log" Jul 08 19:19:30.739477 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:30.739448 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759phldm_048eec7c-f6e5-4f32-9085-6d1c85683bd4/pull/0.log" Jul 08 19:19:30.785317 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:30.785288 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vlh9b_e75d9e1a-840c-417a-a19f-e7eb51aa6d37/extract/0.log" Jul 08 19:19:30.818216 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:30.818185 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vlh9b_e75d9e1a-840c-417a-a19f-e7eb51aa6d37/util/0.log" Jul 08 19:19:30.842059 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:30.842033 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vlh9b_e75d9e1a-840c-417a-a19f-e7eb51aa6d37/pull/0.log" Jul 08 19:19:30.871892 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:30.871865 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vlvcm_6fe8b5ae-c089-4636-a79e-f2a4241f443f/extract/0.log" Jul 08 19:19:30.898738 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:30.898710 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vlvcm_6fe8b5ae-c089-4636-a79e-f2a4241f443f/util/0.log" Jul 08 19:19:30.924402 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:30.924336 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vlvcm_6fe8b5ae-c089-4636-a79e-f2a4241f443f/pull/0.log" Jul 08 19:19:30.959231 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:30.959210 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f8tt8_34fb2988-24da-4856-89ff-161e7ed86c63/extract/0.log" Jul 08 19:19:30.991165 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:30.991126 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f8tt8_34fb2988-24da-4856-89ff-161e7ed86c63/util/0.log" Jul 08 19:19:31.017281 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:31.017251 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f8tt8_34fb2988-24da-4856-89ff-161e7ed86c63/pull/0.log" Jul 08 19:19:31.281452 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:31.281357 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5578968cc4-srdrd_8427523c-36f1-464f-888a-a8d3fade1d2a/authorino/0.log" Jul 08 19:19:31.522941 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:31.522905 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-8h5ns_05d9a127-a2e3-4f54-a423-dc30f5144ee7/manager/0.log" Jul 08 19:19:33.192585 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:33.192555 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d/alertmanager/0.log" Jul 08 19:19:33.213857 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:33.213831 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d/config-reloader/0.log" Jul 08 19:19:33.235891 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:33.235867 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d/kube-rbac-proxy-web/0.log" Jul 08 19:19:33.270123 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:33.270097 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d/kube-rbac-proxy/0.log" Jul 08 19:19:33.293565 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:33.293541 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d/kube-rbac-proxy-metric/0.log" Jul 08 19:19:33.323842 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:33.323794 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d/prom-label-proxy/0.log" Jul 08 19:19:33.341347 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:33.341312 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ff2f3e01-aca5-4ca2-8f4b-addd13c59e8d/init-config-reloader/0.log" Jul 08 19:19:33.440023 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:33.439990 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-fc9r7_437edb92-3ae1-4a09-8f2f-cc1d583a6753/kube-state-metrics/0.log" Jul 08 19:19:33.469969 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:33.469881 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-fc9r7_437edb92-3ae1-4a09-8f2f-cc1d583a6753/kube-rbac-proxy-main/0.log" Jul 08 19:19:33.496812 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:33.496763 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-fc9r7_437edb92-3ae1-4a09-8f2f-cc1d583a6753/kube-rbac-proxy-self/0.log" Jul 08 19:19:33.534737 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:33.534706 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6dcfd8bc94-jlv5w_b4d384fb-fe74-4395-bc16-cffdc9cf9789/metrics-server/0.log" Jul 08 19:19:33.866701 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:33.866618 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nzczn_1ee283a3-c9af-4aea-ab7e-448d83b4df89/node-exporter/0.log" Jul 08 19:19:33.889033 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:33.889006 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nzczn_1ee283a3-c9af-4aea-ab7e-448d83b4df89/kube-rbac-proxy/0.log" Jul 08 19:19:33.914363 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:33.914338 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nzczn_1ee283a3-c9af-4aea-ab7e-448d83b4df89/init-textfile/0.log" Jul 08 19:19:33.941732 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:33.941702 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-jlr4r_ce91fd5d-2c4c-4762-88b3-2d3cef665933/kube-rbac-proxy-main/0.log" Jul 08 19:19:33.962724 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:33.962702 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-jlr4r_ce91fd5d-2c4c-4762-88b3-2d3cef665933/kube-rbac-proxy-self/0.log" Jul 08 19:19:33.990296 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:33.990270 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-jlr4r_ce91fd5d-2c4c-4762-88b3-2d3cef665933/openshift-state-metrics/0.log" Jul 08 19:19:34.260446 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:34.260418 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-d58rk_40364492-d5f5-4bfb-a67a-3beec834dc23/prometheus-operator/0.log" Jul 08 19:19:34.279940 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:34.279911 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-d58rk_40364492-d5f5-4bfb-a67a-3beec834dc23/kube-rbac-proxy/0.log" Jul 08 19:19:34.304403 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:34.304374 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-4cdpm_6c5ec41b-bde9-47f9-963e-c454166a447e/prometheus-operator-admission-webhook/0.log" Jul 08 19:19:34.340171 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:34.340142 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-65bf6dc8b7-wwzgz_0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e/telemeter-client/0.log" Jul 08 19:19:34.365387 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:34.365362 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-65bf6dc8b7-wwzgz_0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e/reload/0.log" Jul 08 19:19:34.404300 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:34.404267 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-65bf6dc8b7-wwzgz_0d91d2f4-2a9f-4017-8cec-c02f8c9fcb5e/kube-rbac-proxy/0.log" Jul 08 19:19:34.448698 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:34.448663 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-579fd557b5-qzlxn_357c8e19-72d5-48e2-bcb6-7f222b54b0e6/thanos-query/0.log" Jul 08 19:19:34.491842 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:34.491817 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-579fd557b5-qzlxn_357c8e19-72d5-48e2-bcb6-7f222b54b0e6/kube-rbac-proxy-web/0.log" Jul 08 19:19:34.557574 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:34.557499 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-579fd557b5-qzlxn_357c8e19-72d5-48e2-bcb6-7f222b54b0e6/kube-rbac-proxy/0.log" Jul 08 19:19:34.593572 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:34.593547 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-579fd557b5-qzlxn_357c8e19-72d5-48e2-bcb6-7f222b54b0e6/prom-label-proxy/0.log" Jul 08 19:19:34.616333 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:34.616308 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-579fd557b5-qzlxn_357c8e19-72d5-48e2-bcb6-7f222b54b0e6/kube-rbac-proxy-rules/0.log" Jul 08 19:19:34.638175 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:34.638147 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-579fd557b5-qzlxn_357c8e19-72d5-48e2-bcb6-7f222b54b0e6/kube-rbac-proxy-metrics/0.log" Jul 08 19:19:36.532408 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:36.532377 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zzt4h_d94a3566-fcf3-47a1-b176-a33b71c3826d/console-operator/1.log" Jul 08 19:19:36.537198 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:36.537176 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zzt4h_d94a3566-fcf3-47a1-b176-a33b71c3826d/console-operator/2.log" Jul 08 19:19:36.715077 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:36.715044 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-prwc8/perf-node-gather-daemonset-zvxb8"] Jul 08 19:19:36.715479 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:36.715466 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b7b4258-9a57-4bb1-820e-2b087abefcac" containerName="maas-api" Jul 08 19:19:36.715526 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:36.715481 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b7b4258-9a57-4bb1-820e-2b087abefcac" containerName="maas-api" Jul 08 19:19:36.715526 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:36.715492 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55664b9e-537b-416d-ac85-79899d694d41" containerName="cleanup" Jul 08 19:19:36.715526 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:36.715498 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="55664b9e-537b-416d-ac85-79899d694d41" containerName="cleanup" Jul 08 19:19:36.715619 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:36.715576 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="55664b9e-537b-416d-ac85-79899d694d41" containerName="cleanup" Jul 08 19:19:36.715619 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:36.715586 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b7b4258-9a57-4bb1-820e-2b087abefcac" containerName="maas-api" Jul 08 19:19:36.718986 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:36.718964 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-prwc8/perf-node-gather-daemonset-zvxb8" Jul 08 19:19:36.721817 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:36.721786 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkn2n\" (UniqueName: \"kubernetes.io/projected/9f09cc29-bb40-42f0-bbfc-84bdcad89627-kube-api-access-jkn2n\") pod \"perf-node-gather-daemonset-zvxb8\" (UID: \"9f09cc29-bb40-42f0-bbfc-84bdcad89627\") " pod="openshift-must-gather-prwc8/perf-node-gather-daemonset-zvxb8" Jul 08 19:19:36.721946 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:36.721845 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9f09cc29-bb40-42f0-bbfc-84bdcad89627-lib-modules\") pod \"perf-node-gather-daemonset-zvxb8\" (UID: \"9f09cc29-bb40-42f0-bbfc-84bdcad89627\") " pod="openshift-must-gather-prwc8/perf-node-gather-daemonset-zvxb8" Jul 08 19:19:36.721946 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:36.721880 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9f09cc29-bb40-42f0-bbfc-84bdcad89627-proc\") pod \"perf-node-gather-daemonset-zvxb8\" (UID: \"9f09cc29-bb40-42f0-bbfc-84bdcad89627\") " pod="openshift-must-gather-prwc8/perf-node-gather-daemonset-zvxb8" Jul 08 19:19:36.721946 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:36.721903 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-prwc8\"/\"kube-root-ca.crt\"" Jul 08 19:19:36.721946 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:36.721932 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9f09cc29-bb40-42f0-bbfc-84bdcad89627-sys\") pod \"perf-node-gather-daemonset-zvxb8\" (UID: \"9f09cc29-bb40-42f0-bbfc-84bdcad89627\") " pod="openshift-must-gather-prwc8/perf-node-gather-daemonset-zvxb8" Jul 08 19:19:36.722194 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:36.722015 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9f09cc29-bb40-42f0-bbfc-84bdcad89627-podres\") pod \"perf-node-gather-daemonset-zvxb8\" (UID: \"9f09cc29-bb40-42f0-bbfc-84bdcad89627\") " pod="openshift-must-gather-prwc8/perf-node-gather-daemonset-zvxb8" Jul 08 19:19:36.723039 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:36.723010 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-prwc8\"/\"default-dockercfg-q5xj4\"" Jul 08 19:19:36.723152 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:36.723061 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-prwc8\"/\"openshift-service-ca.crt\"" Jul 08 19:19:36.729504 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:36.729478 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-prwc8/perf-node-gather-daemonset-zvxb8"] Jul 08 19:19:36.822578 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:36.822491 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9f09cc29-bb40-42f0-bbfc-84bdcad89627-podres\") pod \"perf-node-gather-daemonset-zvxb8\" (UID: \"9f09cc29-bb40-42f0-bbfc-84bdcad89627\") " pod="openshift-must-gather-prwc8/perf-node-gather-daemonset-zvxb8" Jul 08 19:19:36.822578 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:36.822538 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkn2n\" (UniqueName: \"kubernetes.io/projected/9f09cc29-bb40-42f0-bbfc-84bdcad89627-kube-api-access-jkn2n\") pod \"perf-node-gather-daemonset-zvxb8\" (UID: \"9f09cc29-bb40-42f0-bbfc-84bdcad89627\") " pod="openshift-must-gather-prwc8/perf-node-gather-daemonset-zvxb8" Jul 08 19:19:36.822578 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:36.822573 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9f09cc29-bb40-42f0-bbfc-84bdcad89627-lib-modules\") pod \"perf-node-gather-daemonset-zvxb8\" (UID: \"9f09cc29-bb40-42f0-bbfc-84bdcad89627\") " pod="openshift-must-gather-prwc8/perf-node-gather-daemonset-zvxb8" Jul 08 19:19:36.822883 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:36.822590 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9f09cc29-bb40-42f0-bbfc-84bdcad89627-proc\") pod \"perf-node-gather-daemonset-zvxb8\" (UID: \"9f09cc29-bb40-42f0-bbfc-84bdcad89627\") " pod="openshift-must-gather-prwc8/perf-node-gather-daemonset-zvxb8" Jul 08 19:19:36.822883 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:36.822622 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9f09cc29-bb40-42f0-bbfc-84bdcad89627-sys\") pod \"perf-node-gather-daemonset-zvxb8\" (UID: \"9f09cc29-bb40-42f0-bbfc-84bdcad89627\") " pod="openshift-must-gather-prwc8/perf-node-gather-daemonset-zvxb8" Jul 08 19:19:36.822883 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:36.822701 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9f09cc29-bb40-42f0-bbfc-84bdcad89627-podres\") pod \"perf-node-gather-daemonset-zvxb8\" (UID: \"9f09cc29-bb40-42f0-bbfc-84bdcad89627\") " pod="openshift-must-gather-prwc8/perf-node-gather-daemonset-zvxb8" Jul 08 19:19:36.822883 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:36.822719 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9f09cc29-bb40-42f0-bbfc-84bdcad89627-proc\") pod \"perf-node-gather-daemonset-zvxb8\" (UID: \"9f09cc29-bb40-42f0-bbfc-84bdcad89627\") " pod="openshift-must-gather-prwc8/perf-node-gather-daemonset-zvxb8" Jul 08 19:19:36.822883 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:36.822720 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9f09cc29-bb40-42f0-bbfc-84bdcad89627-sys\") pod \"perf-node-gather-daemonset-zvxb8\" (UID: \"9f09cc29-bb40-42f0-bbfc-84bdcad89627\") " pod="openshift-must-gather-prwc8/perf-node-gather-daemonset-zvxb8" Jul 08 19:19:36.822883 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:36.822742 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9f09cc29-bb40-42f0-bbfc-84bdcad89627-lib-modules\") pod \"perf-node-gather-daemonset-zvxb8\" (UID: \"9f09cc29-bb40-42f0-bbfc-84bdcad89627\") " pod="openshift-must-gather-prwc8/perf-node-gather-daemonset-zvxb8" Jul 08 19:19:36.831320 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:36.831289 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkn2n\" (UniqueName: \"kubernetes.io/projected/9f09cc29-bb40-42f0-bbfc-84bdcad89627-kube-api-access-jkn2n\") pod \"perf-node-gather-daemonset-zvxb8\" (UID: \"9f09cc29-bb40-42f0-bbfc-84bdcad89627\") " pod="openshift-must-gather-prwc8/perf-node-gather-daemonset-zvxb8" Jul 08 19:19:37.031027 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:37.030988 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-prwc8/perf-node-gather-daemonset-zvxb8" Jul 08 19:19:37.109182 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:37.109155 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-56984b4675-zpj7r_42bceb24-9f7a-4d9f-8368-510f67655caf/console/0.log" Jul 08 19:19:37.148010 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:37.147977 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-7dwmw_59decd4d-8b27-4cc1-bb59-a9c9dc29f8e3/download-server/0.log" Jul 08 19:19:37.182183 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:37.182160 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-prwc8/perf-node-gather-daemonset-zvxb8"] Jul 08 19:19:37.184190 ip-10-0-143-154 kubenswrapper[2559]: W0708 19:19:37.184163 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9f09cc29_bb40_42f0_bbfc_84bdcad89627.slice/crio-04a7c06520a2a0ca717af04e0e7cabe82e9b0b28101474dd3ae1ca77438e79b5 WatchSource:0}: Error finding container 04a7c06520a2a0ca717af04e0e7cabe82e9b0b28101474dd3ae1ca77438e79b5: Status 404 returned error can't find the container with id 04a7c06520a2a0ca717af04e0e7cabe82e9b0b28101474dd3ae1ca77438e79b5 Jul 08 19:19:37.589679 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:37.589643 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-prwc8/perf-node-gather-daemonset-zvxb8" event={"ID":"9f09cc29-bb40-42f0-bbfc-84bdcad89627","Type":"ContainerStarted","Data":"aadc33918a491a702f9174134abfb0088cf33b3528b999fdf29cbf7c92c22bf2"} Jul 08 19:19:37.590107 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:37.589721 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-prwc8/perf-node-gather-daemonset-zvxb8" event={"ID":"9f09cc29-bb40-42f0-bbfc-84bdcad89627","Type":"ContainerStarted","Data":"04a7c06520a2a0ca717af04e0e7cabe82e9b0b28101474dd3ae1ca77438e79b5"} Jul 08 19:19:37.590107 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:37.589794 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-prwc8/perf-node-gather-daemonset-zvxb8" Jul 08 19:19:37.606656 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:37.606607 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-prwc8/perf-node-gather-daemonset-zvxb8" podStartSLOduration=1.606590988 podStartE2EDuration="1.606590988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:19:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:19:37.605628974 +0000 UTC m=+2420.841927063" watchObservedRunningTime="2026-07-08 19:19:37.606590988 +0000 UTC m=+2420.842889080" Jul 08 19:19:38.868699 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:38.868669 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rdd7l_71020478-23b3-4f75-a526-ed0da0f0ad8f/dns/0.log" Jul 08 19:19:38.888476 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:38.888453 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rdd7l_71020478-23b3-4f75-a526-ed0da0f0ad8f/kube-rbac-proxy/0.log" Jul 08 19:19:38.974022 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:38.973991 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-x72mc_32185124-ea41-4309-9674-13230b4d7a6e/dns-node-resolver/0.log" Jul 08 19:19:39.586273 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:39.586245 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-q9hp4_ceb0a0a0-5ba1-4906-b214-5b1de0abbec3/node-ca/0.log" Jul 08 19:19:40.545666 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:40.545633 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb6782447k_aa13ab45-55ef-49b7-bd0c-614c3d561a86/istio-proxy/0.log" Jul 08 19:19:41.552303 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:41.552276 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9qx6k_cc461fee-bcf4-4a16-92ec-50211c9314c6/serve-healthcheck-canary/0.log" Jul 08 19:19:42.046549 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:42.046512 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-g989c_61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e/insights-operator/0.log" Jul 08 19:19:42.048450 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:42.048424 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-g989c_61cc5a69-f1a7-4a3a-8b33-427c95bd4a9e/insights-operator/1.log" Jul 08 19:19:42.068017 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:42.067991 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-56mxw_37037696-fffa-4456-aacb-6bc63f69254d/kube-rbac-proxy/0.log" Jul 08 19:19:42.088607 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:42.088578 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-56mxw_37037696-fffa-4456-aacb-6bc63f69254d/exporter/0.log" Jul 08 19:19:42.108762 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:42.108740 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-56mxw_37037696-fffa-4456-aacb-6bc63f69254d/extractor/0.log" Jul 08 19:19:43.605679 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:43.605650 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-prwc8/perf-node-gather-daemonset-zvxb8" Jul 08 19:19:44.314975 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:44.314948 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-g7b8j_83674853-108f-4426-924b-38f003c0949a/manager/0.log" Jul 08 19:19:44.420952 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:44.420915 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-589jt_b1814dc5-9140-480f-9eb8-de0de2a37225/manager/0.log" Jul 08 19:19:44.519833 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:44.519804 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-56m64_393da8af-a671-4e2c-8ba2-da604d82cdd3/server/0.log" Jul 08 19:19:44.700425 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:44.700390 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c5cf545b9-n999r_d57edb35-475a-478b-b938-1b9c93165c16/manager/0.log" Jul 08 19:19:44.717419 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:44.717392 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c5cf545b9-n999r_d57edb35-475a-478b-b938-1b9c93165c16/copy-manifests/0.log" Jul 08 19:19:46.129970 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:46.129939 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-58c5974cc6-cc72p_078cff9b-7f14-4518-8c61-1551c7355f81/manager/0.log" Jul 08 19:19:46.785206 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:46.785115 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb6782447k_aa13ab45-55ef-49b7-bd0c-614c3d561a86/istio-proxy/0.log" Jul 08 19:19:48.289967 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:48.289875 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb6782447k_aa13ab45-55ef-49b7-bd0c-614c3d561a86/istio-proxy/0.log" Jul 08 19:19:51.414386 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:51.414354 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-mbchd_d2635c4f-2a5e-4eda-b367-77094e7f9573/migrator/0.log" Jul 08 19:19:51.436873 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:51.436840 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-mbchd_d2635c4f-2a5e-4eda-b367-77094e7f9573/graceful-termination/0.log" Jul 08 19:19:53.152659 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:53.152633 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qw6cl_abf012dc-98c1-40d7-aca4-8e9b5ccccea6/kube-multus-additional-cni-plugins/0.log" Jul 08 19:19:53.174997 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:53.174971 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qw6cl_abf012dc-98c1-40d7-aca4-8e9b5ccccea6/egress-router-binary-copy/0.log" Jul 08 19:19:53.195636 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:53.195608 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qw6cl_abf012dc-98c1-40d7-aca4-8e9b5ccccea6/cni-plugins/0.log" Jul 08 19:19:53.216386 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:53.216358 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qw6cl_abf012dc-98c1-40d7-aca4-8e9b5ccccea6/bond-cni-plugin/0.log" Jul 08 19:19:53.238592 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:53.238567 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qw6cl_abf012dc-98c1-40d7-aca4-8e9b5ccccea6/routeoverride-cni/0.log" Jul 08 19:19:53.262220 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:53.262193 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qw6cl_abf012dc-98c1-40d7-aca4-8e9b5ccccea6/whereabouts-cni-bincopy/0.log" Jul 08 19:19:53.284878 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:53.284852 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qw6cl_abf012dc-98c1-40d7-aca4-8e9b5ccccea6/whereabouts-cni/0.log" Jul 08 19:19:53.321432 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:53.321404 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-kqlfn_b958669d-dcaf-40b5-8b8d-36ff1ce96694/kube-multus/0.log" Jul 08 19:19:53.423984 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:53.423878 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-dc669_14cb8ce5-3cb8-4827-8034-d74d7a8d75bf/network-metrics-daemon/0.log" Jul 08 19:19:53.447143 ip-10-0-143-154 kubenswrapper[2559]: I0708 19:19:53.447118 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-dc669_14cb8ce5-3cb8-4827-8034-d74d7a8d75bf/kube-rbac-proxy/0.log"